JP4671628B2 - Robot apparatus control system and control method - Google Patents

Description

  The present invention relates to a control system and control method for a robot apparatus that controls the operation or action of a robot apparatus having a plurality of movable parts, and more particularly, a robot that simultaneously controls the operation or action of a plurality of robot apparatuses in parallel. The present invention relates to an apparatus control system and a control method.

  More specifically, the present invention relates to a control system and a control method for a robot apparatus that collectively control a demonstration using a plurality of robot apparatuses and other cooperative or synchronous cooperative actions. The present invention relates to a control system and a control method for a robot apparatus that perform robust control against interference, falling, and failure between robot apparatuses when performing cooperative actions in a specific space.

  A mechanical device that uses an electrical or magnetic action to perform a movement resembling human movement is called a “robot”. It is said that the word “robot” comes from the Slavic word “ROBOTA (slave machine)”. In Japan, robots started to spread from the end of the 1960s, but many of them are industrial robots such as manipulators and transfer robots for the purpose of automating and unmanned production operations in factories. Met.

  In recent years, we have modeled the body mechanism and movement of a four-legged animal, such as a dog or cat, that mimics the body mechanism and movement of a four-legged animal, or the biped upright walking movement of a human or monkey. Research and development related to the structure of legged mobile robots such as “humanoid” or “humanoid robots” and stable walking control thereof have progressed, and expectations for practical use are also increasing. These legged mobile robots are unstable compared to crawler robots, making posture control and walking control difficult, but they are superior in that they can realize flexible walking and running operations such as climbing stairs and climbing obstacles. Yes.

  A legged mobile robot that reproduces a human biological mechanism and operation is particularly called a “humanoid” or “humanoid robot”. The humanoid robot can provide, for example, life support, that is, support for human activities in various situations in daily life such as a living environment.

  A legged mobile robot has a high information processing capability, and the robot itself can be regarded as a kind of computer system. In other words, a sophisticated and complex series of motions or motions composed of motion patterns realized on a robot or a combination of a plurality of basic motion patterns, that is, created or edited by operations similar to computer programming. Is done.

  It is indispensable for the robot body to spread that a lot of motion data for operating the actual machine is spread. Therefore, it is strongly desired to establish a development environment for performing motion editing for robots.

  In the future, it is expected that the robot will deeply penetrate not only the industry but also general households and daily life. In particular, for products pursuing entertainment, it is highly desirable that choreographers and designers can create motion content without having advanced knowledge of motion control in addition to computers and computer programming. In addition, it is expected that there are many cases where general consumers purchase and use robots. For such general users, it is preferable to provide a tool for supporting creation and editing of a robot motion sequence relatively easily and efficiently by interactive processing, that is, a motion editing system.

  For example, a motion editing apparatus that edits motion by interpolating motion between poses in accordance with the pose of the machine body input from an operator has been proposed (see, for example, Patent Document 1). According to this motion editing device, after editing a motion, whether or not the motion can be operated while stabilizing the posture on the actual machine is verified based on the ZMP stability determination criterion, and the posture can be stabilized. It is possible to make corrections to various operation patterns.

  In addition, with the advancement of autonomy of robotic devices, it is expected that there will be an era in which a large number of robots exist around us at a relatively high density, like humans. In such a case, a system capable of comprehensively controlling the operations or actions of a plurality of robot apparatuses is required.

  For example, a group robot system that controls a plurality of robots collectively as a group has been proposed in order to eliminate the complexity of operations associated with an increase in the number of robots to be operated (see Non-Patent Document 1, for example). ) The group robot system can control a plurality of agents collectively. A group can be formed by the interaction force between agent solids, and further, by changing the radius of the field of view of the agent, the balance of the interaction force can be intentionally disrupted and the agent can move in the group. is there.

  Moreover, each individual of the organisms that make up the group acts based on only local information, and uses the phenomenon that the overall coordination is generated from the interaction, and the minimum necessary without global information exchange. A group action by a plurality of robots can be performed only by the function (for example, see Non-Patent Document 2). In other words, by simulating the group formation of organisms that make group behaviors, and by reproducing the habits by algorithms, all robots are controlled by the same program, moved in the same direction as a group, and there is no leader Regardless, you can take orderly action.

  Also, as a representative method of cooperative transport by a plurality of robots, a leader-follower algorithm is known in which one robot serves as a demanded leader, and other robots follow it. When this technique is applied to obstacle avoidance in an unknown environment, avoidance can be realized by setting a target position at a point where the robot that has found the obstacle moves away from the obstacle (see Non-Patent Document 3, for example). See

  Here, when a plurality of humanoid robots move in cooperation with holding an object, there is a problem that the cooperative operation is disturbed due to vibration of the trunk during walking. It is conceivable to perform learning to acquire an action for correcting a positional shift due to the disturbance of the movement (for example, see Non-Patent Document 4). For example, methods such as Q learning, which is one of reinforcement learning, and a classification system using a genetic algorithm can be used.

  However, the present inventors consider that research and development related to a multi-robot system in which a plurality of robots operate cooperatively to achieve a specific purpose as a whole is still insufficient. For example, a single purpose or performance must be realized while sharing a role on a work space where a plurality of robots are limited, such as a stage with a performance or a demonstration. In this case, each robot provides a performance with story characteristics as a whole for multiple robots while executing harmony with surrounding robots while executing actions (operation commands or control programs) according to the respective casts. There must be.

  Under such a multi-robot environment, if sufficient synchronization is not ensured between the robot apparatuses, it does not appear to be synchronized and coordinated from the human eye. In addition, when multiple robot devices operate at a relatively high density in a narrow work space, interference may occur between the robot devices during the demonstration, the mutual positional relationship may deviate from the plan, or a fall or failure may occur. obtain. The results of the technology related to the multi-robot system that realizes robust control against such unexpected situations are rare at the time of this application.

JP 2003-266347 A Co-authored by Takashima Satoshi, Makino Koji and Matsuo Yoshiki “Generation of Intra-Group Behavior of Autonomous Mobile Robots Using Omnidirectional Vision Sensors” (21st Robotics Society Conference (September 20-22, 2003) 2B26) Co-authored by Hideaki Takanobu, Tetsuhiro Aiko, Hirokatsu Goto, Yoichi Mizukoshi, and Hirofumi Miura “Study on swarm intelligence with small mobile robots” (21st Robotics Society Conference (September 20-22, 2003) 2B22) Joint work by Kentaro Kato, Kenji Inoue, Takeo Arai and Yasushi Mae “Coordinated transport of single objects by multiple robots with different target positions-transport with obstacle avoidance” (The 19th Annual Conference of the Robotics Society of Japan (2001) 9 Monday 18th-20th) 3C34) Co-authored by Yutaka Inoue, Takahiro Mine, and Masashi Iba “Collaborative Luggage Transfer with Humanoid Robots—Learning for Position Correction” (21st Robotics Society Conference (September 20-22, 2003) 3B18)

  An object of the present invention is to provide an excellent control system and control method for a robot apparatus capable of simultaneously controlling operations or actions of a plurality of robot apparatuses in parallel.

  It is a further object of the present invention to provide an excellent control system and control method for a robot apparatus capable of comprehensively controlling a demonstration using a plurality of robot apparatuses and other cooperative or synchronous cooperative actions. It is in.

  A further object of the present invention is to provide an excellent robot apparatus capable of performing robust control against interference, fall, and failure between robot apparatuses when a plurality of robots perform cooperative actions in a specific space. A control system and a control method are provided.

The present invention has been made in consideration of the above problems, and a first aspect of the present invention is a control system for one or more robot apparatuses that operate on a specific operation field.
Monitoring means for monitoring the position and direction of each robot device on the motion field;
An overall control means for controlling the synchronous and cooperative operation of each robot apparatus based on the monitoring result of the monitoring means and the status information of the robot apparatus acquired from each robot apparatus, comprising communication means with each robot apparatus;
A control system for a robot apparatus, comprising:

  However, “system” here refers to a logical collection of a plurality of devices (or functional modules that realize specific functions), and each device or functional module is in a single housing. It does not matter whether or not.

  As robotic devices become more autonomous, it is anticipated that there will be an era in which a large number of robots will exist at a relatively high density, similar to humans. In such a case, a system capable of comprehensively controlling the operations or actions of a plurality of robot devices is required.

  According to the control system for a robot apparatus according to the present invention, it is possible to realize a specific purpose as a whole in a multi-robot environment in which a plurality of robots operate cooperatively. For example, a single purpose or performance can be realized while sharing a role on a work space where a plurality of robots are limited, such as a stage with a performance or a demonstration. In other words, each robot device performs a behavior (command operation or control program) according to each cast, and provides the audience with performance with story characteristics as a whole, while harmonizing with surrounding robots. be able to.

  In the control system for a robot apparatus according to the present invention, the monitoring means includes an image input means for capturing the motion field, and an image processing means for processing the input image to acquire information on the position and direction of each robot apparatus. I have. The overall control means performs a plurality of operations performed on the operation field (stage, etc.) based on the position and direction of each robot apparatus obtained based on the image processing result and the state information obtained from the robot apparatus by real-time communication. It is possible to control the realization of the performance by the robot device in a centralized manner.

  In such multi-robot performance, the role of each robot apparatus is determined. The operation plan of the robot apparatus for each casting is configured by arranging one or more instruction operations for the corresponding robot apparatus on the time axis. The overall control means manages execution of the operation plan of the robot apparatus determined for each casting on the same time axis, and manages a set of operation plans for each casting as a scenario.

  The overall control unit can manage two or more scenarios on a time axis and manage them as one performance by the multi-robot. Furthermore, a preliminary scenario that is executed as an alternative may be prepared in case it is difficult or impossible to continue due to the occurrence of an abnormal situation in each scenario.

  Here, the overall control unit can use a preliminary scenario in order to skip one or more subsequent scenarios that have become unexecutable and return to the original scenario connection. Alternatively, in the event of an abnormal situation where the performance cannot be completed, a preliminary scenario can be used to complete the performance instead of all subsequent scenarios that become unexecutable. Alternatively, a preliminary scenario corresponding to a scene where an abnormal situation has occurred in one scenario can be used.

  Under such a multi-robot environment, if sufficient synchronization is not ensured between the robot apparatuses, it does not appear to be synchronized and coordinated from the human eye. Therefore, the overall control means includes synchronization acquisition means for acquiring time synchronization with each robot apparatus on the operation field, and each casting robot apparatus executes each operation plan synchronously and coordinated according to the time. I try to control it.

  In addition, when multiple robot devices operate at a relatively high density in a narrow work space, interference may occur between the robot devices during the demonstration, the mutual positional relationship may deviate from the plan, or a fall or failure may occur. obtain. Therefore, the overall control means displays the position, direction, and state of each robot apparatus demonstrating performance on the operation field so that the operator can easily visually confirm the performance execution status.

  For example, the overall control means includes current state display means for displaying the position and direction of each robot apparatus on the operation field on the screen based on information on the position and direction of each robot apparatus obtained by the monitoring means. I have. The current state display means presents a deviation between an actual position and direction obtained by the monitoring means and a target position and direction on the operation plan of each robot apparatus. By presenting each target position and actual position and direction of each robot, the operator visually confirms the target position and current position and orientation via the screen, and accurately determines the current multi-robot state. be able to.

  Further, the overall control means includes a fall area calculation means for calculating a fall area of each robot apparatus on the motion field, a maximum movable range calculation means for calculating a maximum movable range of each robot apparatus, and each of the calculated Interference area display means for displaying on the screen the overlapping area of the robot apparatus or the overlap of the maximum movable range as an interference area between the robot apparatuses is provided.

  In this way, the interference area between the robots is visually displayed in the form of overlapping states of the respective maximum movable ranges, so that the operator can easily understand the performance demonstration problem. Moreover, it is also possible to impose a limit on the fall direction so as not to enter the movable range of the surrounding robot using such information on the interference area.

  For example, when each robot apparatus has one or more overturning operations, the overall control unit prohibits the use of the overturning operation having the interference region in the overturning region or having the interference region in the overturning direction. May be.

  Further, the overall control means calculates the motion characteristics of the robot based on the motion of the robot apparatus performed in the past, and the deviation from the planned value when the future motion plan is executed is used as the motion characteristics of the robot apparatus. Deviation prediction means for predicting based on, and deviation prediction display means for displaying the predicted deviation on the screen. For example, when there is a scene that must move to the place where the microphone is placed in a certain motion plan, or a scene that must move to the place where the microphone may fall at the end of the motion field In such a case, the operator can use the deviation estimation to determine the content to be treated before a serious trouble occurs.

Dividing Here, the deviation prediction means is configured to observe the error caused when running the motion in the past predetermined period t _s ~t _m1, motion executed in the past predetermined period t _s ~t _m1 per element movement Then, by estimating the error that occurs for each element motion, and further dividing the motion to be executed during the period from t _{m2 to} t _e into element motions, and summing the estimation errors of these element motions, Deviations from the target values of the position and direction at t _e can be estimated.

  The overall control means includes a trajectory display means for displaying a trajectory on the operation field of the robot apparatus. The trajectory display means may display a trajectory of a step with the Z axis as a time axis. In general, the work area of the robot does not change frequently in the plane, that is, in the Z-axis direction, and the Z-axis is less important for spatially displaying the steps. Therefore, by drawing the robot trajectory by designating the Z axis as the time axis, the movement to the XY axis and the trajectory of the posture (roll, pitch, yaw) can be easily visually confirmed over time. ing.

  The current state display means includes position / direction correction means for correcting a deviation between a target position and direction in an operation plan of each robot apparatus and an actual position and direction obtained by the monitoring means. The deviation of the position and direction from the target value of the robot apparatus affects the appearance given to the audience and also affects the performance itself such as interference between the robot apparatuses. This is because the robot will stand in a place where it should not be, and will interfere or collide with other robots that should be in the same place. Therefore, in the present invention, when the positional deviation between the current position and the target position of any robot is large, or when the positional deviation between the position estimated in the future and the target position is large, the position and direction correction processing is performed. It started up.

  The position / direction correction means is a “correction plan” in which an error in the position or direction of the robot device that is executing the motion plan is corrected using an existing step in the motion plan, or the robot device executes the motion plan. Using a scene that has not been made, it is possible to create a “movement plan” in which one or more new steps are generated to correct the positional deviation.

  Further, the overall control means defines an abnormal state according to the degree of abnormality in a multi-robot environment in which a plurality of robots execute a scenario in the operation field, and performs appropriate repair processing when falling into each abnormal state An abnormality processing means is provided. That is, according to the present invention, means for performing appropriate processing in consideration of the overall configuration, such as whether or not a robot in an abnormal state can be restored, and whether or not other robot operations interfere with the restoration operation, is established. ing. The overall control unit can detect such an abnormal state from the position / direction information, the robot state, and the like, and when the abnormal state is detected, can execute processing corresponding to the state.

  The abnormality processing means executes abnormality processing for the robot apparatus in a first abnormality state where the abnormality processing is completed with only one robot apparatus.

  In addition, the abnormality processing means waits in or out of the operation field, in the second state where the abnormality processing is required for the entire multi-robot, in the casting state change between the robot devices operating in the operation field. A spare robot is introduced into the operation field, or the formation in the multi-robot is corrected.

  The abnormality processing means corrects the entire formation in the multi-robot in a third abnormal state where a part of the scenario cannot be continued.

  In addition, the abnormality processing unit stops the continuation of the scenario in a fourth abnormal state where the entire scenario cannot be continued.

  ADVANTAGE OF THE INVENTION According to this invention, the control system and control method of the outstanding robot apparatus which can control simultaneously the operation | movement or action of a several robot apparatus simultaneously can be provided.

  In addition, according to the present invention, there is provided an excellent control system and control method for a robot apparatus capable of comprehensively controlling a demonstration using a plurality of robot apparatuses and other cooperative or synchronous cooperative actions. be able to.

  Further, according to the present invention, an excellent robot apparatus capable of performing robust control against interference, fall, and failure between the robot apparatuses when a plurality of robots perform cooperative actions in a specific space. The control system and the control method can be provided.

  Other objects, features, and advantages of the present invention will become apparent from more detailed description based on embodiments of the present invention described later and the accompanying drawings.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  The present invention relates to a multi-robot system in which a plurality of robots operate cooperatively to realize a specific purpose as a whole. The multi-robot system according to the present invention is particularly intended for legged mobile robots that have left and right movable legs and perform walking motions. One or more legged mobile robots are used in an operation field composed of a stage and other work areas. Realize demonstration operation.

  In conventional group robot systems, it is not possible to automatically deal with troubles such as slipping on the road surface or the state of individual robot devices during operation, such as not reaching the target position at a predetermined time, falling over, or malfunctioning. I could not but rely on the knowledge, experience, and manual work of the operator or technical supporter who operates the robot. On the other hand, according to the present invention, instead of relying on manual operation, appropriate and adaptive processing is automatically performed for trouble during operation, and trouble is avoided in advance or an operator is prompted to avoid the trouble. be able to.

A. Configuration of Multi-Robot System FIG. 1 shows an outline of a multi-robot system according to an embodiment of the present invention. As shown in the figure, the multi-robot system includes one or more robots that can be active on a motion field such as a stage or a work area, a monitor system that monitors the motion of these robots on the motion field, and these robots. It is composed of a general controller that centrally manages and controls the operation of the robot, and performs theater or demonstration operations using multi-robots.

  The target robot apparatus in this embodiment is a legged mobile robot that includes left and right movable legs and performs a walking motion, and the configuration thereof will be described later.

  Each robot has a predetermined role in advance, and is given a motion plan according to the role of the leading role or the supporting role. The operation plan is configured by combining instruction operations including reproduction motions on the time axis. A set of motion plans performed by all robot devices on the stage constitutes one scenario such as a play or demonstration. Details of the operation plan and scenario will be described later. An operation plan corresponding to the cast may be installed in advance for each robot on the stage, or the instruction operation may be transferred in real time each time performance such as performance or demonstration progresses.

  When executing the motion plan determined according to the casting, the motion performance or motion characteristics required of the robot device varies (for example, the role of the lower limb, the role of the upper limb, the role of the upper limb, or a specific stationary posture). Center role). On the other hand, there are individual differences in the robot apparatus, and the movement performance and movement characteristics possessed are different for each apparatus. Therefore, there is also a need for a casting arrangement system that assigns roles to the aircraft by combining the motion performance required for each casting with the motion performance of each available robot apparatus.

  One or more of the plurality of robots constituting the multi-robot may be assigned to “reserved robots” (shown) as spares that stand by in preparation for a failure of a robot apparatus that has been cast on the stage. .

  The monitor system acquires the position and direction of each robot by, for example, performing image processing on a video captured by a camera having a panoramic view of an operation field such as a stage, and notifies this to the overall controller. FIG. 2 shows a side view of the camera capturing the multi-robot on the stage from above. FIG. 3 shows a bird's-eye view of the stage under a multi-robot environment captured from a camera above the stage. Of course, the monitor system may measure the position and direction of the robot using a sensor system other than the camera.

  Based on information such as the position and direction of each robot obtained from the monitor system, the overall controller controls the execution of the scenario while comprehensively managing the robot's operation, thereby enabling the performance or demonstration of multi-robots. Realize operation.

  In addition, each robot with the general controller is connected by wireless communication such as a wireless LAN (Local Area Network) such as IEEE 802.11, and the general controller acquires the state of the robot executing the scenario in real time. Can do. Then, the general controller, based on the position and direction of the robot obtained from the monitor system and the state obtained from the robot, etc., instructs the robot to operate, instructs the robot to correct the position and direction, and generates an abnormality (or The operation instruction to the robot at the time of prediction is performed in real time via the wireless LAN.

  In the illustrated example, the overall controller and the monitor system are depicted as separate devices, but it is of course possible to implement them as individual applications that operate on a single computer system.

B. In the multi-robot system for instructing the robot, each robot is assigned a role in advance, and a motion plan according to the cast such as the main role or the side role is given. The operation plan is configured by combining instruction operations including reproduction motions on the time axis. A set of motion plans performed by all robot devices on the stage constitutes one scenario such as a play or demonstration.

  The instruction motion given to the robot is roughly classified into voice, light emission, and body motion (external output realized by motion of the body such as walking, dancing, posture transition, and emotion expression). In addition, a plurality of instruction operations such as uttering while walking may be combined to form one instruction operation (see FIG. 4).

  The machine operation can be edited using, for example, the motion editing apparatus described in Japanese Patent Laid-Open No. 2003-266347 already assigned to the present applicant. According to this motion editing device, after editing a motion, whether or not the motion can be operated while stabilizing the posture on the actual machine is verified based on the ZMP stability determination criterion, and the posture can be stabilized. It is possible to make corrections to various operation patterns.

  By combining these instruction operations on the time axis, one operation plan is constructed (see FIG. 5). In a multi-robot system in which multiple robots operate cooperatively to achieve a specific purpose as a whole, each robot is assigned a role such as a leading role or a supporting role. An action plan is given. That is, the set of instruction actions constituting the action plan is different for each casting.

  In a multi-robot system, each robot to which a role is assigned executes each motion plan synchronously and cooperatively on the same time axis, thereby realizing one performance such as a play or demonstration according to a scenario. For this reason, the motion plan includes the “start time in the scenario” corresponding to the start timing of the motion plan, and the start position and direction (standing position) of the robot on the motion field, in addition to the instruction motion of the robot alone. A corresponding format (see FIG. 6) with a corresponding “starting position direction in the scenario” is used for the scenario.

  A scenario is a description of the performance of a multi-robot system, such as a play or demonstration, and consists of a set of action plans for all robots that have been given roles. FIG. 7 schematically shows the scenario structure. The role of each robot is determined by combining a plurality of motion plans for performing a role on the time axis. As illustrated, in the scenario, combinations of operation plans for each role are described. Each action plan describes the importance of the role (main role, supporting role, etc.).

  In a multi-robot system, in addition to performing only with a single scenario, it is also possible to connect and operate multiple scenarios. FIG. 8 shows a configuration example in which a plurality of scenarios are connected. In this case, the basic form is to connect a plurality of scenarios in series on the time axis.

  In addition, it is possible to prepare a preliminary scenario that is executed as an alternative in preparation for cases where it is difficult or impossible to continue due to the occurrence of an abnormal situation in each scenario. The preliminary scenario can be used to skip one or more subsequent scenarios that have become unexecutable and return to the original scenario connection. In the illustrated example, the preliminary scenario 1 includes an operation plan group for skipping the scenario 2 that has become unexecutable and joining the scenario 3 when an abnormality occurs during the execution of the scenario 1.

  Alternatively, the preliminary scenario can be used to complete the performance in place of all subsequent scenarios that become unexecutable when an abnormal situation occurs where the performance cannot be completed. In the example shown in the figure, in the case of the preliminary scenario 2 and the preliminary scenario 3, when an abnormal situation in which the performance cannot be completed during the execution of the scenario 3, the subsequent scenario 4 is completely cancelled, and the performance is finished safely. It can be used to achieve the initial operational purpose.

  Also, a plurality of alternative preliminary scenarios may be prepared in case a continuation is difficult or impossible due to an abnormal situation in a certain scenario. Furthermore, a plurality of alternative preliminary scenarios may be prepared according to the time when the abnormal state occurs. In the example shown in the figure, scenario 4 is prepared as spare scenario 4 and spare scenario 5. When an abnormality occurs in the first half of the scenario, spare scenario 4 is used, and when an abnormality occurs in the latter half, spare scenario 5 is used. By using each, it looks as if the performance seemed to have finished successfully, or it is used to achieve the initial operation purpose.

C. Time management of each robot in the multi-robot system As described above, in the multi-robot system, each robot to which a role is assigned executes an assigned motion plan on the motion field such as the stage. It is possible to behave in a coordinated and coordinated manner and present performance according to the scenario.

  Here, under the multi-robot environment, there is a problem of time synchronization that, if the synchronization between the robot apparatuses is not sufficiently ensured, it does not appear to be synchronized and coordinated from the human eyes. When the robots are operated in synchronism with each other, and in particular, the same operation is performed, the faster the movement, the more noticeable the difference in the apparent operation.

  In addition, the voice and music of the robot may be output from a line output from a device outside the robot instead of from a speaker built in the robot. This is because the venue is large and there are cases where the voice and music of the robot does not reach the audience of the venue, or there is a case where it is desired to use high-quality sound. In order to synchronize the operation of the robot and the sound from the external device, the time counted by these devices needs to be synchronized with the above accuracy.

  The problem of time synchronization affects not only the appearance effect but also performance situations such as interference between robot devices. This is because the synchronization is not sufficiently ensured, so that the robot stands in a place where it should not be, and interferes or collides with other robots that should originally be in the same place.

  A method is also conceivable in which the general controller synchronizes by transferring a command related to the instruction operation in accordance with the execution timing of the operation plan. That is, a command is sent to the robot via a wireless LAN after an arbitrary time or elapsed time using a clock or timer in the PC. In this case, the time is not shared between the robots or between the robot and the PC, and the robot executes it immediately upon receiving the command. For example, when attempting to perform robot motion playback and music playback based on scenarios such as musicals and plays using multiple robots, commands are sent from the general controller to each robot in parallel at the time at which they want to operate. To synchronize. However, in this case, there is a problem that the arrival delay of the command occurs due to the radio wave condition of the radio used, and the robots do not operate synchronously.

  Therefore, in this embodiment, time synchronization is performed in advance between the robots or between the robot and the general controller (or other device that manages time), and the robot is operated when the time comes.

  FIG. 9 illustrates a mechanism for synchronizing time between robots and between a robot and a general controller. In the example shown in the figure, the system running the time management system (general controller) and the application software that controls the wired cable and cable output are used to communicate the time on the operating system of the general controller with each robot. Set.

  FIG. 10 illustrates another example of a mechanism for synchronizing time between robots and between a robot and a general controller. In the example shown in the figure, the time on the operating system of the general controller using the system (overall controller) operating the time management system, the wireless communication device connected to the system, and the application software for controlling the wireless communication device. Is transmitted wirelessly, and the time is set by each robot receiving this wireless signal.

D. The controller / application controller controls the multi-robots so that the scenario progresses smoothly by comprehensively managing the robot operations based on information such as the position and direction of each robot obtained from the monitor system. operate. In addition, the position of each robot on the operation field, the maximum movable range and fall range of each robot, and the interference area between the robots are displayed via the GUI screen of the general controller, and the operator can visually confirm these. Can do.

  FIG. 11 shows a configuration example of a multi-robot operation GUI screen provided by the overall controller.

  The main view of the GUI screen shown in the figure can be switched by selecting a tab between the display of the current state of the multi-robot on the operation field, the display of the interference area between the robots, the display of the deviation prediction, and the trajectory display. It has become.

  In the example shown in FIG. 11, the main view displays the current state of the multi-robot. In the illustrated example, there are four robots named Chalie, Audrey, Ken, and Marco on the motion plan in the motion field. The current state display is presented in two ways, for example, a two-dimensional display by a bird's eye view and a perspective three-dimensional display, but may be displayed in other fields of view. In the illustrated example, in the bird's eye view, the target position and the actual position and direction of each robot are presented by these footprints, so the operator visually indicates the target position and current position and orientation via the screen. It is possible to confirm and accurately determine the current multi-robot state.

  Further, below the main view, the position information of each robot is presented numerically together with the two-dimensional and three-dimensional display of the position information regarding the multi-robot. In the example shown in FIG. 11, as a form of numerical display of position information, display of position deviation, target position, current position, state, and operation plan is switched by selecting a tab. In the example shown in the figure, the position deviation is selected, and the deviations between the target position and the current position and direction for the six axes X, Y, Z and roll, pitch, and yaw of each of the four robots are displayed numerically. ing.

  The current position and direction of each robot can be obtained by the monitor system processing the captured image of the camera. When the position deviation is large, the position and direction correction processing can be started, but this point will be described later.

Interference area:
When the interference area tab is selected in the main view, the screen is switched to an interference area confirmation screen as shown in FIG. 12, and the operator can visually confirm the safe distance between the robots. In the figure, four large circles represent places where each robot can be used for falling. An ellipse included in each circle represents the maximum movable range of the robot in operation.

  As the maximum movable range, as a more detailed information, a range in which an area used by the robot is previously introduced from an operation plan assigned to the robot can be used.

  In this way, the interference area between the robots is visually displayed in the form of the overlapping state of the respective maximum movable ranges. Moreover, it is also possible to impose a limit on the fall direction so as not to enter the movable range of the surrounding robot using such information on the interference area.

  FIG. 13 shows a processing procedure for imposing a restriction on the direction in which the robot falls over in the form of a flowchart.

  First, an interference area between the robot devices is calculated (step S1), and a fall area corresponding to the fall operation of each robot is calculated (step S2). For example, a fall region of a fall operation that forcibly falls backward even when the fall direction is in front occurs behind the robot.

  Next, the overlapping area of the interference area and the falling area is calculated between the robot devices (step S3), and it is determined whether there is an overlapping area (step S4).

  Here, if there is an overlap area, a fall area group including the overlap area is calculated (step S5), and among the fall area groups included in each robot, the fall operation corresponding to the fall area group including the overlap area is performed. Is disabled (step S6).

  On the other hand, when there is no overlapping area between the robot devices (step S4), the overturning operation group that has been set to be prohibited for use in each robot device is enabled (step S7).

When the process is continued (step S8), the process returns to the top of the process routine, and when the process can be completed, the entire process routine is terminated.
The above example is about the restriction of the fall direction in consideration of the fall overlap area when the fall between the robot devices, but the fall direction restriction is when the robot device is at the end of the operation field or there are obstacles other than the robot device Cases are also considered.

Deviation prediction:
When the deviation prediction tab is selected in the main view, the screen is switched to a deviation estimation screen showing the deviation from the target position of each robot at a specific time as shown in FIG. While operating the scenario, the operator can estimate the positional deviation at a specific time and confirm the estimated deviation on the screen. For example, when there is a scene that must move to the place where the microphone is placed in a certain motion plan, or a scene that must move to the place where the microphone may fall at the end of the motion field In such a case, the operator can use the deviation estimation to determine the content to be treated before a serious trouble occurs.

  In this embodiment, the deviation in the position direction of the robot is predicted based on the result of the motion executed in the past. Based on the movement of the robot performed in the past, the motion characteristic of the robot can be calculated. Then, an error from a planned value when a motion plan to be performed in the future is executed is predicted based on the motion characteristics of the robot.

More specifically, while observing the error that occurred when the motion was executed in the past fixed period t _{s to} t _m1 , the motion executed in the past fixed period t _{s to} t _m1 is divided for each element motion, Estimate the error that occurs for each element motion. The element motion here refers to a motion divided for each time axis or part motion. Then, the motion to be executed in the period from t _{m2 to} t _e is divided into element motions, and the error of position and direction at time t _e is estimated by summing the estimation errors of these element motions. be able to.

  FIG. 15 shows a processing procedure for estimating the deviation in the position direction in the form of a flowchart.

First, it sets the time _{t s, t m1, t m2} , t e ( step S11). Usually, t _m1 = t _m2 .

Then, the movement error and the direction error caused by the element motion j of the l-th robot executed between times t _s and t _m1 are calculated (step S12). The movement error and the direction error are expressed as follows.

Further, the target motion sequence scheduled to be executed between times t _m2 and t _e is decomposed into the number n _j of element motions j (step S13).

Further, a movement error and a direction error (initial error) at time t _m2 are acquired (step S14). The movement error and the direction error are expressed as follows.

Next, an estimated value of a movement error and a direction error generated when the target motion sequence scheduled to be executed between times t _m2 and t _e is executed using the l-th robot is calculated (step S15). The movement error and the direction error are expressed as follows.

Then, to calculate the estimated value of the movement error and the direction error of the l-th robot at time t _e (step S16). The movement error and the direction error are expressed as follows.

  When the deviation in the position direction is large, the position and direction correction processing can be started, but this point will be described later.

Orbit display:
Selecting the trajectory display tab in the main view switches to a screen that displays the trajectory of each robot. A simple example of the trajectory display is that the robot's past, present, and future steps, that is, footprints (footprints) are arranged at corresponding locations on a bird's eye view or a perspective motion field road surface. In a legged mobile robot, since an action plan can be made based on walking, the display of the trajectory using the walking in this way is extremely effective. At this time, color-coded display may be performed according to the time axis (for example, according to past or future), or footprints may be displayed in different colors for each robot.

  Here, when displaying the trajectory of a step on a 3D screen, the display of the step is concentrated in a specific place in a scene where there are many steps or the same place is visited many times like dance. Therefore, there is a problem that it becomes difficult to confirm the trajectory of the step with the passage of time on the screen display.

  Therefore, in this embodiment, the trajectory display of the step with the Z axis as the time axis is performed. The work area of a legged mobile robot generally has a low frequency of change in the plane, that is, in the Z-axis direction, and the Z-axis is less important for spatially displaying steps. Therefore, the Z axis is designated as the time axis, and the movement to the XY axis and the trajectory of the posture (roll, pitch, yaw) can be visually confirmed as time passes.

  FIG. 16 shows an example of 3D display of the trajectory of a step with the Z axis as the time axis. In the example shown in the figure, the Z-axis positive direction is set as the time axis, and a plurality of steps are stacked and displayed in the Z-axis direction at the place where the step is performed. Of course, the time axis may be assigned to the negative direction instead of the Z-axis positive direction.

  Although the frequency of the work area of the legged mobile robot changing in the Z-axis direction is low, it is not completely absent. For example, raising and lowering stairs. FIG. 17 shows a 3D display example of a normal step when the robot moves up and down the stairs. On the other hand, FIG. 18 shows a frontal plane, a sagittal plane, and a perspective view in which the time axis is set in the negative direction of the Z axis and the steps when the stairs are raised and lowered are displayed.

  In the operation of scenarios, it is very important to grasp the position of each robot as time passes, and the operator can judge the situation by combining interference area confirmation and deviation display. You will be able to do it more accurately.

E. Monitoring of the position and direction of each robot in the multi-robot system, state monitoring As described above, in the multi-robot system, each robot to which a role is assigned executes its assigned motion plan in a synchronized manner. It is possible to present the performance according to the scenario by behaving in a coordinated action on the operation field.

  Each robot is assigned with each combination, and the start time and the start position direction corresponding to the combination are described in the scenario (see the above and FIG. 6). However, during the operation of the scenario, the target position has not been reached at the predetermined time or the target direction has been reached due to external factors such as road surface conditions such as slipping and tilting, and internal factors such as the state of individual robot devices. It can happen. If such a current state is accumulated during a series of operations, the planned operation cannot be performed, and this becomes manifest as a shift in the position and direction of the multi-robot.

  Therefore, in this embodiment, the monitor system obtains the position and direction of each robot by performing image processing on the video captured by the camera that can overlook the operation field, and the overall controller obtains each robot obtained from the monitor system. Based on information such as position and direction, the position, direction, and state of each robot that changes every moment within the motion field are grasped.

  A ceiling camera is installed above the operation field such as the stage, and the state of the multi-robot executing the scenario can be obtained as a bird's eye view. The monitor system can recognize the image captured by the ceiling camera and identify the position and direction of each robot that changes from time to time (see FIG. 19). Then, the overall controller can acquire information on the position and direction of each robot at a certain time from the monitor system (see FIG. 20).

  FIG. 21 shows, in the form of a flowchart, a processing procedure for the general controller to acquire information on the position and direction of the robot at a certain time from the monitor system. In the illustrated example, the acquisition of the position information is performed on a request basis from the overall controller. That is, when the general controller issues a monitor system data acquisition request, the monitor system acquires time, position, and direction data and transfers them to the general controller.

  FIG. 22 is a flowchart showing a processing procedure for the monitor system to acquire the position and direction of the robot based on the image taken by the ceiling camera and transfer it to the general controller.

  When the monitor system acquires image data from the ceiling camera, it performs predetermined image processing to create position and direction data of each robot within the visual field or on the operation field. When there is a data request from the general controller, the created data is transferred.

  Regarding the detection of the robot state, the robot itself can be recognized by a sensor built in the robot. The state of the robot mentioned here includes current posture information (standing posture, sleeping posture), whether the operation is normal or abnormal, the remaining battery level, and the like. The overall controller is connected to each robot via a wireless LAN, and can acquire the state of the robot in real time.

F. Revision plan of each robot's position and direction in the multi-robot system In the multi-robot system, each robot to which a role is assigned executes the given motion plan on the motion field such as the stage. By performing coordinated and coordinated actions, it is possible to present performance according to the scenario.

  Each robot is assigned with each combination, and the start time and the start position direction corresponding to the combination are described in the scenario (see the above and FIG. 6). However, during the operation of the scenario, the target position has not been reached at the predetermined time or the target direction has been reached due to external factors such as road surface conditions such as slipping and tilting, and internal factors such as the state of individual robot devices. It can happen. If such a current state is accumulated during a series of operations, the planned operation cannot be performed, and this becomes manifest as a shift in the position and direction of the multi-robot.

  Such a shift in position and direction affects the appearance given to the audience and also affects the performance itself, such as interference between robot devices. This is because the robot will stand in a place where it should not be, and will interfere or collide with other robots that should be in the same place.

  Therefore, in this embodiment, the monitor system obtains the position and direction of each robot by performing image processing on the video captured by the camera that can overlook the operation field, and the overall controller obtains each robot obtained from the monitor system. The multi-robot is operated so that the scenario proceeds smoothly by comprehensively managing the operation of the robot based on information such as the position and direction.

  The general controller can check the deviation between the target position direction of the robot and the current position direction at any time by the controller application described above (see FIG. 11). Furthermore, when the position / direction deviation between the current position and the target position of any robot is large, or when the position / direction deviation between the position estimated in the future and the target position is large, the position and direction correction processing may be started. it can.

  In the position and direction correction plan according to the present embodiment, two types of position direction correction methods are prepared depending on the situation. One is a “correction plan”, in which a positional deviation is corrected by correcting an existing step in the motion plan while the robot is executing the motion plan. The other is a “movement plan”, which uses a scene where the robot is not executing a motion plan to generate one or more new steps and correct the positional deviation. .

  FIG. 23 shows the procedure of the position and direction correction plan in the form of a flowchart.

  Based on the data acquired from the monitor system, the general controller determines whether or not it is necessary to correct the position and direction of any robot active in the operation field (step S21).

  If it is determined that any of the robots needs to be corrected in position and direction, it is determined whether or not the robot is currently executing a scenario (step S22). Here, if the scenario is not being executed, it is determined to make a movement plan (step S25). Details of the processing procedure for making a movement plan will be described later.

  If the robot is executing a scenario, it is further determined whether the robot is executing an operation plan within the scenario, an interruption between operation plans, or a movement plan (step S23). . Here, when the robot is not executing either the motion plan or the movement plan, it is decided to make a movement plan (step S25).

  On the other hand, when the robot is executing either the motion plan or the movement plan, it is decided to make a correction plan (step S24). The correction plan is determined to make a correction plan that corrects the existing steps in the plan so that the currently executing plan ends successfully.

  When there is a plan to notify the robot (step S26), the robot is notified of the plan (step S27). If the processing routine can be ended, the processing routine ends. Otherwise, the processing routine returns to step S21 and the same processing as described above is repeated.

F-1. In the correction plan operation plan, the trajectory of the leg is managed with a “step” serving as the sole trajectory of the robot as one unit. FIG. 24 shows how the leg trajectory is managed by walking. A step at a certain time is represented by parameters (x, y, z, roll, pitch, yaw), and the coordinate origin is usually the start point of the motion plan. As long as the unit represents a space, the Euler angle space may not be used. The walking parameters shown in FIG. 24 are expressed as shown in the table below, for example.

単位
ｘ，ｙ，ｘ：［ｍ］
ｒｏｌｌ．ｐｉｔｃｈ．ｙａｗ：［ｄｅｇ］

Unit x, y, x: [m]
roll. pitch. yaw: [deg]

  In this example, the relative value with the origin of coordinates as the initial position and the start time as 0 is used, but an absolute value may be used for each coordinate and time.

Next, the step deviation will be described. The accumulated amount of positional direction deviation that occurs during the operation plan operation is expressed as (x ₁ , y ₁ , z ₁ , roll ₁ , pitch ₁ , yaw ₁ ). The purpose of position correction is basically to set each component of the position direction deviation at a certain time to 0, and correction amounts (x ₂ , y ₂ , z ₂ , roll ₂ , pitch ₂ , yaw calculated from the deviation). ₂ ) is added to the planned steps, and this is achieved by the robot executing it. Here, what is obtained by adding a correction amount to a planned step is referred to as a “correction step”. The corrected steps (X, Y, Z, ROLL, PITCH, YAW) can be expressed as follows:

  In this specification, a trajectory plan in which correction steps are combined so that there is no deviation in the motion plan is referred to as a “correction plan”.

  FIG. 25 shows deviations generated during operation plan operation with respect to the planned trajectory shown in FIG. In addition, the deviation of the illustrated steps is shown in the table below.

単位
ｘ，ｙ，ｘ：［ｍ］
ｒｏｌｌ．ｐｉｔｃｈ．ｙａｗ：［ｄｅｇ］

Unit x, y, x: [m]
roll. pitch. yaw: [deg]

The correction amount (x ₂ , y ₂ , z ₂ , roll ₂ , pitch ₂ , yaw ₂ ) is obtained from the deviation amount, and can be basically expressed as the following equation.

  However, in practice, the amount of correction is not limited to the operational restrictions imposed on the robot due to the characteristics of the robot actually used, the state of the stage (motion field), the leg joint angle, actuator speed, acceleration, and self-interference. It is necessary to calculate in consideration. For this reason, the actual correction amount is generally expressed as a function of the accumulated amount of time and position direction deviation occurring during operation of the motion plan, as shown in the following equation.

Here, the characteristics of the robot are investigated in advance under a reference environment. Since the average deviation amount for each step is quantified in the same way as other parameters (x ₃ , y ₃ , z ₃ , roll ₃ , pitch ₃ , yaw ₃ ), this value is subtracted from the deviation amount. The corrected value is the correction amount.

Also, the behavior of the robot depends on the state of the motion field. In particular, the deviation amount of the step is greatly influenced by the road surface environment in the operation field. When the operation plan is used, the deviation statistics of the steps from the start are taken and the average deviation amount (x ₄ , y ₄ , z ₄ , roll ₄ , pitch ₄ , yaw ₄ ) for each step is calculated. This value is the predicted amount of the deviation that will occur in the future, and the correction amount is calculated in consideration of this value in advance.

  The restrictions imposed on the robot include the restrictions imposed on the aircraft and operational restrictions. The trajectory of the leg joint is calculated using inverse kinematics or the like with respect to the trajectory of the step obtained by the above [Formula 5] and [Formula 7]. Furthermore, the speed / acceleration and self-interference for each joint are also examined (if possible in time) to determine whether the respective restrictions are satisfied. If the restrictions are not met, the correction amount is reduced according to a certain standard. In addition, with respect to the constraints of the robot body, stronger constraints may be imposed in consideration of appearance and safe operation.

  FIG. 26 shows a processing procedure for making a correction plan in the form of a flowchart.

First, the overall controller determines a correction deviation (x ₁ , y ₁ , z ₁ , roll ₁ , pitch) from the planned value in the direction of the position of the corresponding robot obtained from the scenario and the current value acquired from the monitor system. ₁ , yaw ₁ ) is calculated (step S31).

  Next, after the current time, a step to be corrected is selected from existing steps in the motion plan, and a provisional amount of correction is derived (step S32).

  Next, the provisional correction amount is corrected to a correction amount that takes into account the characteristics of the robot and the state of the motion field (step S33).

  Furthermore, the correction amount obtained in this way is updated to a strict correction amount considering the current robot posture and robot constraints (joint angle limitation, angular velocity limitation, angular acceleration limitation, self-interference, etc.) (step S34).

  Then, the correction amount is updated with respect to the correction amount considering the constraints of the robot based on the constraints considering appearance and safety (step S35).

The overall controller reflects the obtained correction amounts (x ₂ , y ₂ , z ₂ , roll ₂ , pitch ₂ , yaw ₂ ) on the correction steps (X, Y, Z, ROLL, PITCH, YAW) of the corresponding robot. (Step S36).

  FIG. 27 shows a detailed processing procedure in the form of a flowchart for deriving a provisional correction amount for the step to be corrected in step S32 described above.

  The overall controller refers to the operation plan of the robot subject to the position / direction correction processing from the scenario, and obtains the number of steps that can be used for correction after the current time (step S41).

  Next, one or more steps used in the correction plan are selected from the number of steps (step S42).

  Next, an approximate upper limit value of correction that can be estimated from the parameters (step length, turning angle, speed, etc.) of the step is obtained for the selected step (step S43).

  Then, based on the obtained information, the correction amount of the next step to be corrected is determined (step S44).

F-2. When the robot whose movement plan position direction is to be corrected is currently executing the motion plan or the movement plan, the correction is performed by the correction plan combining the correction steps as described above. On the other hand, when the robot that has to correct the position and direction has not made any motion plan, it creates a new step and creates a movement plan that corrects the position and direction before executing the next motion plan. Stand up (mentioned above). FIG. 28 shows a processing procedure for making a movement plan in the form of a flowchart.

  First, on the spot, a gait having a good impression (giving a good appearance or an impression suitable for the spot) is used, for example, by selecting from a gait library consisting of a set of different footsteps (step S51). Trajectory planning is performed according to the priority order of the winning combination (step S52).

  Here, the remaining energy of the battery for driving the robot apparatus is estimated (step S53).

  If it is estimated that the remaining energy is exhausted (step S54), it is searched whether there is another gait with higher efficiency rather than a good impression (step S58). If there is a step with higher efficiency, it is changed to that (step S59), and the process returns to step S52. On the other hand, if there is no more efficient gait, the movement plan is terminated.

  If it is estimated that there is remaining energy (step S54), the remaining time is further estimated (step S55).

  Next, it is determined whether or not the gait has been changed to a more efficient gait (step S56).

  If the gait has been changed to a more efficient gait, it is further determined whether or not the remaining time until the next motion plan starts is a certain time or more (step S57). If the remaining time is equal to or longer than a certain time, the movement plan is terminated. If the remaining time is less than a certain time, the movement plan is canceled, that is, the correction of the position direction until the next operation plan starts is given up.

  If the gait has not been changed to a more efficient gait, it is further determined whether or not the remaining time until the next motion plan starts is a certain time or more (step S60). If the remaining time is equal to or longer than a certain time, the movement plan is terminated. On the other hand, if the remaining time is equal to or shorter than the predetermined time, if there is a faster gait (step S61), the gait is changed to a faster gait (step S62), and the process returns to step S52.

G. When operating an abnormal processing scenario in a multi-robot system , a fall avoidance action may be triggered or fall down due to external factors such as road surface conditions / tilt, internal factors of the robot itself, or contact between robots. is there. Also, various abnormal states such as operation stop due to lack of energy such as a decrease in the remaining battery capacity are assumed.

  Conventionally, processing of an abnormal state requires the skill of an operator, and it has been very difficult to recover from the abnormality while predicting the operation of another robot.

  On the other hand, in the multi-robot system according to the present embodiment, it is appropriate to consider the entire configuration, such as whether or not the robot in an abnormal state can be restored, and whether or not other robot operations interfere with the return operation. Means for performing processing have been established. That is, the overall controller can detect such an abnormal state from position / direction information, a robot state, and the like. When an abnormal state is detected, processing corresponding to the situation is executed.

  In the multi-robot system according to the present embodiment, the following four levels of abnormal states are defined according to the degree of abnormality.

Abnormal level 1: State where abnormality processing is completed with only one robot Abnormal level 2: State where abnormality processing is necessary for the entire multi-robot system Abnormal level 3: State abnormality where part of the scenario cannot be continued Level 4: The entire scenario cannot be continued

  Hereinafter, each abnormality level will be described.

Abnormal level 1:
> Determine whether or not to continue the “motion plan” of a robot that is in an abnormal state Using a predetermined evaluation formula, determine whether or not the purpose of the “motion plan” of the robot can be achieved due to the occurrence of an abnormal state. Here, a method for determining the possibility of continuing the robot motion plan and its evaluation formula will be described.

  Each robot device that is given a role in the scenario has an operation plan such as a motion to be executed defined in the scenario. Motion can be divided into elemental motions (described above). Element movements are right turn, left turn, forward, backward, right side, left side, forward jump, reverse jump, right turn jump, left turn jump, forward drive, reverse drive, right turn, left turn. For example, traveling, right lateral traveling, left lateral traveling, etc. are the minimum units of the operation command.

Each elemental motion can be broken down into unit motions. Here, the concept of a performance vector is introduced in order to determine the feasibility of the motion for the element motion. the j-th element exercise unit exercise number n _j of, and the k-th role to plan performance unit element exercise the number of decomposed after time t of the robot that has been given, the performance vector P _{k (t)} is of the following formula It is expressed as follows.

On the other hand, _assuming that the achievement degree per unit motion of the element motion j of the l-th robot is a _jl , the 1-unit performance vector S _{l at} the time t of the robot l is expressed by the following equation.

  During re-pass planning, element movements that do not meet the tolerance are removed from the usable element movements. Then, an evaluation value for performing the operation continuation possibility is defined as in the following expression. If this value is equal to or larger than the allowable value ε, it is determined that the planned operation can be continued.

When there is an evaluation weight between each element motion, the weight diagonal matrix I _{k (t)} after time t is evaluated as a product of the performance vector P _{k (t)} ^{t or} the like as P _{k (t)} ^t. A final evaluation may be performed by calculating a value.

> Return from abnormal state As an example of abnormal level 1 abnormal state, consider a case where a robot falls. If the robot is in a state of being able to get up, the robot is instructed to get up. The determination of whether or not to get up is determined by whether or not the communication between the general controller and the robot has been established, there is no problem in the internal state of the robot that has fallen, and there is no problem of interference with other robots. The robot that gets up continues the “motion planning” while correcting the positional direction deviation.

Abnormal level 2:
> Casting change Casting is changed when a robot becomes abnormal, especially when the robot plays an important role such as the main role. For example, when the robot B performs the role of the robot A, the robot B is attached to the position of the robot A, and thereafter, the “operation plan” of the robot A is executed.

> Introduction of a spare robot When a robot is in an abnormal state and a spare robot is prepared, the robot moves into the motion field or is placed in the motion field by a person and the robot Execute “Operation Plan”. FIG. 29 shows a processing procedure for introducing the spare robot in the form of a flowchart.

  First, a moving trajectory to the introduction location is planned (step S71).

  Next, the movement trajectory is set as an “motion plan”, which is executed after the introduction of the spare robot, and merged with a series of “motion plans” in the scenario (step S72).

  Next, the merged series of “motion plans” is uploaded to the spare robot (step S73).

  Then, in consideration of the time progress of the scenario, the general controller starts the uploaded “operation plan” automatically or at the scene where it is desired to introduce a spare robot (step S74).

> Correcting a formation If a robot becomes in an abnormal state and interference with other robots is expected during the return operation, the formation can be partially corrected to increase the area for the abnormal robot to recover. Secure.

Abnormal level 3:
> Re-planning of the entire formation When a robot becomes abnormal and cannot move, it cannot be removed, resulting in an area where other robots cannot enter in the motion field, and in the future If it is predicted that interference will occur, the trajectory will be re-planned within the overall “motion plan”. In that case, a plan is made so that there is no change in the trajectory of the leading role as much as possible.

Abnormal level 4:
> Scenario stop If it is determined that the scenario is no longer valid due to the abnormal condition that occurred, the entire process is stopped. Each robot shifts from the operating state to the stopped state. After the scenario is stopped, one of the following four processes is executed depending on the situation (or at the operator's discretion) (see FIG. 8).

・ Start the current scenario from the beginning (return)
・ Skip the current scenario and start the following scenario (skip)
-Start preliminary scenario-Discard all scenarios

  FIG. 30 shows a processing procedure for determining an abnormal level in the form of a flowchart.

  First, it is determined whether or not the abnormal robot can be restored under the following conditions (step S81).

  If it is determined that the abnormal robot can be restored, it is then determined whether the robot can be restored without interfering with another robot (step S82).

  If the abnormal robot can return without interfering with other robots, it is further determined whether the role can be continued after the return (step S83). If the role can be continued, it is determined that the level is abnormal.

  On the other hand, if the robot in the abnormal state cannot continue the role after returning, it is determined whether or not the cast can be changed (step S84). If the cast can be retreated, it is determined as an abnormal level 2.

  If the abnormal robot cannot return without interfering with other robots (step S82), it is determined whether the return can be made by changing the entire formation (step S85). If it is determined that the recovery is possible, it is determined that the abnormality level is 2.

  On the other hand, if the entire formation cannot be restored (step S85), or if it is determined that the abnormal robot cannot be restored (step S81), the abnormal state is removed from the operation field (stage) or It is further determined whether or not it can be left (step S86).

  If the abnormal robot can be removed or left, it is determined whether or not the spare robot (described above) can be put out (step S87). When the spare robot can be put out, it is determined that the abnormality level is 2.

  If the spare robot cannot be taken out (step S87), it is determined whether or not the scenario can be continued even if the abnormal robot is removed or left (step S88). If the scenario can be continued, it is determined as abnormal level 3.

  If the robot that cannot recover from the abnormal state cannot be removed or left (step S86), it is determined whether or not the scenario can be continued with the robot remaining in the operation field (step S89).

  The abnormal level 3 is also determined when the scenario can be continued with the abnormal robot remaining in the operation field.

  On the other hand, when the scenario cannot be continued with the abnormal robot remaining in the operation field (step S89), and when the scenario cannot be continued by removing or leaving the abnormal robot (step S89). S88), it is determined that the abnormality level is 4.

Other processing to be performed in case of abnormality:
> Log data collection When a robot becomes abnormal, log data stored in the robot is collected. Analyzing the collected data helps to analyze the cause of the abnormality.

> Self-processing of a robot that cannot communicate with the controller The robot that has recognized that communication with the controller has been disabled determines that control from the outside has been disabled, and stops its operation as self-processing. Depending on the execution environment, processing such as erasure is performed so that highly important information such as “operation plan” is not leaked to the outside.

H. Configuration of the robot device until this has been described in terms of structure and function of a multi-robot system. In this section, a robot apparatus that can operate as a member of a multi-robot will be described in detail.

  FIG. 31 and FIG. 32 show a state in which the “humanoid” or “humanoid” robot apparatus 100 used for carrying out the present invention is viewed from the front and the rear. As shown in the figure, the robot apparatus 100 includes a trunk, a torso, a head, left and right upper limbs, and left and right lower limbs that perform legged movement. For example, the robot apparatus 100 is built in the trunk. The operation of the robot apparatus is comprehensively controlled by a control unit (not shown).

  Each of the left and right lower limbs is composed of a thigh, a knee joint, a shin part, an ankle, and a foot, and is connected to the lowermost part of the waist by a hip joint. The left and right upper limbs are composed of an upper arm, an elbow joint, and a forearm, and are connected to the left and right side edges above the trunk by shoulder joints. The head is connected to the substantially uppermost center of the trunk by a neck joint.

  The robot apparatus 100 configured as described above can realize bipedal walking by whole body cooperative operation control by a control unit (not shown in FIGS. 31 and 32). Such biped walking is generally performed by repeating a walking cycle divided into the following operation periods. That is,

(1) Single leg support period with left leg lifted right leg (2) Both leg support period with right leg grounded (3) Single leg support period with right leg lifted with left leg (4) Both legs with left leg grounded Support period

  The control unit includes a controller (main control unit) that processes drive control of each joint actuator constituting the robot device 100 and external input from each sensor (described later), and a housing in which a power supply circuit and other peripheral devices are mounted. Is the body. In addition, the control unit may include a communication interface and a communication device for remote operation.

  ZMP is used as a norm for determining the stability of walking for posture stability control of the robot apparatus including correction of walking motion trajectory. For this reason, interpolation calculation using a fifth-order polynomial is performed so that the position, speed, and acceleration for reducing the deviation from ZMP are continuous. The standard for discriminating the stability by ZMP is the principle of d'Alembert that gravity and inertia force from the walking system to the road surface, and these moments balance with the floor reaction force and the floor reaction force moment as a reaction from the road surface to the walking system. based on. As a result of the dynamic reasoning of the principle, there is a point where the pitch axis and roll axis moments become zero, that is, ZMP, inside the support polygon formed by the sole contact point and the road surface.

  FIG. 33 schematically shows a joint degree-of-freedom configuration of the robot apparatus 100. As shown in the figure, the robot apparatus 100 includes an upper limb that includes two arms and a head 1, a lower limb that includes two legs that realize a moving operation, and a trunk that connects the upper limb and the lower limb. And a structure having a plurality of limbs composed of the waist.

  The neck joint (Neck) that supports the head has four degrees of freedom: a neck joint yaw axis 1, first and second neck joint pitch axes 2 a and 2 b, and a neck joint roll axis 3.

  Each arm portion has a degree of freedom as a shoulder joint pitch axis 4 at the shoulder, a shoulder joint roll axis 5, an upper arm yaw axis 6, an elbow joint pitch axis 7 at the elbow, and a wrist ( Wrist) is composed of a wrist joint yaw axis 8 and a hand portion. The hand part is actually a multi-joint / multi-degree-of-freedom structure including a plurality of fingers.

  The trunk (Trunk) has two degrees of freedom: a trunk pitch axis 9 and a trunk roll axis 10.

  Further, each leg part constituting the lower limb includes a hip joint yaw axis 11 at the hip joint (Hip), a hip joint pitch axis 12, a hip joint roll axis 13, a knee joint pitch axis 14 at the knee (Knee), and an ankle (Ankle). ), An ankle joint pitch axis 15, an ankle joint roll axis 16, and a foot.

  However, the robot apparatus 100 does not have to be equipped with all the above-described degrees of freedom or is not limited to this. It goes without saying that the degree of freedom, that is, the number of joints, can be increased or decreased as appropriate in accordance with design / manufacturing constraints and required specifications.

  Each degree of freedom of the robot apparatus 100 as described above is actually mounted using a rotary actuator, and motion control is performed based on these rotational position controls. These joint actuators must be small and light because of the need to eliminate extra bulges in appearance and approximate human body shape, and to perform posture control on unstable structures such as biped walking. Is preferred.

  In this embodiment, a small AC servo actuator of a gear direct connection type and a servo control system of a single chip built in a motor unit is mounted. This type of AC servo actuator is disclosed in, for example, Japanese Patent Laid-Open No. 2000-299970 already assigned to the present applicant. Each joint actuator is provided with a torque sensor for detecting motor torque and an angle / position sensor for detecting a rotational position or a joint position.

  Further, in this embodiment, by adopting a reduced speed gear as the direct connection gear of the actuator / motor, the passive characteristic of the drive system required for the robot 100 of the type that places importance on physical interaction with humans is obtained. ing.

  FIG. 34 schematically shows a control system configuration of the robot apparatus 100. As shown in the figure, the robot device 100 performs adaptive control for realizing the cooperative operation between the mechanism units 30, 40, 50R / L, and 60R / L representing the human extremities and the mechanism units. It is comprised with the control unit 80 (however, each of R and L is a suffix which shows each of right and left, and so on).

  The overall operation of the robot apparatus 100 is comprehensively controlled by the control unit 80. The control unit 80 exchanges data and commands between a main control unit 81 configured by main circuit components (not shown) such as a CPU (Central Processing Unit) and a memory, and each component of the power supply circuit and the robot 100. The peripheral circuit 82 includes an interface (none of which is shown).

  The peripheral circuit 82 referred to here is an interface for connecting external peripheral devices, charging stations (not shown), and other peripheral devices connected through cables and radio, in addition to peripheral devices mounted on the robot apparatus.・ Include connectors.

  In realizing the present invention, the installation location of the control unit 80 is not particularly limited. Although it is mounted on the trunk unit 40 in FIG. 34, it may be mounted on the head unit 30. Alternatively, a control unit 80 may be provided outside the robot apparatus 100 to communicate with the robot apparatus 100 main body by wire or wirelessly.

Each joint freedom degree in the robot apparatus 100 shown in FIG. 33 is implement | achieved by the actuator corresponding to each. That is, the head unit 30 includes a neck joint yaw axis actuator M ₁ , neck joint pitch axis actuator M ₂ , neck joint roll representing the neck joint yaw axis 1, neck joint pitch axis 2, and neck joint roll axis 3. axis actuator M ₃ is disposed.

The trunk unit 40 is provided with a trunk pitch axis actuator M ₉ and a trunk roll axis actuator M ₁₀ that represent the trunk pitch axis 9 and the trunk roll axis 10, respectively.

Further, the arm unit 50R / L is subdivided into an upper arm unit 51R / L, an elbow joint unit 52R / L, and a forearm unit 53R / L, but a shoulder joint pitch axis 4, a shoulder joint roll axis 5, an upper arm Shoulder joint pitch axis actuator M ₄ , shoulder joint roll axis actuator M ₅ , upper arm yaw axis actuator M ₆ , elbow joint pitch axis actuator M ₇ representing each of yaw axis 6, elbow joint pitch axis 7, and wrist joint yaw axis 8. , wrist joint yaw axis actuator M ₈ is arranged.

The leg unit 60R / L is subdivided into a thigh unit 61R / L, a knee unit 62R / L, and a shin unit 63R / L, but the hip joint yaw axis 11, hip joint pitch axis 12, hip joint The hip joint yaw axis actuator M ₁₁ , the hip joint pitch axis actuator M ₁₂ , the hip joint roll axis actuator M ₁₃ , the knee joint pitch representing the roll axis 13, the knee joint pitch axis 14, the ankle joint pitch axis 15, and the ankle joint roll axis 16. An axis actuator M ₁₄ , an ankle joint pitch axis actuator M ₁₅ , and an ankle joint roll axis actuator M ₁₆ are provided.

The actuators M ₁ , M ₂ , M ₃ ... Used for each joint are more preferably a small AC servo actuator of the type directly mounted on a gear and mounted in a motor unit with a servo control system integrated into a single chip. ).

  For each mechanism unit such as the head unit 30, the trunk unit 40, the arm unit 50, and each leg unit 60, sub-control units 35, 45, 55, and 65 for actuator drive control are disposed.

  An attitude sensor 95 and an acceleration sensor 96 are disposed on the waist 41. The acceleration sensor 96 is arranged in each XYZ axial direction. Further, by providing an acceleration sensor 96 on the waist 41, the waist 41, which is a part with a large mass operation amount, is set as a control target point, and the posture and acceleration at that position are directly measured, and the posture based on ZMP Stable control can be performed. The attitude sensor 95 and the acceleration sensor 96 are configured as an acceleration sensor A1 and a gyro sensor G1, respectively, in FIG.

  In addition, ground check sensors 91 and 92 and acceleration sensors 93 and 94 are disposed on the legs 60R and 60L, respectively. The ground contact confirmation sensors 91 and 92 are configured by, for example, mounting a pressure sensor on the sole, and can detect whether the sole has landed based on the presence or absence of a floor reaction force. The acceleration sensors 93 and 94 are arranged at least in the X and Y axial directions. By disposing the acceleration sensors 93 and 94 on the left and right feet, the ZMP equation can be directly assembled with the feet closest to the ZMP position. In FIG. 33, sensors A2 and A2 for measuring acceleration at the foot and gyro sensors G2 and G3 for measuring the posture of the foot are arranged on the left and right ankles, respectively. In addition, force sensors F1 to F4 and F5 to F8 for measuring ground contact and floor reaction force are disposed at the four corners of the left and right soles.

  Here, when the acceleration sensor is arranged only on the waist 41 that is a part where the mass operation amount is large, only the waist 41 is set as the control target point, and the state of the foot is relative to the calculation result of the control target point. Therefore, the following condition must be satisfied between the foot and the road surface.

(1) The road surface does not move no matter what force or torque is applied.
(2) The friction coefficient with respect to translation on the road surface is sufficiently large, and no slip occurs.

  On the other hand, in the present embodiment, a reaction force sensor system (such as a floor reaction force sensor) that directly applies a force to the ZMP is provided on a foot that is a contact portion with the road surface, and local coordinates used for control and the coordinates thereof An acceleration sensor for directly measuring is provided. As a result, the ZMP balance equation can be assembled directly at the foot closest to the ZMP position. Therefore, more rigorous posture stabilization control can be realized at high speed. As a result, even on a gravel where the road surface moves when force or torque is applied, on a carpet with long bristle feet, or on a residential tile that cannot easily secure a sufficient translational friction coefficient, it can easily slide. The stable walking (motion) of the device can be guaranteed.

  The main control unit 80 can dynamically correct the control target in response to the outputs of the sensors A1 to A3, G1 to G3, and F1 to F8. More specifically, adaptive control is performed on each of the sub-control units 35, 45, 55, and 65 to realize a whole body movement pattern in which the upper limbs, trunk, and lower limbs of the robot apparatus 100 are driven in cooperation. To do.

The whole body movement of the robot apparatus 100 sets foot movements, ZMP trajectories, trunk movements, upper limb movements, waist heights, etc., and commands for instructing movements according to these setting contents to the sub-control units 35, Forward to 45, 55, 65. Then, in each of the sub-control section 35, 45 ... interprets the command received from the main control unit 81 outputs a drive control signal to each actuator _{M 1, M 2, M 3} .... Here, “ZMP” refers to a point on the floor where the moment due to floor reaction force during walking is zero, and “ZMP trajectory” refers to, for example, ZMP during the walking motion period of the robot 100. It means the trajectory that moves.

  The legged mobile robot can use ZMP as a norm for determining the stability of walking. The stability criterion for ZMP is that the system forms an appropriate ZMP space, and if there is ZMP inside the support polygon, the system does not generate rotational or translational motion, and solves the equations of motion related to rotation and translation. There is no need. On the other hand, when there is no ZMP inside the support polygon or when there is no support action point for the outside world, it is necessary to solve the equation of motion instead of the ZMP equation. For example, since a support polygon does not exist at the time of getting off, such as when jumping or jumping off a hill, the equation of motion may be solved instead of the ZMP equation.

The ZMP equation describes the balance relationship of each moment on the target ZMP. In the control system according to this embodiment has a mass properties relates to a robot device (i.e. distribution information of mass) represents the robot in a number of mass points m _i. Then, when these mass points m _i and the control target point, in all expressions ZMP equilibrium equation for obtaining the orbit of each control point total is zero moment on the target ZMP generated at the control target point m _i is there.

  The ZMP balance equation described in the world coordinate system is as follows.

  The ZMP balance equation described in the local coordinate system of the robot apparatus is as follows.

Each of the above equations represents the sum of moments around the ZMP (radius r _i −P _zmp ) generated by the acceleration component applied at each mass point (or control target point) m _i and the applied j-th external force moment. It is described that the sum of M _{j and} the sum of moments around the ZMP generated by the external force F _k (the point of action of the _kth external force F _k is S _k ) are balanced.

  This ZMP balance equation includes a floor reaction force moment (moment error component) T in the target ZMP. By suppressing this moment error to zero or within a predetermined allowable range, the posture stability of the robot apparatus is maintained. In other words, correcting the movement (such as the foot movement and the trajectory of each part of the upper body) so that the moment error is zero or less than the allowable value is the essence of posture stabilization control using ZMP as a stability criterion. It is.

  As described above, when the acceleration sensor is provided on the foot sole that is the contact portion with the road surface, the local coordinate system of the real robot with respect to the world coordinate system is set, and the foot sole as the origin is set. And the ZMP balance equation can be derived directly. Furthermore, by arranging an acceleration sensor at a control target point having a large mass manipulated variable including the waist 41, the moment amount around the ZMP for each control target point can be directly derived using the acceleration sensor output value. Can do.

  Here, since a real robot as a control target is a mobile device, it is difficult to obtain a position vector on the world coordinates at each control target point. As an alternative, the position vector of the control target point on the local coordinates is obtained by a relatively easy calculation method such as inverse kinematics calculation. Therefore, the actual posture stabilization process may be performed using the latter ZMP balance equation described in the local coordinate system of the robot apparatus. (Since it is difficult to accurately measure the ZMP trajectory in the world coordinate system with the state detector mounted on the robot, an external measuring instrument fixed to the world coordinate system is generally required. On the other hand, the ZMP trajectory can be obtained accurately and directly in local coordinates by measuring the ZMP trajectory by placing the load sensors F1 to F8 on the ground contact portion. In addition, in the present embodiment, in the local coordinate system, it is possible to directly measure information that is dominant in high-speed motion by arranging an acceleration sensor near the local coordinate origin. ZMP balance equation is used.)

  The robot apparatus 100 has a center of gravity set at the waist position, is an important control target point for posture stability control, and constitutes a “base” of the apparatus.

  The present invention has been described in detail above with reference to specific embodiments. However, it is obvious that those skilled in the art can make modifications and substitutions of the embodiments without departing from the gist of the present invention.

  The gist of the present invention is not necessarily limited to a product called a “robot”. That is, as long as it is a mechanical device that performs an exercise resembling human movement using an electrical or magnetic action, the present invention can be applied to products belonging to other industrial fields such as toys. Can be applied.

  In short, the present invention has been disclosed in the form of exemplification, and the description of the present specification should not be interpreted in a limited manner. In order to determine the gist of the present invention, the claims section described at the beginning should be considered.

FIG. 1 is a diagram showing an outline of a multi-robot system according to an embodiment of the present invention. FIG. 2 is a side view of the stage showing the camera capturing the multi-robot on the stage from above. FIG. 3 is a diagram showing a bird's-eye view of the stage under a multi-robot environment captured from a camera above the stage. FIG. 4 is a diagram illustrating an example of an instruction operation. FIG. 5 is a diagram illustrating a configuration example of an operation plan. FIG. 6 is a diagram illustrating a configuration example of an operation plan. FIG. 7 is a diagram showing a configuration example of a scenario. FIG. 8 is a diagram illustrating a configuration example in which a plurality of scenarios are connected. FIG. 9 is a diagram for explaining a mechanism for synchronizing time between robots and between a robot and a general controller. FIG. 10 is a diagram for explaining a mechanism for synchronizing time between robots and between a robot and a general controller. FIG. 11 is a diagram illustrating a configuration example of a multi-robot operation GUI screen provided by the overall controller. FIG. 12 is a diagram illustrating a configuration example of an interference area confirmation screen. FIG. 13 is a flowchart showing a processing procedure for imposing restrictions on the direction in which the robot falls. FIG. 14 is a diagram illustrating a configuration example of the deviation estimation screen. FIG. 15 is a flowchart showing a processing procedure for estimating the deviation between the position and the direction. FIG. 16 is a diagram for explaining a trajectory display of steps with the Z axis as a time axis. FIG. 17 is a diagram for explaining a trajectory display of steps with the Z axis as a time axis. FIG. 18 is a diagram for explaining a trajectory display of steps with the Z axis as a time axis. FIG. 19 is a diagram for explaining a monitoring mechanism of the position and direction of the robot in the multi-robot system. FIG. 20 is a diagram for explaining a monitoring mechanism of the position and direction of the robot in the multi-robot system. FIG. 21 is a flowchart showing a processing procedure for the general controller to acquire information related to the position and direction of the robot at a certain time from the monitor system. FIG. 22 is a flowchart showing a processing procedure for the monitor system to acquire the position and direction of the robot on the basis of the image taken by the ceiling camera and transfer it to the general controller. FIG. 23 is a flowchart showing the procedure of the position and direction correction plan. FIG. 24 is a diagram showing how the planned trajectory of the legs is managed by walking. FIG. 25 is a diagram showing a deviation that occurs during operation of the motion plan with respect to the planned trajectory shown in FIG. FIG. 26 is a flowchart showing a processing procedure for making a correction plan. FIG. 27 is a flowchart illustrating a processing procedure for deriving a provisional correction amount for a step to be corrected. FIG. 28 is a flowchart showing a processing procedure for making a movement plan. FIG. 29 is a flowchart showing a processing procedure for introducing a spare robot. FIG. 30 is a flowchart showing a processing procedure for determining an abnormal level. FIG. 31 is a view showing a state in which the “humanoid” or “humanoid” robot apparatus 100 used for carrying out the present invention is viewed from the front. FIG. 32 is a view showing a state in which the “humanoid” or “humanoid” robot apparatus 100 used for carrying out the present invention is viewed from the rear. FIG. 33 is a diagram schematically illustrating a joint degree-of-freedom configuration included in the robot apparatus 100. FIG. 34 is a diagram schematically showing a control system configuration of the robot apparatus 100.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Neck joint yaw axis 2A ... 1st neck joint pitch axis 2B ... 2nd neck joint (head) pitch axis 3 ... Neck joint roll axis 4 ... Shoulder joint pitch axis 5 ... Shoulder joint roll axis 6 ... Upper arm yaw axis 7 ... Elbow joint pitch axis 8 ... Wrist joint yaw axis 9 ... Trunk pitch axis 10 ... Trunk roll axis 11 ... Hip joint yaw axis 12 ... Hip joint pitch axis 13 ... Hip joint roll axis 14 ... Knee joint pitch axis 15 ... Ankle joint pitch Axis 16 ... Ankle joint roll axis 30 ... Head unit, 40 ... Trunk unit, 41 ... Lumbar unit 50 ... Arm unit, 51 ... Upper arm unit 52 ... Elbow joint unit, 53 ... Forearm unit 60 ... Leg unit, 61 ... thigh unit 62 ... knee joint unit, 63 ... shin part unit 80 ... control unit, 81 ... main control unit 82 ... peripheral circuit 91, 92 ... grounding confirmation sensor 93, 94 ... acceleration sensor 95: Posture sensor 96 ... Acceleration sensor 100 ... Legged mobile robot

Claims (53)

In a control system of a plurality of robot devices operating on a specific operation field,
Monitoring means for monitoring the position and direction of each robot device on the motion field;
An overall control means for controlling the synchronous and cooperative operation of each robot apparatus based on the monitoring result of the monitoring means and the status information of the robot apparatus acquired from each robot apparatus, comprising communication means with each robot apparatus;
Equipped with,
The overall control means includes a fall area calculating means for calculating a fall area of each robot apparatus on the operation field, a maximum movable range calculating means for calculating a maximum movable range of each robot apparatus, and each calculated robot apparatus. An interference area display means for displaying on the screen as an interference area between the robot devices,
A control system for a robotic device. The monitoring unit includes an image input unit that captures the motion field, and an image processing unit that processes the input image to acquire information on the position and direction of each robot apparatus.
The robot apparatus control system according to claim 1. Casting of each robot device on the operation field is determined,
The overall control means manages a set of operation plans of the robot apparatus determined for each casting as a scenario,
The robot apparatus control system according to claim 1. The operation plan of the robot apparatus for each casting is configured by arranging one or more instruction operations for the corresponding robot apparatus on the time axis,
The overall control means manages execution of the operation plan of each casting on the same time axis,
The robot apparatus control system according to claim 3. The overall control means connects two or more scenarios on a time axis and manages them as one performance by a multi-robot.
The robot apparatus control system according to claim 3. The overall control means prepares a preliminary scenario that is executed in an alternative manner in case it becomes difficult or impossible to continue due to the occurrence of an abnormal situation in each scenario.
The robot apparatus control system according to claim 5. The overall control means skips one or more scenarios that have become unexecutable and uses a preliminary scenario to return to a subsequent scenario connected on the time axis .
The robot apparatus control system according to claim 6. The overall control means uses a preliminary scenario to complete the performance in place of all subsequent scenarios that have become unexecutable when an abnormal situation in which the performance cannot be completed occurs,
The robot apparatus control system according to claim 6. The overall control means uses a preliminary scenario corresponding to a scene where an abnormal situation has occurred in one scenario.
The robot apparatus control system according to claim 6. The overall control means includes synchronization acquisition means for acquiring time synchronization with each robot apparatus on the operation field, so that each casting robot apparatus executes each operation plan synchronously and cooperatively according to the time. To control,
The robot apparatus control system according to claim 1. The overall control means includes a current state display means for displaying the position and direction of each robot apparatus on the operation field on the screen based on information on the position and direction of each robot apparatus obtained by the monitoring means.
The robot apparatus control system according to claim 2. The current state display means presents a deviation between an actual position and direction obtained by the monitoring means and a target position and direction on an operation plan of each robot apparatus.
The robot apparatus control system according to claim 11. Each robotic device has one or more overturning movements,
The overall control means prohibits the use of a fall operation that has the interference area in a fall area or has the interference area as a fall direction,
The robot apparatus control system according to claim 1. The overall control means calculates a motion characteristic of the robot based on the motion of the robot apparatus performed in the past, and calculates a deviation from a planned value when a future motion plan is executed based on the motion characteristics of the robot apparatus. A deviation predicting means for predicting,
The robot apparatus control system according to claim 2 . The deviation prediction means divides a means for observing the error caused when running the motion in the past predetermined period t _s ~t _m1, motion executed in the past predetermined period t _s ~t _m1 per element movement elements means for estimating the error occurring in each exercise, by subsequently dividing the motion to be executed during the period up to t _{m @ 2} ~t _e element motion, summing the estimated error these elements exercise has, time t _e Means for estimating a deviation from a target value of the position and direction at
The robot apparatus control system according to claim 14 . The overall control unit includes a deviation prediction display unit that displays the predicted deviation on a screen.
The robot apparatus control system according to claim 14 . The overall control means includes a trajectory display means for displaying a trajectory on the operation field of the robot apparatus.
The robot apparatus control system according to claim 1 . The trajectory display means displays a trajectory of a step with the Z axis as a time axis.
The robot apparatus control system according to claim 17 . The overall control unit includes a position / direction correction unit that corrects a deviation between a target position and direction on an operation plan of each robot apparatus and an actual position and direction obtained by the monitoring unit.
The robot apparatus control system according to claim 1 . The position / direction correction unit includes a correction plan unit that corrects an error in the position or direction of the robot apparatus that is executing the motion plan using an existing step in the motion plan.
The robot apparatus control system according to claim 19 . The position / direction correction unit includes a movement plan unit that generates one or more new steps and corrects a deviation in the position / direction using a scene where the robot apparatus is not executing an operation plan.
The robot apparatus control system according to claim 19 . The overall control means defines an abnormal state according to the degree of abnormality in a multi-robot environment in which a plurality of robots execute a scenario in the operation field, and performs appropriate repair processing when falling into each abnormal state Provided with abnormality processing means,
The robot apparatus control system according to claim 1 . The abnormality processing means executes abnormality processing for the robot apparatus in a first abnormality state where the abnormality processing is completed with only one robot apparatus.
The control system for a robot apparatus according to claim 22 . In the second state where the abnormality processing is necessary for the entire multi-robot, the abnormality processing means is a standby robot that waits in or out of the operation field, in which a cast is changed between the robot devices operating in the operation field. To the operation field, or to correct the formation in the multi-robot,
The control system for a robot apparatus according to claim 22 . The abnormality processing means corrects the overall formation in the multi-robot in a third abnormal state where a part of the scenario cannot be continued.
The control system for a robot apparatus according to claim 22 . The abnormality processing means stops the continuation of the scenario in a fourth abnormal state where the entire scenario cannot be continued.
The control system for a robot apparatus according to claim 22 . Further comprising a reserve robot waiting in or outside the motion field;
The overall control means controls the input of the spare robot to the operation field;
The robot apparatus control system according to claim 1 . In a control method of a plurality of robot devices operating on a specific operation field,
A monitoring step of monitoring the position and direction of each robotic device on the motion field;
A communication step with each robot device;
Based on the monitoring result in the monitoring step and the state information of the robot apparatus acquired by communication from each robot apparatus, an overall control step for controlling the synchronous cooperative operation by each robot apparatus;
Have
The overall control step includes a fall area calculating step for calculating a fall area of each robot apparatus on the operation field, a maximum movable range calculating step for calculating a maximum movable range of each robot apparatus, and the calculated robot apparatuses. And an interference area display step for displaying on the screen as an interference area between the robot devices, the fall area of
A method for controlling a robot apparatus, comprising: The monitoring step includes an image input step of photographing the operation field, and an image processing step of processing the input image to acquire information on the position and direction of each robot device.
The method of controlling a robot apparatus according to claim 28, wherein: Casting of each robot device on the operation field is determined,
In the overall control step, a set of operation plans of the robot apparatus determined for each casting is managed as a scenario.
The method of controlling a robot apparatus according to claim 28 , wherein: The operation plan of the robot apparatus for each casting is configured by arranging one or more instruction operations for the corresponding robot apparatus on the time axis,
In the overall control step, the execution of the action plan of each casting is managed on the same time axis.
The method of controlling a robot apparatus according to claim 30 , wherein In the overall control step, two or more scenarios are connected on a time axis and managed as one performance by a multi-robot.
The method of controlling a robot apparatus according to claim 30 , wherein In the overall control step, in preparation for a case where it is difficult or impossible to continue due to occurrence of an abnormal situation in each scenario, a preliminary scenario that is executed alternatively is prepared.
The method of controlling a robotic device according to claim 32 . In the overall control step, a preliminary scenario is used to skip one or more subsequent scenarios that have become unexecutable and return to the subsequent scenario linkage linked on the time axis.
34. The method of controlling a robot apparatus according to claim 33. In the overall control step, when an abnormal situation in which the performance cannot be completed occurs, a preliminary scenario is used in order to complete the performance instead of all subsequent scenarios that cannot be executed.
34. The method of controlling a robot apparatus according to claim 33 . In the overall control step, a preliminary scenario corresponding to a scene where an abnormal situation has occurred in one scenario is used.
34. The method of controlling a robot apparatus according to claim 33 . In the overall control step, time synchronization with each robot apparatus on the operation field is obtained, and control is performed so that each casting robot apparatus executes each operation plan synchronously in accordance with the time.
The method of controlling a robot apparatus according to claim 28 , wherein: The overall control step includes a current state display step of displaying the position and direction of each robot apparatus on the operation field on the screen based on information on the position and direction of each robot apparatus obtained by the monitoring unit.
30. The method of controlling a robot apparatus according to claim 29 . In the current state display step, the deviation between the target position and direction on the operation plan of each robot apparatus and the actual position and direction obtained in the monitoring step is presented.
40. The method of controlling a robotic device according to claim 38 . Each robotic device has one or more overturning movements,
In the overall control step, the use of the overturning operation with the interference region in the overturning region or with the interference region in the overturning direction is prohibited.
The method of controlling a robot apparatus according to claim 28 , wherein: The overall control step calculates a motion characteristic of the robot based on the motion of the robot apparatus performed in the past, and calculates a deviation from a planned value when a future motion plan is executed based on the motion characteristics of the robot apparatus. Having a deviation prediction step to predict,
30. The method of controlling a robot apparatus according to claim 29 . The deviation prediction step is configured to observe the error caused when running the motion in the past predetermined period t _s ~t _m, divides the motion executed in the past predetermined period t _s ~t _m per element movement elements motion error estimates the occurring for each, divided into motion element motion to be executed during the period up to the subsequent t _m ~t _e, by summing the estimated error these elements exercise has a position and at time t _e Estimate the deviation from the target value in the direction,
42. The method of controlling a robot apparatus according to claim 41, wherein: A deviation prediction display step for displaying the predicted deviation on a screen;
42. The method of controlling a robot apparatus according to claim 41 , wherein: A trajectory display step for displaying a trajectory on the operation field of the robot apparatus;
The method of controlling a robot apparatus according to claim 28 , wherein: In the trajectory display step, a trajectory display of a step with the Z axis as a time axis is performed.
45. The method of controlling a robot apparatus according to claim 44 , wherein: A position and direction correction step for correcting a deviation between a target position and direction in the operation plan of each robot apparatus and an actual position and direction obtained in the monitoring step;
The method of controlling a robot apparatus according to claim 28 , wherein: In the position direction correction step, a correction plan for correcting an error in the position or direction of the robot apparatus that is executing the motion plan using an existing step in the motion plan is established.
The method of controlling a robot apparatus according to claim 46, wherein: In the position / direction correction step, using a scene in which the robot apparatus is not executing an operation plan, a movement plan is generated that generates one or more new steps and corrects a deviation in the position / direction.
The method of controlling a robot apparatus according to claim 46 , wherein: In a multi-robot environment in which a plurality of robots execute a scenario in the operation field, the robot has an abnormality processing step for performing an appropriate repair process when falling into each abnormal state defined according to the degree of abnormality.
The method of controlling a robot apparatus according to claim 28 , wherein: In the abnormality processing step, in the first abnormal state where the abnormality processing is completed with only one robot apparatus, the abnormality processing for the robot apparatus is executed.
The method for controlling a robot apparatus according to claim 49 , wherein: In the abnormality processing step, in a second state where the abnormality processing is necessary for the entire multi-robot, a change of casting between robot devices operating in the operation field, a standby robot waiting in or outside the operation field To the operation field, or to correct the formation in the multi-robot,
The method for controlling a robot apparatus according to claim 49 , wherein: In the abnormality processing step, in the third abnormal state where a part of the scenario cannot be continued, the overall formation in the multi-robot is corrected.
The method for controlling a robot apparatus according to claim 49 , wherein: In the abnormality processing step, the continuation of the scenario is stopped in a fourth abnormal state where the entire scenario cannot be continued.
The method for controlling a robot apparatus according to claim 49 , wherein:

Images