JP2006088328A - Communication robot and its development supporting device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a communication robot capable of taking communicating action not monotonous and with no contradiction and to provide a development supporting device of the communication robot capable of easily grasping a relation among a large number of modules. <P>SOLUTION: The communication robot 10 is furnished with a plurality of the modularized action modules and a plurality of rules specifying fundamental execution order of the action modules. The action module to be executed next is selected and executed in accordance with execution hysteresis of the actual action module, that is, hysteresis of communication action of the robot 10 and a human on the robot 10. A visualized screen on which each of the action modules is expressed in an icon image is displayed on a display device 22 of the development supporting device 12 of such the robot 10. Each of icons is arranged in accordance with strength of a relation between each of the action modules, and its display form is discriminated in accordance with a retrieval result. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a communication robot and its development support device, and in particular, for example, defines a plurality of behavior modules obtained by modularizing behavior in a specific situation, defines a plurality of rules regarding the basic execution order of the behavior modules, and the plurality of rules. The present invention relates to a communication robot that takes a communication action by executing a plurality of action modules based on the above and a development support apparatus for developing such a communication robot.

Conventionally, there is a technique for converting a robot action into a software module and executing the software module (see, for example, Non-Patent Document 1).
Ishiguro, Ono, Imai, Maeda, Kanda, Nakatsu, “Robovie: an interactive humanoid robot”, Int. J. Industrial Robotics, 2001, Vol. 28, Vol. 6, p. 498-503

  In order to realize a non-monotonous behavior in a conventional robot, it was necessary to execute a behavior module based on a random element, so that behavior that contradicts the behavior that the robot itself has taken in the past has occurred. There was a risk of it. On the other hand, starting the behavior module so that there is no contradiction results in a monotonous behavior. For example, if all the action modules to be executed are determined in advance, it becomes a monotonous action that is determined regardless of the situation. Such behavior is suitable as an industrial robot that performs a predetermined operation within a certain range of environment, but is not suitable as a communication robot that performs various communications with humans.

  Further, when developing a robot having a large number of modules, there is no means for expressing the relationship between existing modules, and it is difficult to grasp the relationship. Therefore, it is difficult to develop a new module relationship without having to grasp the relationship that the already installed module has, and to change the existing relationship. In general, there is a graphical program method in which only the program input means is visualized, but even in such a graphical program method, the relationship between a large number of modules cannot be grasped.

  Therefore, a main object of the present invention is to provide a communication robot that can take communication actions that are not monotonous and consistent.

  Another object of the present invention is to provide a communication robot development support device that can easily grasp the relationship between a large number of modules.

  A first invention includes a plurality of behavior modules obtained by modularizing behavior in a specific situation, and a plurality of rules that define a basic execution order of the behavior modules, and the plurality of behavior modules are executed based on the plurality of rules. A communication robot that takes communication action by an action module storage means for storing a plurality of action modules, a rule storage means for storing a plurality of rules, and comparing an execution history of the action modules with a plurality of rules. Candidate selecting means for selecting a candidate corresponding to a rule corresponding to the above and a next executable action module, a next action selecting means for selecting an action module to be executed next from candidates selected by the candidate selecting means, and a next action selecting means Action module execution means for executing the action module selected by Obtain, it is a communication robot.

  The second invention includes a plurality of behavior modules obtained by modularizing behavior in a specific situation, and a plurality of rules that define a basic execution order of the plurality of behavior modules, and the plurality of behavior modules are executed based on the plurality of rules. A communication robot development support device for developing a communication robot that takes communication action by performing screen display means for visualizing a relationship between a plurality of action modules, and a plurality of action modules respectively An arrangement calculating means for calculating the arrangement of the image to be displayed on the visualization screen based on the strength of the relationship between the plurality of behavior modules, and comparing the specified search situation with a plurality of rules, and corresponding to the search situation Search means for selecting rules, search for image display form of multiple behavior modules Display mode setting means for setting differently based on the situation and the search result by the processing of the search means, and displaying images of a plurality of behavior modules on the visualization screen based on the calculated arrangement and the set display form It is a communication robot development support apparatus provided with the action module display means to do.

  The communication robot of the first invention includes a plurality of behavior modules and a plurality of rules concerning the basic execution order of the behavior modules. The behavior module includes, for example, a precondition for executing this behavior module, an instruction unit for presenting the robot behavior, and a recognition unit for recognizing a human reaction to the robot behavior, and only when the precondition is satisfied Is executed, and then the recognition unit is executed. In this robot, an action module to be executed next is selected and executed based on a history of communication actions that the robot has performed so far. That is, the candidate selecting means compares the execution history of the behavior module with a plurality of stored rules, and selects a rule corresponding to the execution history. The rule basically defines the action module to be performed next in a specific situation, and the specified action module is selected as a candidate of the next executable action module. Then, the next action selection means selects the action module to be executed next from the selected candidates. For example, a behavior module that matches the result of the behavior module that was being executed is selected, and one that satisfies the preconditions defined in the behavior module is selected. In this way, the behavior module selected based on the execution history is executed by the behavior module execution means.

  Moreover, the priority may be set to each rule. In this case, when a plurality of rules conflict, the rule to be applied can be selected according to this priority.

  In the development support device of the second invention, the screen display means displays a visualization screen for visualizing the relationship between the plurality of behavior modules mounted on the robot. Images (icons) representing a plurality of behavior modules are displayed on this visualization screen. In visualization, the arrangement of the image of each behavior module is calculated by the arrangement calculation means based on the strength of the relationship between the behavior modules. As a result, for example, the more rules that connect the behavior modules, the stronger the relationship and the closer they are arranged on the visualization screen. Further, the display form setting unit distinguishes and sets the display form of the image of each behavior module. In distinguishing the display forms, settings are made based on the designated search situation and the search result by the search means. By this search means, a rule corresponding to the situation designated by the user who is the developer is selected. For example, this apparatus has three search modes. By selecting the current search mode, a search can be performed with the current state as the designated status, and in the history by selecting the past search mode. A search can be performed with the state at a past point in time as a specified status, and an arbitrary status can be input and specified by selecting a character search mode. In the search means, a rule corresponding to a specified situation in each search mode is selected, and a candidate of an action module that can be executed next to the situation is selected. By this search means, each behavior module has a relationship with the behavior module at the time of interest in each search mode, that is, corresponds to what has been executed in the past, or what is executed next It becomes clear whether it corresponds to. Each behavior module is distinguished according to the relationship based on the search result of each search mode, that is, the relationship of each behavior module to the behavior module at the time of interest for each search mode. For example, a red and rectangular current icon is set for the action module at the point of interest, and a blue and rectangular past icon is set for the action module past the point of interest. A green and rectangular next icon is set for the module, and a gray and small square irrelevant icon is set for the action module not related to the action module at the time of interest. In this manner, the behavior module display means displays the behavior module images on the visualization screen so as to have the calculated arrangement and the set display form, and the relationship between the behavior modules is visualized. The

  Further, the search result display means may display the search result selected by the search means, that is, the rule corresponding to the specified situation, for example, on a search screen. In this case, the developer can also check the contents of the rules on the screen.

  Further, the execution information receiving means may receive information related to the execution of the behavior module in the robot. In other words, the information necessary for visualization in the current search mode and the past search mode (robot execution history information, rules selected based on the history and the next executable action module) are used in the action selection process in the robot. Since this information is obtained, this information is transmitted from the robot so that the development support apparatus omits the search process.

  In addition, when an input means for editing a rule (new creation, correction, deletion, etc.) is provided, the developer can perform an editing operation while grasping the relationship of the behavior modules with reference to the visualization screen. The rule can be updated by registering the rule edited by this input means. The development support apparatus can visualize the relationship between the behavior modules based on the edited rules.

  Further, editing information relating to the rules edited by the input means may be transmitted to the robot by the editing information transmitting means. In this case, in the robot, the rule is updated according to the editing information, and the subsequent action selection process is selected based on the edited rule.

  According to the first invention, the next executable action module is selected and executed based on the actual action module execution history, that is, the history of communication activities between the robot and the human. Therefore, it is possible to take communication actions that are not monotonous and consistent.

  According to the second invention, the relationship between modules can be intuitively grasped by visualization. Therefore, while grasping the relationship between these multiple action modules, for example, modify or delete existing rules or create new rules so as to realize harmonized communication behavior without any contradiction in the situation. Can do.

  The above object, other objects, features and advantages of the present invention will become more apparent from the following detailed description of embodiments with reference to the drawings.

  Referring to FIG. 1, a communication robot (hereinafter, simply referred to as “robot”) 10 of this embodiment is an interaction-oriented communication robot for the purpose of communicating with a human. A communication robot development support device (hereinafter, simply referred to as “development support device”) 12 is for developing such a communication robot 10.

  The development support apparatus 12 is configured by a computer such as a personal computer (PC) or a workstation, for example, and includes a CPU 14 as shown in FIG. A memory 18, a hard disk device (HDD) 20, a display device 22 such as a liquid crystal display or a CRT, an input device 24 such as a mouse and a keyboard, and a communication circuit 26 are connected to the CPU 14 via a bus 16. The HDD 20 as the main storage device stores a program for executing this development support process, and also stores various data such as screen data displayed on the display device 22. Also, the HDD 20 stores the same action module and episode rule data described later as those stored in the robot 10. The memory 18 is, for example, a RAM and is used as a working memory or a buffer memory. A developer as a user operates the input device 24 to select or input various buttons or settings on the screen. The communication circuit 26 is a communication interface, which enables the development support apparatus 12 to communicate data with the robot 10 via a network or directly. The development support apparatus 12 and the robot 10 may be connected by wire, but it is desirable to perform wireless communication using a wireless LAN or the like so as not to hinder the action of the robot 10. Various types, modes, and the like of the wireless LAN can be applied. For example, a radio system, an IrDA infrared system, a Bluetooth system, and the like may be used, and an ad hoc mode or an infrastructure mode may be used.

  Here, first, the robot 10 to be developed will be described. The communication robot 10 is literally an interaction-oriented robot for the purpose of communicating with humans, and is different from a task-oriented robot such as an industrial robot that executes a specific task in a restricted area. Such a robot 10 is provided with a large number of interactive actions in various situations in order to take various communication actions with humans. Such an action is modularized and executed as an “action module”. In addition, the execution order of each action is defined as a rule (referred to as “episode rule”), and a communication action that maintains a consistent context or a harmonious situation in the long term is realized. That is, the robot 10 sequentially executes the behavior modules, and the execution order of the behavior modules is guided by the episode rule.

  In addition, the behavior module and episode rules will be described in detail. The behavior module includes a precondition section, an instruction section, and a recognition section, as shown in FIG. When executing an action module, first, the precondition section is executed, so that the robot 10 confirms whether or not the action module is in an executable state. For example, an action module that implements an action about the weather by retrieving weather information from the Internet is not executable and does not satisfy the preconditions when the robot 10 is not connectable to the Internet. If the precondition is satisfied, the instruction unit is executed next. As a result, the robot 10 takes an action that interacts with a human, and specifically presents a predetermined action to the human using utterances and gestures. A human will have some reaction to the behavior of the robot 10. For example, in the handshake action module, the robot 10 says "Let's shake hands" and presents its hand. In this case, when the person reacts to the action of the robot 10, the person will extend his hand around the hand of the robot 10. Therefore, the recognition unit is set to recognize several human reactions that are expected to be taken with respect to the action presented by the robot 10. In this way, the behavior module can create a specific situation and then recognize human behavior under that particular situation. The action module realizes action-reaction pairs as interactive and reactive actions in specific situations.

  In the robot 10 of this embodiment, the action module is set, for example, more than 100, for example, "HELLO" (with the greeting of Hello, recognize the reply), held out his hand to say "AKUSHU" ( "Let's shake hands." "ZYAN" (let's say "Janken", put your hand out and stare at the surroundings to see if someone put your hand), "HUG" “FROM” (recognizes where you came from and recognizes the name of the place) “BYE” (says bye-bye), “TURN” (the person who struck the shoulder when it was struck To humans such as “P_EYE” (pauses to rub eyes), “P_HEAD” (pauses to crawl the head), etc. Similar actions that do nothing and “EXPLORE” To) including the day-to-day activities such as work such as.

  The transition or transition of the behavior module includes a sequential transition and a reactive transition, as shown in FIG. 4, and the reactive module may be activated. The robot 10 recognizes a human reaction by executing the recognition unit after executing the instruction unit of the current behavior module. Thereafter, the robot 10 ends the execution of the current behavior module, records a result value corresponding to the recognition result, and transitions to the next executable behavior module (FIG. 4A). The next executable behavior module is determined by the result value of the current behavior module and the history of behavior module execution. This sequential transition is guided by episode rules. Communication also has problems that cannot be solved by such sequential transitions, ie obstructions and deviations. For example, when some people are talking and the phone rings, they will stop talking and answer the phone. Such disturbances and deviations are handled by reactive transitions and reactive modules. If the reactive transition is determined for the current situation and satisfies the preconditions of the corresponding next behavior module, the robot 10 stops executing the current behavior module and immediately transitions to the next behavior module. (FIG. 4B). This reactive transition is also guided by episode rules. On the other hand, when the reactive module is activated, the robot 10 does not stop the execution of the current behavior module, and executes the reactive module in parallel with this (FIG. 4C). For example, when the robot 10 is talking to a person and the person touches the arm of the robot 10, the robot 10 stares at the touched part by executing a reactive module (touches the eyeball or head). Let them know that they are aware of it.

  Episodic rules are rules related to the basic execution order of behavior modules. The basic execution order means a short-term behavior transition or transition, which may be, for example, a connection or relationship of several behavior modules, but a long-term (for example all day) or transition of all behaviors. It is not defined. In other words, it is possible to determine a short connection or ordering of a certain amount of behavior, such as shaking hands when greeting, but it is impossible to predetermine all the behaviors of the whole day, and episode rules As such, this short connection or ordering is prescribed. The episode rule basically defines an action module to be executed next in a certain situation (history). The episode rule includes a negative rule for suppressing execution of the next behavior module. For example, if the robot 10 asks the person where it came from and asks the same question a few minutes later, the situation is inconsistent and contradictory and inappropriate. Inappropriate behavioral transitions are suppressed by episode rules.

  Episodic rules are described according to a predetermined grammar as shown below. Each behavior module has a unique identifier called a module ID. First, “<ModuleID = result value>” is a rule for referring to the execution history and result value of the behavior module. “<ModuleID1 = result value 1> <ModuleID2 = result value 2>...” ”Means a rule that refers to the order of action modules executed previously. "<...> | <...>" means a selective group ("OR") of executed behavior modules. "(...)" means a block composed of a behavior module, a sequence of behavior modules, or a selective group of behavior modules. "(...) {n, m}" means repeating the block n times or more and m times or less. Similarly, "(...) *" is repeated 0 times or more, and "(...) +" is 1 or more times. Means repetition. Also, denial of all episode rules is specified by “!”. For example, "! <...> ... <...> NextModuleID" means that NextModuleID is not executed when this episode rule matches the current situation specified by "<...> ... <...>". The negation of the module ID and the result value is designated by “^”.

  In this embodiment, for example, more than 880 episode rules are set, for example, “<HELLO> AKUSHU” (shake a handshake when greeting), “<TURN> AKUSHU” (shake a handshake when you hit the shoulder) “<AKUSHU = Success> ZYAN” (If you get a handshake, you'll be Janken), “<AKUSHU = Failure> BYE” (If you don't get a handshake, it ’s called Bye Bye), “<TURN> <AKUSHU = Success> HUG” (If you get a handshake after being hit by the shoulder, I will do it), “! <TURN> <AKUSHU = failure> HELLO” (If you do not get a handshake after being hit by your shoulder, do not greet) , “<^ HELLO> <AKUSHU = failure> <HELLO> ZYAN” (Because you don't have a handshake after doing an action module other than the greeting, if you say hello, you'll be confused), “! <FROM> (<^ BYE >) + FROM ”(Listing where you came from and repeating the action module other than bye-bye more than once When, including asked not), etc. and came from where. In the example of the episode rule, the result value is described as “success” or “failure”, but these are actually described by being replaced with predetermined numerical values.

  Episodic rules guide robot 10 to a new episode of human interaction by controlling transitions between behavioral modules. In the robot 10, all episode rules are compared to the behavior module execution history to determine which behavior module to execute next. The robot 10 performs the comparison during the execution of the current behavior module and creates a list of behavior modules that can be executed next. After execution of the current behavior module, the robot 10 checks the preconditions for each behavior module in the list. If the precondition is satisfied, the action module is entered. Episodic rules have a priority, and if several episode rules conflict, the episode rule with the higher priority is applied.

  FIG. 5 shows an example of behavior module transition. As an initial state, the robot 10 investigates the environment by “EXPLORE”. At this time, when a human touches the shoulder of the robot, the episode rule 1 corresponding to this situation is applied, and a reactive transition according to the episode rule 1 occurs. This episode rule 1 stipulates that “Turn is performed when EXPLORE is interrupted”. The robot 10 turns to the person by executing “TURN”. When the result of the action of the instruction unit is successful, that is, for example, the person can be recognized, the episode rule 2 corresponding to this situation is applied. This episode rule 2 stipulates that “If the TURN is successful, execute GREET”. The robot 10 starts to greet the person by executing “GREET”. FIG. 5 shows an example in which a plurality of episode rules compete when determining a behavior module to be executed next to “GREET”. That is, the action module execution history and all episode rules are compared. The execution history of the action module is an action module executed in the past, a result value thereof, and an action module currently being executed. By this comparison, episode rules 3, 4 and 5 suitable for the current situation are obtained. Elected. Here, Episode Rule 3 stipulates that BYE is executed when GREET ends with an unresponsive result, and Episode Rule 4 states that if TURN succeeds and GREET is executed (the result is (Even if) Executes AKUSHU ”, and Episode Rule 5 does not execute BYE when an action module other than AKUSHU is repeated one to five times after EXPLORE is interrupted " That is, “BYE” is a candidate for the next executable module selected by the episode rule 3, “AKUSHU” is a candidate for the next executable module selected by the episode rule 4, and the episode rule 5 Suppresses the transition to “BYE”. In this case, the episode rule to be applied is determined according to the priority order of the competing episode rules, and the action module to be executed next is determined. For example, if the priority of episode rule 5 is higher than that of episode rule 3, “BYE” is not the next action module to be executed. If the priorities are further the same, the next execution module is determined at random, for example.

  Although the configuration of the software surface for performing various communication actions of the robot 10 has been described, the configuration of the hardware surface of the robot 10 will be described below.

  With reference to FIG. 6, the robot 10 includes a carriage 28, and wheels 30 for autonomously moving the robot 10 are provided on the lower surface of the carriage 28. The wheel 30 is driven by a wheel motor (indicated by reference numeral “86” in FIG. 7), and the carriage 28, that is, the robot 10 can be moved in any direction. Although not shown, a collision sensor (indicated by reference numeral “94” in FIG. 7) is attached to the front surface of the carriage 28, and this collision sensor is used for a person or other obstacle to the carriage 28. Detect contact. When contact with an obstacle is detected during the movement of the robot 10, the driving of the wheels 30 is immediately stopped to suddenly stop the movement of the robot 10 to prevent a collision.

  In this embodiment, the height of the robot 10 is set to about 100 cm so as not to intimidate people, particularly children. However, this height can be arbitrarily changed.

  A polygonal column sensor mounting panel 32 is provided on the carriage 28, and an ultrasonic distance sensor 34 is mounted on each surface of the sensor mounting panel 32. The ultrasonic distance sensor 34 measures the distance between the mounting panel 32, that is, the person around the robot 10 mainly.

  On the carriage 28, the lower part is surrounded by the above-described mounting panel 32, and the body of the robot 10 is attached so as to stand upright. The body includes a lower body 36 and an upper body 38, and the lower body 36 and the upper body 38 are connected by a connecting portion 40. Although not shown, the connecting portion 40 has a built-in lifting mechanism. By using this lifting mechanism, the height of the upper body 38, that is, the height of the robot 10 can be changed. As will be described later, the elevating mechanism is driven by a waist motor (indicated by reference numeral “84” in FIG. 7). The height 100 cm of the robot 10 described above is a value when the upper body 38 is at its lowest position. Therefore, the height of the robot 10 can be 100 cm or more.

  One omnidirectional camera 42 and one microphone 44 are provided in the approximate center of the upper body 38. The omnidirectional camera 42 photographs the surroundings of the robot 10 and is distinguished from an eye camera 64 described later. The microphone 44 captures ambient sounds, particularly human voice.

  Upper arms 48R and 48L are attached to both shoulders of upper body 38 by shoulder joints 46R and 46L, respectively. The shoulder joints 46R and 46L each have three axes of freedom. That is, the shoulder joint 46R can control the angle of the upper arm 48R around each of the X axis, the Y axis, and the Z axis. The Y axis is an axis parallel to the longitudinal direction (or axis) of the upper arm 48R, and the X axis and the Z axis are axes orthogonal to the Y axis from different directions. The shoulder joint 46L can control the angle of the upper arm 48L around each of the A, B, and C axes. The B axis is an axis parallel to the longitudinal direction (or axis) of the upper arm 48L, and the A axis and the C axis are axes orthogonal to the B axis from different directions.

  Forearms 52R and 52L are attached to the tips of upper arms 48R and 48L via elbow joints 50R and 50L, respectively. The elbow joints 50R and 50L can control the angles of the forearms 52R and 52L around the axes of the W axis and the D axis, respectively.

  In the X, Y, X, W axes and the A, B, C, D axes that control the displacement of the upper arms 48R and 48L and the forearms 52R and 52L (FIG. 6), “0 degree” is the home position. In this home position, the upper arms 48R and 48L and the forearms 52R and 52L are directed downward.

  Although not shown, touch sensors are provided on the shoulder portion of the upper body 38 including the shoulder joints 46R and 46L, the upper arms 48R and 48L, and the forearms 52R and 52L, respectively. , It detects whether a person has touched these parts of the robot 10. These touch sensors are also generally indicated by reference numeral 92 in FIG.

  Spheres 54R and 54L corresponding to hands are fixedly attached to the tips of the forearms 52R and 52L, respectively. In place of the spheres 54R and 54L, a “hand” in the shape of a human hand can be used when a finger function is required unlike the robot 10 of this embodiment.

  Although the shape, dimensions, etc. of the robot 10 are set as appropriate, in other embodiments, for example, the upper body 38 includes a front surface, a back surface, a right side surface, a left side surface, a top surface, and a bottom surface. The surface may be formed so that the surface faces obliquely forward. In other words, the width of the front surface may be shorter than the width of the back surface, and the shape of the upper body 38 viewed from above may be a trapezoid. In such a case, the shoulder joints 46R and 46L are attached to the right side surface and the left side surface via left and right support portions whose surfaces are parallel to the left and right side surfaces, respectively. The rotation ranges of the upper arm 48R and the upper arm 48L are restricted by the left and right side surfaces or the surface (attachment surface) of the support portion, and the upper arms 48R and 48L do not rotate beyond the attachment surface. However, if the inclination angle of the left and right side surfaces, the distance between the B axis and the Y axis, the lengths of the upper arms 48R and 48L, the lengths of the forearms 52R and 52L, etc. are appropriately set, the upper arms 48R and 48L will exceed the front. Since it can rotate to the inner side, the arms of the robot 10 can cross forward even if there is no degree of freedom of the arms by the W axis and D axis. Therefore, even when the degree of freedom of the arm is small, close communication such as embracing with a person located in front can be achieved (for example, an action module of “HUG”).

  A head 58 is attached to an upper center of the upper body 38 via a neck joint 56. The neck joint 56 has three degrees of freedom and can be angle-controlled around each of the S, T, and U axes. The S-axis is an axis that goes directly from the neck, and the T-axis and the U-axis are axes that are orthogonal to the S-axis in different directions. The head 58 is provided with a speaker 60 at a position corresponding to a human mouth. The speaker 60 is used for the robot 10 to communicate with the people around it by voice or voice. However, the speaker 60 may be provided in another part of the robot 10, for example, the body.

  The head portion 58 is provided with eyeball portions 62R and 62L at positions corresponding to the eyes. Eyeball portions 62R and 62L include eye cameras 64R and 64L, respectively. The right eyeball part 62R and the left eyeball part 62L may be collectively referred to as the eyeball part 62, and the right eye camera 64R and the left eye camera 64L may be collectively referred to as the eye camera 64. The eye camera 64 captures the video signal by photographing the face of the person approaching the robot 10 and other parts or objects.

  Note that each of the omnidirectional camera 42 and the eye camera 64 described above may be a camera using a solid-state image sensor such as a CCD or a CMOS.

  For example, the eye camera 64 is fixed in the eyeball unit 62, and the eyeball unit 62 is attached to a predetermined position in the head 58 via an eyeball support unit (not shown). The eyeball support unit has two degrees of freedom and can be controlled in angle around each of the α axis and the β axis. The α axis and the β axis are axes set with respect to the head 58, the α axis is an axis in a direction toward the top of the head 58, the β axis is orthogonal to the α axis and the front side of the head 58 It is an axis in a direction orthogonal to the direction in which (face) faces. In this embodiment, when the head 58 is at the home position, the α axis is set to be parallel to the S axis and the β axis is set to be parallel to the U axis. In such a head 58, the tip (front) side of the eyeball unit 62 or the eye camera 64 is displaced by rotating the eyeball support portion around each of the α axis and β axis, and the camera axis, that is, the line-of-sight direction is changed. Moved.

  In the α axis and β axis that control the displacement of the eye camera 64, “0 degree” is the home position. At this home position, the camera axis of the eye camera 64 is the head 58 as shown in FIG. The direction of the front side (face) is directed, and the line of sight is in the normal viewing state.

  The configuration of the control system of the robot 10 shown in FIG. 6 is shown in the block diagram of FIG. As shown in FIG. 7, the robot 10 includes a microcomputer or CPU 66 for overall control. The CPU 66 is connected to a memory 70, a motor control board 72, a sensor input / output board 74, and a voice through a bus 68. An input / output board 76 is connected.

  The memory 70 includes a ROM and a RAM (not shown), and the ROM 10 is pre-stored with a control program for the robot 10, and includes data for a plurality of behavior modules, data for a plurality of episodes and rules, and each behavior module. The voice or voice data to be generated from the speaker 60 when performing the above, angle data for indicating a predetermined gesture, and the like are stored. The RAM is used as a temporary storage memory and can be used as a working memory.

  The motor control board 72 is configured by a DSP (Digital Signal Processor), for example, and controls each axis motor such as each arm, head, and eyeball. That is, the motor control board 72 receives the control data from the CPU 66 and controls the angles of the three motors for controlling the X, Y, and Z axes of the right shoulder joint 46R and the axis W of the right elbow joint 50R. The rotation angles of a total of four motors, one motor, (collectively shown as “right arm motor” in FIG. 7) 78 are adjusted. Further, the motor control board 72 has a total of four motors including three motors for controlling the angles of the A, B and C axes of the left shoulder joint 46L and one motor for controlling the angle of the D axis of the left elbow joint 50L. (In FIG. 4, collectively shown as “left arm motor”.) The rotation angle of 80 is adjusted. The motor control board 72 also adjusts the rotation angle of three motors 82 (collectively shown as “head motors” in FIG. 4) that control the angles of the S, T, and U axes of the head 58. To do. The motor control board 72 also controls a waist motor 84 and two motors 86 (hereinafter collectively referred to as “wheel motors”) 86 that drive the wheels 30.

  Furthermore, the motor control board 72 adjusts the rotation angles of two motors 88 (collectively shown as “right eyeball motor” in FIG. 7) that control the angles of the α axis and β axis of the right eyeball portion 62R. In addition, the rotation angles of two motors 90 that collectively control the angles of the α axis and the β axis of the left eyeball portion 62L (collectively shown as “left eyeball motor” in FIG. 4) 90 are adjusted.

  The above-described motors of this embodiment are stepping motors or pulse motors for simplifying the control except for the wheel motors 86, but may be direct current motors similarly to the wheel motors 86.

  Similarly, the sensor input / output board 74 is also constituted by a DSP, and takes in signals from each sensor and camera and gives them to the CPU 66. That is, data relating to the reflection time from each of the ultrasonic distance sensors 34 is input to the CPU 66 through the sensor input / output board 74. Further, a video signal from the omnidirectional camera 42 is input to the CPU 66 after being subjected to predetermined processing by the sensor input / output board 74 as required. Similarly, the video signal from the eye camera 64 is also supplied to the CPU 66. In FIG. 7, the touch sensors described in FIG. 6 are collectively represented as “touch sensors 92”, and signals from the touch sensors 92 are given to the CPU 66 via the sensor input / output board 74. .

  The synthesized sound data is given to the speaker 60 from the CPU 66 via the sound input / output board 76, and the sound or voice according to the data is outputted from the speaker 60 accordingly. Then, the voice input from the microphone 44 is taken into the CPU 66 via the voice input / output board 76.

  Similarly, the communication LAN board 96 is constituted by a DSP, and sends transmission data sent from the CPU 66 to the wireless device 98 and causes the wireless device 98 to transmit the transmission data. The communication LAN board 96 receives data via the wireless device 98 and supplies the received data to the CPU 66. The communication LAN board 96 and the wireless device 98 allow the robot 10 to perform wireless communication with the development support device 12 or a wired LAN access point, and perform wireless communication with other robots 10 after completion. Can do.

  The robot 10 shows a predetermined gesture by controlling the movement of the left and right arm portions, the head 58, the eyeball unit 62, and the like, and utters by outputting a sound from the speaker 60, and displays the instruction unit of the action module. Execute. Then, the robot 10 uses the camera 42, the eye camera 64, the microphone 44, the touch sensor 92, and the like to execute the action module recognition unit, that is, to recognize a human reaction.

  The development support apparatus 12 (FIG. 1) is useful for developing the robot 10 having the above-described behavior modules and episode rules as basic configurations. The development support apparatus 12 mainly includes a behavior module visualization function, an episode rule search function, and an episode rule input function.

  That is, the development support apparatus 12 has a function of visualizing the execution of a large number of behavior modules and a complex relationship between the behavior modules. And it becomes possible to grasp easily. Further, the development support apparatus 12 has a function of searching for the mounted episode / rule, and based on the search condition, the corresponding episode / rule and the next executable module candidate are selected. Further, the development support apparatus 12 has a function of inputting an episode rule, that is, a function of correcting an existing episode rule and creating a new episode rule. With this input function, the developer can An episode rule that controls the behavior of the robot 10 can be developed. Specifically, it will be particularly useful when completing the action of the robot 12 by partially modifying an existing episode rule or creating a negative episode rule.

  In the development support apparatus 12, when the development support process is started, for example, a main screen as shown in FIG. The main screen includes a visualization screen 100 on which the behavior module relationship is displayed. The visualization screen 100 in FIG. 12 shows an initial state, and rectangular icon images of behavior modules are displayed in an aligned manner.

  In addition, on the right side of the screen in FIG. 12, a module screen or the like on which a list of installed behavior modules, a description of contents, and the like are displayed can be switched between display and non-display by a boundary switching button.

  Furthermore, the main screen includes a search screen 102 on which search results are displayed, for example, as shown in FIG. Note that the search screen 102 can also be switched between display and non-display by a boundary switching button, and the search screen 102 is in a non-display state in FIG.

  On the search screen 102, the sentence of the episode rule corresponding to the search condition is displayed together with the number of hits according to the search result. There are three types of search modes: current search, past search, and character search. The search type can be selected by a current search button 104, a past search button 106, and a character search button 108 provided at the top of the search screen 102. Normally, search is currently selected.

  On the visualization screen 100, the relationship between the behavior modules is graphically visualized and displayed based on a search result corresponding to each search type. That is, each behavior module is iconified, and an icon image of the behavior module is displayed on the visualization screen 100. The arrangement of each icon is set in consideration of the relationship between all action modules. In addition, the display form of each icon is set in accordance with the relationship of each action module with respect to the time of interest for each search mode.

  The visualization screen 100 in FIG. 13 shows an example in the current search mode. The behavior module icons are arranged based on the power of the relationship between the behavior modules. The strength of the relationship between the behavior modules becomes stronger as more relationships (episode rules) linking the behavior modules are defined, and appear closer to each other on the visualization screen 100. The force of the relationship between the two behavior modules is calculated by the following equation (Equation 1) known as the spring model method.

[Equation 1]
F _ij = K _ij · D _ij -R · D _ij ^-7
Here, i and j are each one behavior module, and F _ij , D _ij and K _ij are the force, distance and spring constant acting between the behavior modules i and j, respectively. R is a constant for repulsive force.

The position of each behavior module is calculated by performing convergence calculation so that F _ij is balanced in all behavior modules.

  In addition, the display form of the icon of each behavior module is set, for example, by changing the color, shape, size, and the like according to the relationship to the point of interest for each search mode.

  Since the current search is to investigate the current state, the point of interest is now. Therefore, in the current search, the display form is distinguished by the relationship with the currently executing action module. The “current icon” is used as the icon of the action module currently being executed. For example, the current icon is colored red, rectangular, and large. Also, “past icon” is used as the icon of the action module executed in the past than the current action module. For example, the past icon is colored blue, is rectangular, and is set to a medium size that is smaller than the current icon. Further, the past icon and the current icon are connected by, for example, a yellow connecting line according to the execution order. Further, “next icon” is used as an icon of a candidate for a next executable action module obtained as a result of the search. For example, the next icon is colored green, is rectangular, and is set as large as the current icon. In this example, the behavior module ID (ModuleID) is displayed in the past, present, and next icon frames. In addition, an “irrelevant icon” is used for the icons of the remaining behavior modules that are not directly related to the current behavior module at the time of interest. Irrelevant icons are colored gray, square, and very small.

  In addition, in the visualization screen 100 of FIG.13, 14 and 15, the reference symbol G is attached | subjected to the present icon and the reference symbol K is attached | subjected to the past icon. A rectangular icon without a reference sign is the next icon.

  Since the current search process is the same as the process for determining the next execution module performed by the robot 10, in this embodiment, the execution history of the action module, the corresponding episode / rule, and the data of the next executable module are obtained from the robot 10. It is received as current status information. The action module execution history data includes action modules that have been executed in the past, their result values, and action modules that are currently being executed. Specifically, in this process, the execution history of the behavior module is compared with all episode rules, and the corresponding episode rule is selected, and the next execution is possible based on the selected episode rule. Candidate for a behavioral module is selected. The display form of each behavior module is set according to the search result obtained in this way, that is, the current status information, icons are displayed on the visualization screen 100, and the search screen 102 also includes the corresponding episode / The rules are displayed.

  FIG. 14 shows a state after the robot 10 moves to the next action from the state of FIG. That is, FIG. 14 shows the result of the current search after one next execution module is selected from the next executable module candidates in FIG. 13 and executed. Specifically, the behavior module ("ASOBO") displayed as the current behavior module (reference symbol G) as the current behavior module in FIG. 13 is replaced with the past icon (reference symbol) as one of the past behavior modules in FIG. K), and the behavior module ("T_TSU") displayed as the next executable module in FIG. 13 with the next icon is displayed as the current behavior module with the current icon (reference symbol G) in FIG. In addition, the current icon and the past icon (“ASOBO” and “T_TSU”) that have been changed are newly connected by a connecting line in FIG. Also, a new next executable module candidate in this state is displayed with the next icon. In addition, the search screen 102 is also updated, and a new applicable episode / rule is displayed.

  The past search is a search for a state at a certain point in the past in the execution history up to now. In the past search, a past action module is specified. That is, for example, the “<” button (return button) 110 to be moved to the past side and the “>” button (decimal button) 112 to be moved to the current side are clicked, or the number of the past is directly entered in the search number input field 114. By inputting, it is possible to specify how far in the past. Since the current status information at the past time point has already been acquired in the current search process (received from the robot 10 in this embodiment), the current status information at the specified past time point may be referred to as the past search process. . As a result, data necessary for display can be acquired.

  Since the past search is for investigating a past state in the history, the time point of interest is the designated past time point. Therefore, in the past search, the display form is distinguished by the relationship with the specified past action module. That is, for example, the specified past action module corresponds to the “current action module” in the past search, and the action module executed before the specified past action module is, for example, “executed in the past”. Corresponding to the “action module”. Therefore, in past search visualization, the specified past action module is set as the current icon, the action module executed before the specified past module is set as the past icon, and the next execution obtained as a result of the search is executed. The possible behavior module candidates are set as the next icon, and the remaining behavior modules are set as the irrelevant icons, and each is displayed on the visualization screen 102. Similarly, the search screen 102 displays the corresponding episode / rule.

  The character search is to investigate the state of the robot 10 in an arbitrary situation. In the character search, as shown in FIG. 15, the situation can be designated by directly inputting it into the situation input field 116. Then, a search is performed using the specified status as a search condition. In other words, in the character search process, the input situation is compared with all implemented episode rules, and the corresponding episode rule is selected, and the next executable module candidate is selected based on the corresponding episode rule. Is done. In FIG. 15, “<TURN = 1> <AKUSHU = 1>” is entered in the input field 116 as the designation status.

  In the character search mode, action module candidates that can be executed next to the designated situation are selected, and the attention time point is the time of the last action module in the designated situation. Therefore, in the character search, the display form is distinguished by the relationship with the last action module in the specified situation. In other words, for example, the last action module in the specified situation is equivalent to the “current action module”, and the action module before the last action module in the input situation is equivalent to the “action module executed in the past”. To do. Therefore, in the visualization of the character search, the last action module (in this example, “AKUSHU”) in the input situation is set to the current icon (reference symbol G), and the action module other than the last in the input situation ( In this example, “TURN”) is set as the past icon (reference symbol K), the next executable module candidate obtained as a result of the search is set as the next icon, and the remaining behavior modules are set as the irrelevant icons. Each is displayed on the visualization screen 100. Similarly, the corresponding episode / rule is displayed on the search screen 102.

  A control screen is displayed on the right side of the screen in FIG. In this control screen, various settings related to visualization can be made. For example, display of behavior module IDs on icons, enlarged display, position calculation, search episode / rule display conditions, and the like can be set.

  In this way, the developer can visually grasp the communication state and transition in real time while actually causing the robot 10 to take action in the current search mode. Further, the past search mode allows the robot 10 to look back and confirm the actions that have been performed so far. In addition, by using the character search mode, it is possible to check the episode / rule corresponding to the situation to be investigated from all the episode / rules. Using such visualization results, the developer opens the input screen when he / she feels the necessity for correction or change due to, for example, inconsistencies or inconsistencies in the transition of the behavior of the robot 10. By correcting the already-registered episode rule or creating a new episode rule, the robot 10 can be finished as a whole in a harmonious manner.

  The input screen is a window that is different from the main screen by selecting the episode / rule displayed on the search screen 102 by, for example, double-clicking, or selecting the “Create Episode” button 118 provided on the search screen 102. Is displayed. With this input screen, the user can perform episode / rule editing work (correction, new creation, deletion, etc.).

  The input screen includes, for example, an episode / rule sentence display field 120, an editing screen 122, and an operation screen 124 as shown in FIG. The episode / rule sentence display field 120 includes a registration field 120a and an edit field 120b. The registration field 120a displays an already registered episode / rule sentence selected for editing, and the edit field 120b The episode / rule sentence in the middle state is displayed. The editing screen 122 graphically displays the episode / rule being edited.

  FIG. 16 shows an example of the input screen when the episode rule “<MANUAL = 255> CHARGE1” with the ID number 451 is selected. As can be seen from the edit column 120b, an action module frame is added between “<MANUAL = 255>” and “CHARGE1”. On the editing screen 122, the action module or its block is displayed as a square icon, which is connected by an arrow icon, and the structure of the episode rule can be easily grasped by such visualization.

  The operation screen 124 is provided with various operation buttons for editing episodes and rules. For example, the past insertion button 126 can additionally insert a behavior module frame (“<>”) on the past side of the selected behavior module, and the next insertion button 128 can select the selected behavior module. The following action module frame can be additionally inserted for. Further, according to the expansion button 130, when the icon selected on the editing screen 122 is a block, the configuration in the block can be expanded and displayed in a row below (FIG. 17). As shown in FIG. 17, the expanded block is displayed as an open book icon in the original column. The delete button 132 can delete the selected action module. The block button 134 can block the selected action module ("(...)").

  Further, the operation screen 124 is provided with a reference / explanation column in which a list of IDs of the installed behavior modules and the result values thereof are displayed together with their contents explanation. Also, editing setting fields such as block and negation of the result value ("^") and block repetition are provided. In this edit setting field, the designated contents are entered in the action module frame selected on the edit screen 122 by appropriately specifying the action module, result value, block and result value negation, block repetition, etc. Can do. In addition, when an action module frame or block for which an action module and a result value have already been input is selected, their IDs and contents are displayed in the setting column. Although not shown in the operation screen 124, negative ("!") Episode rules can be specified, and priorities can be set.

  By operating such an operation screen 124, the edited episode / rule sentence is displayed in the edit column 120b. By selecting the new registration button 136, the created episode rule can be newly registered. The correction registration button 138 allows the edited episode / rule to be registered by replacing the existing episode / rule. In addition, the registration deletion button 140 can delete the registration of the selected episode / rule displayed in the registration field 120a. In this embodiment, data relating to editing such as new episode / rule registration, modification registration or registration deletion is created as editing information (input information) and transmitted to the robot 10. Then, the robot 10 processes and stores the episode rule according to the input information. The input screen can be closed by the end button 142.

  Specifically, the CPU 14 of the development support apparatus 12 executes development support processing according to the flowcharts shown in FIGS.

  When the process is started, the CPU 14 first reads the activation instruction data from the HDD 20 and supplies it to the communication circuit 26 in step S1. As a result, the activation instruction data is transmitted to the robot 10. The robot 10 under development is activated according to the instruction data and starts its activity. The robot 10 operates according to the flowcharts shown in FIGS. 10 and 11, for example. The operation of the robot 10 will be described later.

  Next, in step S <b> 3, the CPU 14 reads main screen data, behavior module data, and the like from the HDD 20 and displays a main screen as shown in FIG. 12 on the display device 22, for example. In the visualization screen 100 of FIG. 12, as an initial state, all the behavior modules are arranged vertically and horizontally regardless of the strength of the relationship.

  In subsequent steps S5, S7 and S9, the type of search mode is determined. In the initial state, the search mode is currently selected. The search mode can be selected by each search button 104, 106 and 108 as shown in FIG. If “NO” in steps S5 to S9, that is, if another button is selected, the process proceeds to step S29 in FIG.

  If the current search button 104 is selected, “YES” is determined in the step S5, and it is determined whether or not the current state information data is transmitted from the robot 10 in a step S11. Since the current search process is the same as the next action module selection process (steps S75 and S77 in FIG. 10) executed in the robot 10, in this embodiment, the module obtained by the process on the robot 10 side is executed. The history, the corresponding episode / rule, and the candidate for the next executable module are transmitted from the robot 10 as current information, and the development support apparatus 12 omits the current search process. If “NO” in the step S11, the process returns to the step S5. If “YES” in the step S11, the received current status information is stored in the HDD 20 in a succeeding step S13. When the process of step S13 is completed, the process proceeds to step S19.

  On the other hand, if the past search button 106 is selected, “YES” is determined in the step S7, and the past search process is executed in a subsequent step S15. Specifically, in the past search, for example, the past time point is specified by the number input in the return button 110, the advance button 112, or the search number input field 114. Obtained from the current status information stored in the HDD 20. Note that the past search is for examining and confirming the actions performed by the robot 10, and therefore, when there is no current information at the specified time, for example, an error display is performed. When the process of step S15 is completed, the process proceeds to step S19.

  On the other hand, if the character search button 108 is selected, “YES” is determined in the step S9, and the character search process is executed in a subsequent step S17. Specifically, in the character search, since a situation to be investigated is designated in the situation input field 116, a retrieval process using the designated situation as a search condition is executed. That is, the specified situation and the episode rule are compared to select the corresponding episode rule, and the next action module specified in the selected episode rule can be executed next to the input situation. Selected as an action module candidate. When the process of step S17 is completed, the process proceeds to step S19.

  When the process of step S13, S15 or S17 is completed, the setting of the icon display form of each behavior module is processed in the subsequent step S19.

  Here, in the case of the current search mode, the current state information stored in step S13 is referred to, and the display form is set based on the current state information. In the case of the past search mode, the current state at the specified time acquired in step S15. Refer to the information and make settings based on this. In the character search mode, before setting the display mode, the last action module in the specified situation is set as the “current action module”, and the action modules other than the last action module in the specified situation are set. Set as “Past Action Module”. In the display mode setting, the current icon is set for the current action module, the past icon is set for the past action module, and the next icon is set for the next executable module candidate. In addition, irrelevant icons are set for the remaining behavior modules. Further, connection lines are set between past icons and between past icons and current icons according to the execution order.

  Subsequently, in step S21, it is determined whether or not an action module arrangement calculation process is necessary. Arrangement calculation is required for the first display and for the first display after episode rule update (new registration, modification registration or deletion). That is, when the episode / rule is updated, the relationship between the behavior modules is changed, so that the arrangement may be changed. Therefore, in these cases, “YES” is determined in the step S21, the arrangement calculation is processed in a step S23, and then the process proceeds to the step S25.

In step S <b> 23, the placement of each behavior module is calculated based on the strength of the relationship between all the behavior modules installed, and the placement data is stored in the HDD 20. As described above, the position of each behavior module is calculated by performing convergence calculation so that F _ij represented by Equation 1 is balanced in all behavior modules.

  On the other hand, if “NO” in the step S21, that is, if the arrangement calculation is not necessary, the arrangement data already calculated may be used, and the process proceeds to the step S25 as it is.

  Subsequently, in step S25, the sentence of the corresponding episode / rule data obtained as a result of the search, the number of hits, and the like are displayed on the search screen 102. In step S27, the icons of the behavior modules are displayed on the visualization screen 100 according to the arrangement calculated in step S23 and the display form set in step S19. Through these processes, the search results in the respective search modes are displayed on the search screen 102 and the visualization screen 100 of the main screen.

  For example, in the current search, the current state of the robot 10 as shown in FIG. 13 is shown. Then, in the robot 10, the execution of the currently executing action module is finished, the action module to be executed next is selected from the candidates for the next executable module, and when the execution is started, a new current state is obtained from the robot 10. Information is sent. In the development support apparatus 12, the visualization process from step S11 is repeated, and for example, visualization based on the following current status information as shown in FIG. 14 is performed. As described above, in the current search, the communication behavior of the robot 10, that is, the behavior history up to now, the current behavior, the candidate for the next behavior module to be executed, the episode / rule corresponding to the current state, etc. are grasped in real time. can do. Therefore, in the current search, the actual behavior of the robot 10 can be observed directly and through the screen, and the consistency and harmony of the behavior of the robot 10 can be comprehensively determined.

  In the past search, details of the action of the robot 10 executed so far can be confirmed afterwards. That is, for each past action module or time point, the history up to that point, the action module currently being executed, the action module candidate that can be executed next, and the episode rule that corresponds to the state at that time are visually reviewed. Can be grasped.

  In the character search, a search result that matches the search condition is visualized for each input specified situation (FIG. 15). Therefore, it is possible to check the episode rule implementation state of the robot 10 under development, that is, what state the robot 10 under development is in for a specific situation designated.

  In this way, the developer can determine the necessity of editing the episode rule implemented in the robot 10 under development, for example. For example, since the arrangement of each behavior module is calculated on the visualization screen 100 based on the strength of the relationship, based on the distance and direction between the past behavior module, the current behavior module, and the next behavior module, etc. The suitability of the action of the robot 10 in the situation may be determined.

  If the process of step S27 is completed or if “NO” in steps S5, S7, and S9, the process proceeds to step S29 of FIG.

  In step S29, it is determined whether episode rule input (edit) has been selected. That is, for example, when an episode / rule sentence on the search screen 102 is selected or when the episode / rule creation button 118 is selected, “YES” is determined in the step S29, and the input is performed in the next step S31. Data on the screen (FIG. 16) is read from the HDD 20 and displayed on the display device 22. As a result, the input screen is displayed as a separate window on the main screen, and the developer can edit episodes and rules while referring to the visualization screen 100.

  On this input screen, as described above, the developer can edit episodes and rules. In other words, for example, the action module frame is inserted or blocked as appropriate, and the action module, result value, negation, repetition, etc. are set, and new episode rules are created, modified, or deleted. Can do. When there is such an episode / rule input, the subsequent processing from step S33 is executed. In step S33, each episode / rule editing process is executed in accordance with various operation buttons 126 to 134 provided on the developer's input screen and operations on settings and the like.

  Subsequently, when editing is registered, that is, when the new registration button 136, the correction registration button 138, and the registration deletion button 140 are selected, “YES” is determined in the step S35, and a step S37. Proceed to On the other hand, if there is no registration, “NO” is determined in the step S35, and the process returns to the step S5 in FIG. 8 to repeat the process.

  In step S37, the stored episode rule data is updated according to the edited episode rule information. In the case of new registration, newly created episode rule data is added. In the case of correction registration, the original episode rule is replaced with the corrected episode rule data, and in the case of registration deletion, the episode rule data is deleted. In step S39, the editing information data is provided to the communication circuit 26 and transmitted to the robot 10. The editing information includes, for example, information on new, correction or deletion, and an identifier of an episode rule, and further includes an episode rule created in the case of new and correction. The robot 10 receives this editing information and updates the data of the episode rule stored in the storage device accordingly. Therefore, after that, the robot 10 performs processing based on the edited episode rule.

  If “NO” in the step S29, it is determined whether or not the process is ended in a step S41. That is, if an end menu (not shown) on the main screen is selected, it is determined to be “YES”, an end process is executed in the subsequent step S43, and this development support process is ended. In this end process, the end instruction data may be transmitted to the robot 10 to end the operation of the robot 10.

  According to the development support apparatus 12, it is possible to visually grasp the relationship between a large number of modules. Therefore, for example, an existing episode rule can be corrected or deleted, or a new episode rule can be created so as to realize a harmonized communication behavior with no contradiction in the situation.

  In the above-described embodiment, the current search process is the same process as the next action module selection process (steps S75 and S77 in FIG. 10) executed in the robot 10, and therefore the data obtained by this process is used. Therefore, the necessary data is transmitted from the robot 10 as the current status information, but it is needless to say that the current search process may be performed on the development support apparatus 12 side.

  Needless to say, each screen shown in the above-described embodiment is an example and can be changed as appropriate.

  The robot 10 developed in this way has a plurality of behavior modules and a plurality of episode rules, and the actual action taken, that is, the execution history of the behavior module, the implemented episode rules, and By comparing these, the action module to be executed next is selected. In addition, the robot 10 under development can receive the episode / rule editing information transmitted from the development support apparatus 12 and update the mounted episode / rule data according to the editing information. The development support apparatus 12 also stores current status information for the current search and past search in the development support apparatus 12, that is, data related to action module execution history, episode rules corresponding to the current state, and action module candidates that can be executed next. 12 can be transmitted.

  Specifically, the robot 10 executes processing according to flowcharts shown in FIGS. 10 and 11, for example. When the robot 10 is activated, first, in step S61, a predetermined behavior module to be performed first is read from the memory 70 and selected. As this first action module, for example, a module that investigates the surrounding environment such as “EXPLORE” described above may be set.

  Next, in step S63, execution of the selected behavior module is started. That is, the data of the selected action module is read from the memory 70, and first, the action specified by the instruction unit (FIG. 3) is performed. For example, in the case of an action module accompanied by utterance, audio data is given to the audio input / output board 76. In the case of an action module with gesture, motor control data for performing the action is given to the motor control board 72. As a result, the sound is output from the speaker 60, and each motor is driven to perform its action (gesture). Then, the recognition unit of the action module is executed to recognize a human reaction to the action of the robot 10 or to achieve the predetermined action in the case of a self-contained action that does not interact with a human being. I try to recognize it. That is, information obtained from each sensor or microphone 44 or a timer (not shown) or the like is analyzed to determine whether or not a result matching the expected result value defined in the recognition unit has been recognized. In addition, when an interruption or deviation that matches the operation of the reactive module (FIG. 4C) is recognized, the reactive module is executed.

Note that when the robot 10 recognizes the predicted result defined in the recognition unit, the robot 10 moves from the action module being executed to the next action module, and these processes are performed in step S81 and subsequent steps. In the robot 10 of this embodiment, the execution of the behavior module is started in step S63, and the processing from step S65 to step S81 is performed in parallel. That is, in step S65, it is determined whether or not the processing is finished. For example, if development is in progress, it is determined whether or not there has been an end instruction from the development support apparatus 12 or whether or not an end button for stopping the robot 10 has been pressed. If “YES” in the step S65, an end process is executed in a subsequent step S67, and the operation process of the robot 10 is ended. In this end process, each part of the body of the robot 12 may be returned to the home position.

  If “NO” in the step S65, it is determined whether or not the editing information from the development support apparatus 12 is transmitted through the wireless LAN or the like in a step S69. As described above, the development support apparatus 12 can edit an episode / rule on the input screen, and the edited episode / rule data is transmitted. If “YES” in the step S69, the editing information is received and written in the memory 70 in a succeeding step S71, and the data of the episode rule stored in the memory 70 is updated in accordance with the editing information in a step S73. . That is, in the case of newly registered data, the episode rule is newly added to the episode rule data stored in the memory 70. In the case of the correction registration data, the corresponding episode rule among the episode rules stored in the memory 70 is replaced with the received data of the corrected episode rule. In the case of registered deletion data, the deletion target episode / rule is deleted from the episode / rule data stored in the memory 70. Through the processing of steps S69 to S73, the robot 10 implements (stores) the edited episode / rule data, and thereafter, the processing is executed based on the edited episode / rule.

  If “NO” in the step S69, a next executable action module candidate selection process is executed in the steps S75 and S77. Specifically, in step S75, the execution history of the behavior module so far is stored in the memory 70 as an episode. The data of the execution history of the episode, that is, the behavior module includes the behavior module that has been executed in the past, the data of the result values thereof, and the data of the behavior module that is currently being executed. In the subsequent step S77, the stored episode is compared with all episode rules implemented in the memory 70, and an episode rule corresponding to this episode is selected, and the selected episode rule is selected. The specified action module is selected as a candidate of the next executable action module. In this manner, the next possible behavior module candidate is selected during the execution of the current behavior module.

  In subsequent step S79, the current status information data for visualization under development is given to the communication LAN board 96 and transmitted to the development support apparatus 12 through the wireless apparatus 98. As described above, this process is for using the data obtained by the process for selecting the next action of the robot 10 for visualization of the development support apparatus 12. In this embodiment, the current status information includes the action module execution history, the selected episode / rule, and the next executable action module candidate data. The development support apparatus 12 receives the current status information and performs visualization in each search mode as described above.

  In step S81, it is determined whether to end the execution of the behavior module. That is, it is determined whether or not a result that matches the expected result value defined in the recognition unit of the behavior module is recognized by various sensors and the speaker 60 or the like. If “YES” in the step S81, the process proceeds to a step S83 in FIG. 11 to execute a process of selecting a behavior module to be executed next.

  In step S83, the recognized result value is stored in the memory 70. This result value is used for the execution history data of the behavior module.

  In the subsequent step S85, the episode module that satisfies the recognized result value is selected from the episode rule and the next executable action module candidate selected in step S77, and the behavior module that satisfies the preconditions is selected. Is elected. The episode rule is basically defined as the action module to be executed when the currently executing action module ends with a predetermined result value (only “success” in episode rule 2 in FIG. 5). It is necessary to determine the appropriateness of the selected episode rule based on the result value. In addition, since the precondition includes whether or not the Internet connection as described above is possible, it is also determined whether or not such a condition on the equipment is satisfied. Further, for example, in the behavior module “TURN” of FIG. 5, it is a precondition that a person touches the shoulder, so the precondition is also determined from the recognized result value.

  In step S87, it is determined whether there are any episode rule and behavior module candidates that satisfy the result value and the precondition. If “YES” in the step S87, a predetermined behavior module is read from the memory 70 and selected in a succeeding step S89. That is, the robot 10 is an action module that is executed when there is no action module that satisfies the result value or the precondition among the candidates for the next executable action module selected from the comparison with the execution history of the action module. have. In such a behavior module, for example, a specific behavior module that realizes a “stop state” is set. When the process of step S89 ends, the process proceeds to step S95.

  If “NO” in the step S87, that is, if there is a satisfactory behavior module, it is determined whether or not a plurality of episode rules conflict in a succeeding step S91. That is, it is determined whether there are a plurality of applied episode rules. If “YES” in the step S91, an episode rule to be applied is selected based on the priority order of the episode rules in a step S93, and an action to be executed next based on the episode rule having the higher priority order. Select a module. When the priority order of competing episode rules is the same, the episode rules to be applied are selected at random, for example. If the high priority episode rule is a negative episode rule and the execution of the next action module of the competing episode rule is denied, the action to be selected is the same as in step S87. Since this corresponds to the case where there is no module, a predetermined behavior module to be executed in such a case is selected in the same manner as in step S89.

  If “NO” in the step S91, that is, if the episode rule does not conflict, or if the process of the step S89 or S93 is finished, the selected action module is executed next in the step S95. Select (determine) as the action module to perform. When the process of step S95 is completed, the process returns to step S63 of FIG. 10, and the process is repeated. Therefore, in the robot 10, the execution of the next behavior module selected in step S95 is started. In this way, the robot 10 can communicate with humans by sequentially executing action modules that are consistent with the execution history.

  According to the robot 10 of this embodiment, the action module to be executed next is selected and executed based on the actual action module execution history, that is, the communication history between the robot 10 and the human. Therefore, it does not become a regular monotonous action, and it is possible to take a consistent communication action in harmony with the current state of the robot 10.

  Further, the communication robot 10 having a large-scale behavior pattern can be realized by sequentially adding new behavior modules and episode rules using, for example, the development support device 12.

  FIG. 10 shows a flowchart including the process under development. However, in the completed robot 10 developed using this development support apparatus 12, for example, a process necessary only during development (step S69). To S73 and S79) may not be processed or provided.

It is an illustration figure which shows the development assistance apparatus and communication robot of one Example of this invention. It is a block diagram which shows the internal structure of the development assistance apparatus of FIG. 1 Example. It is an illustration figure which shows the structure of the action module with which the robot of FIG. 1 Example is provided. It is an illustration figure which shows the transition pattern of the action module of FIG. 1 Example. It is an illustration figure which shows an example of the transition of the action module of the robot of FIG. 1 embodiment, and an applied episode rule. It is an illustration figure (front view) which shows the external appearance of the robot of FIG. 1 Example. It is a block diagram which shows the internal structure of the robot of FIG. 1 Example. It is a flowchart which shows a part of operation | movement of the development assistance apparatus of FIG. 1 Example. It is a flowchart which shows a part of other operation | movement of the development assistance apparatus of FIG. 1 Example. It is a flowchart which shows a part of operation | movement of the robot of FIG. 1 Example. It is a flowchart which shows a part of other operation | movement of the robot of FIG. 1 Example. It is an illustration figure which shows an example of the initial screen of the main screen displayed on the display apparatus of the development assistance apparatus of FIG. 1 Example. FIG. 13 is an illustrative view showing one example of a screen in a current search mode of the main screen of FIG. 12 embodiment; FIG. 14 is an illustrative view showing one example of a screen after shifting from the main screen to the next behavior module in the embodiment in FIG. 13; It is an illustration figure which shows an example of the screen in the character search mode of the main screen of FIG. 12 Example. It is an illustration figure which shows an example of the input screen displayed on the display apparatus of the development assistance apparatus of FIG. 1 Example. FIG. 17 is an illustrative view showing another example of the input screen of the embodiment in FIG. 16;

Explanation of symbols

10 ... Communication robot 12 ... Communication robot development support device 14 ... CPU
18 ... Memory 20 ... HDD
22 ... Display device 24 ... Input device 26 ... Communication circuit 66 ... CPU
70 ... Memory 72 ... Motor control board 74 ... Sensor I / O board 76 ... Voice I / O board

Claims (7)

A plurality of behavior modules that modularize the behavior in a specific situation, and a plurality of rules that define a basic execution order of the behavior modules, and the communication behavior by executing the plurality of behavior modules based on the plurality of rules A communication robot that takes
Behavior module storage means for storing the plurality of behavior modules;
Rule storage means for storing the plurality of rules;
Candidate selection means for comparing the execution history of the behavior module and the plurality of rules to select the rule corresponding to the execution history and the candidate of the next executable behavior module;
A next action selecting means for selecting the next action module to be executed next from the candidates selected by the candidate selecting means; and an action module executing means for executing the action module selected by the next action selecting means, Communication robot. Each of the plurality of rules has a priority order,
The communication robot according to claim 1, wherein the next action selection unit includes a priority selection unit that selects the rule to be applied based on each priority when the plurality of applicable rules conflict. A plurality of behavior modules obtained by modularizing behavior in a specific situation, and a plurality of rules that define a basic execution order of the plurality of behavior modules, and executing the plurality of behavior modules based on the plurality of rules. A communication robot development support device for developing a communication robot that takes communication action,
Screen display means for displaying a visualization screen for visualizing relationships between the plurality of behavior modules;
An arrangement calculating means for calculating the arrangement of the images representing the plurality of behavior modules on the visualization screen based on the strength of the relationship between the plurality of behavior modules;
A search means for comparing the specified search situation with the plurality of rules and selecting the rule corresponding to the search situation;
Display mode setting means for distinguishing and setting the display mode of the images of the plurality of behavior modules based on the search status and a search result by the processing of the search means, and the calculated arrangement and the set display A communication robot development support device comprising behavior module display means for displaying the images of the plurality of behavior modules on the visualization screen based on a form.   The communication robot development support device according to claim 3, further comprising search result display means for displaying the rule selected by the search means.   The communication robot development support apparatus according to claim 3, further comprising execution information receiving means for receiving information related to execution of the behavior module in the communication robot.   The communication robot development support apparatus according to claim 3, further comprising an input unit for editing the rule.   The communication robot development support device according to claim 6, further comprising edit information transmission means for transmitting the rule edit information created by the input means to the communication robot.

Images