EP1669172B1 - Steuersystem für kommunikationsroboter - Google Patents

Steuersystem für kommunikationsroboter Download PDF

Info

Publication number
EP1669172B1
EP1669172B1 EP04771835.8A EP04771835A EP1669172B1 EP 1669172 B1 EP1669172 B1 EP 1669172B1 EP 04771835 A EP04771835 A EP 04771835A EP 1669172 B1 EP1669172 B1 EP 1669172B1
Authority
EP
European Patent Office
Prior art keywords
behavior
robot
emotional
communication robot
emotional expression
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Lifetime
Application number
EP04771835.8A
Other languages
English (en)
French (fr)
Other versions
EP1669172A4 (de
EP1669172A1 (de
Inventor
Takayuki ADVANCED TELECOMM. RESEARCH INST. KANDA
Hiroshi ADVANCED TELECOMM. RESEARCH INST ISHIGURO
Takahiro ADVANCED TELECOMM. RESEARCH MIYASHITA
Kiyoshi ADVANCED TELECOMM. RESEARCH INST. KOGURE
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
ATR Advanced Telecommunications Research Institute International
Original Assignee
ATR Advanced Telecommunications Research Institute International
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by ATR Advanced Telecommunications Research Institute International filed Critical ATR Advanced Telecommunications Research Institute International
Publication of EP1669172A1 publication Critical patent/EP1669172A1/de
Publication of EP1669172A4 publication Critical patent/EP1669172A4/de
Application granted granted Critical
Publication of EP1669172B1 publication Critical patent/EP1669172B1/de
Anticipated expiration legal-status Critical
Expired - Lifetime legal-status Critical Current

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J13/00—Controls for manipulators
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Programme-controlled manipulators
    • B25J9/16—Programme controls
    • B25J9/1656—Programme controls characterised by programming, planning systems for manipulators
    • B25J9/1671—Programme controls characterised by programming, planning systems for manipulators characterised by simulation, either to verify existing program or to create and verify new program, CAD/CAM oriented, graphic oriented programming systems
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Programme-controlled manipulators
    • B25J9/16—Programme controls
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/004—Artificial life, i.e. computing arrangements simulating life
    • G06N3/008—Artificial life, i.e. computing arrangements simulating life based on physical entities controlled by simulated intelligence so as to replicate intelligent life forms, e.g. based on robots replicating pets or humans in their appearance or behaviour

Definitions

  • the present invention relates to a control system for communication robot, an action input support method an action input support program, and a storage medium storing a program.
  • communication robots intended to communicate with a person by voice and gesture are required to perform not only fixed spontaneous motions for carrying out communications but also reactive motions in response to behavior of a person as a communication partner in order to realize more natural communications.
  • a person has emotions and may exhibit the same action in a different manner depending on his emotions.
  • the communication robots are also needed to make emotional motions.
  • the prior art as disclosed in document JP-A-2002120174 , accepts only input of fixed spontaneous motions, and there exists no system which supports input of actions to be performed by a robot including reactive motions according to behavior of a person and emotional motions.
  • a control system for communication robot supports input of interactive actions to be performed by a communication robot, and comprises a storage means for storing information on a plurality of behaviors including a reflex behavior to be performed in response to behavior of a person and a spontaneous behavior to be performed in a spontaneous manner, a display means for displaying a list of the plurality of behaviors in a user-selectable manner based on the information stored in the storage means, a behavior decision means for deciding a behavior to be performed by the communication robot from the list of behaviors displayed by the display means according to a user's operation, and a generation means for generating reproductive motion information for interactive actions to be performed by the communication robot, based on a history of the behavior decided by the behavior decision means.
  • the information on the plurality of behaviors including a reflex behavior (reactive motion) responsive to behavior of a person as a communication partner as well as a spontaneous behavior, is pre-stored in the storage means.
  • the display means Based on the information on the plurality of behaviors prepared in advance, displays the list of the plurality of behaviors in a user-selectable manner. That is, this list presents the reflex behavior as well in a user-selectable manner.
  • the user can select from the displayed list of behaviors a behavior to be performed by the communication robot.
  • the behavior decision means decides a behavior to be performed by the communication robot from the list of behaviors according to the user's operation.
  • the generation means generates the reproductive motion information for interactive actions to be performed by the communication robot, based on the historical record of the decided behavior. Therefore, according to the present invention of claim 1, by selecting a behavior to be performed by the communication robot from the list of behaviors including the reflex behavior, the user can easily input each of behaviors forming interactive actions and generate the reproductive motion information of the interactive actions including the reflex behavior.
  • a control system for communication robot depends on the present invention of claim 1.
  • the display means further displays a list of a plurality of emotional expressions in a user-selectable manner.
  • the behavior decision means further decides an emotional expression which is to be added to the behavior to be performed by the communication robot, from the list of emotional expressions according to the user's operation.
  • the generation means generates reproductive motion information based on the historical records of the behavior and emotional expression decided by the behavior decision means.
  • the list of plurality of emotional expressions is further displayed by the display means as well as the list of behaviors.
  • the user can select an emotional expression desired to be added to the behavior.
  • the behavior decision means decides the behavior to be performed by the communication robot and further the emotional expression to be added to the behavior according to the user's operation.
  • the generation means generates the reproductive motion information based on the historical records of the decided behavior and emotional expression.
  • a control system for communication robot depends on the present invention of claim 2.
  • the behavior decision means further includes a determination means for determining whether or not the emotional expression selected by the user is appropriate to the selected behavior, and does not permit the emotional expression to be added to the behavior if the determination means determines that the emotional expression is not appropriate to the behavior.
  • the determination means determines whether or not the emotional expression selected by the user is appropriate to the selected behavior.
  • the emotional expression is added to the behavior in accordance with the determination result. That is, if it is determined that the emotional expression is inappropriate to the behavior, the addition of the emotional expression to the behavior is not permitted. Consequently, according to the present invention of claim 3, it is possible to prevent a contradiction between a behavior as an emotional expression and an additional emotional expression.
  • a control system for communication robot depends on any one of the present inventions of claims 1 to 3 and further comprises a transmission means for, when the behavior decision means has decided the behavior to be performed by the communication robot, transmitting an execution instruction for the behavior to the communication robot.
  • the transmission means transmits the execution instruction for the decided behavior to the communication robot. According to the execution instruction, the communication robot performs the decided behavior. Therefore, the user can check the actual motion of the communication robot and thus easily produce the interactive actions.
  • the display means displays the list of behaviors classified by region of the communication robot.
  • the list of behaviors classified by region is displayed, which makes it possible to present the plurality of behaviors to the user in an easy-to-understand manner.
  • the user can easily input a desired behavior.
  • a control system for communication robot according to the present invention of claim 6, depends on any one of claims 1 to 5.
  • the display means displays an image of an appearance of the communication robot performing the behavior.
  • An action input support program is an action input support program for supporting input of interactive actions to be performed by a communication robot, on a control system for communication robot comprising a storage means for storing information on a plurality of behaviors including a reflex behavior to be performed in response to behavior of a person and a spontaneous behavior to be performed in a spontaneous manner by the communication robot.
  • This program causes a processor of the control system for communication robot to execute a display step of displaying a list of the plurality of behaviors in a user-selectable manner based on the information stored in the storage means, a behavior decision step of deciding a behavior to be performed by the communication robot from the list of behaviors displayed by the display step according to the user's operation, and a generation step of generating reproductive motion information for interactive actions to be performed by the communication robot, based on a history of the behavior decided by the behavior decision step.
  • An action input support program depends on the present invention of claim 7.
  • the display step further displays a list of a plurality of emotional expressions in a user-selectable manner.
  • the behavior decision step further decides an emotional expression which is to be added to the behavior to be performed by the communication robot, from the list of emotional expressions according to the user's operation.
  • the generation step generates reproductive motion information based on the history of the behavior and emotional expression decided by the behavior decision step.
  • the behavior decision step further includes a determination step of determining whether or not the emotional expression selected by the user is appropriate to the selected behavior, and does not permit the emotional expression to be added to the behavior if the determination step determines that the emotional expression is not appropriate to the behavior.
  • a storage medium storing a program according to the present invention of claim 10 is a storage medium storing a program for supporting input of interactive actions to be performed by a communication robot, on a control system for communication robot comprising a storage means for storing information on a plurality of behaviors including a reflex behavior to be performed in response to behavior of a person and a spontaneous behavior to be performed in a spontaneous manner by the communication robot.
  • the program causes a processor of the control system for communication robot to execute a display step of displaying a list of the plurality of behaviors in a user-selectable manner based on the information stored in the storage means, a behavior decision step of deciding a behavior to be performed by the communication robot from the list of behaviors displayed by the display step according to the user's operation, and a generation step of generating reproductive motion information for the interactive actions to be performed by the communication robot, based on a history of the behavior decided by the behavior decision step.
  • a storage medium storing a program according to the present invention of claim 11 depends on the present invention of claim 10.
  • the display step further displays a list of a plurality of emotional expressions in a user-selectable manner.
  • the behavior decision step further decides from the list of emotional expressions an emotional expression which is to be added to the behavior to be performed by the communication robot, according to the user's operation.
  • the generation step generates reproductive motion information based on the history of the behavior and emotional expression decided by the behavior decision step.
  • the behavior decision step further includes a determination step of determining whether or not the emotional expression selected by the user is appropriate to the selected behavior, and does not permit the emotional expression to be added to the behavior if the determination step determines that the emotional expression is not appropriate to the behavior.
  • An action input support method is an action input support method for supporting input of interactive actions to be performed by a communication robot, on a control system for communication robot comprising a storage means for storing information on a plurality of behaviors including a reflex behavior to be performed in response to behavior of a person and a spontaneous behavior to be performed in a spontaneous manner by the communication robot.
  • This method includes a display step of displaying a list of the plurality ofbehaviors in a user-selectable manner based on the information stored in the storage means, a behavior decision step of deciding a behavior to be performed by the communication robot from the list of behaviors displayed by the display step according to a user's operation, and a generation step of generating reproductive motion information for interactive actions to be performed by the communication robot, based on a history of the behavior decided by the behavior decision step.
  • An action input support method depends on the present invention of claim 13.
  • the display step further displays a list of a plurality of emotional expressions in a user-selectable manner.
  • the behavior decision step further decides an emotional expression which is to be added to the behavior to be performed by the communication robot, from the list of emotional expressions according to the user's operation.
  • the generation step generates reproductive motion information based on the history of the behavior and emotional expression decided by the behavior decision step.
  • the behavior decision step further includes a determination step of determining whether or not the emotional expression selected by the user is appropriate to the selected behavior. If the determination step determines that the emotional expression is not appropriate to the behavior, the addition of the emotional expression to the behavior is not permitted.
  • the present invention of claims 7 to 9, the present invention of claim 10 to 12, or the present invention of 13 to 15 makes it possible to easily input and generate the reproductive motion information of interactive actions including a reflex behavior, as in case of the above described present invention of claims 1 to 3.
  • the list of the plurality of behaviors including the reflex behavior in response to behavior of a person is displayed so as to allow the user to make a selection, and thus it is possible to easily input and generate interactive actions to be performed by the communication robot. Since the generated interactive actions include not only a spontaneous behavior but a reflex behavior as well, it is possible to initiate the reflex behavior according to behavior of a communication partner. Consequently, this makes it easy to make the communication robot realize more natural, diverse interactive actions.
  • the list of the emotional expressions to be added to the behaviors is further displayed together with the list of the behaviors for the user to make a selection, it is possible to easily input an emotional motion and also easily input and generate emotional interactive actions. Accordingly, it is possible to cause the communication robot to realize with ease more diverse interactive actions including emotional reactive motions and emotional spontaneous behaviors.
  • a control system for communication robot (hereinafter also referred to as just “system") 10 of this embodiment is for supporting input of interactive actions to be performed by a communication robot (hereinafter also referred to as just “robot”) 12.
  • the communication robot 12 is a robot intended for communications with a person 14 by voice and gesture.
  • the robot 12 includes a truck 16.
  • a wheel 18 for making the robot 12 move autonomously.
  • the wheel 18 is driven by a wheel motor (indicated by a reference numeral "74" in Figure 3 showing an internal structure of the robot 12), and is capable of moving the truck 16, i.e., the robot 12 in an arbitrary direction of forward, backward, leftward, or rightward.
  • the truck 16 has a collision sensor (indicated by a reference numeral "82" in Figure 3 ) attached to its front surface. The collision sensor detects contact of the truck 16 with a person or another obstacle. Then, if any contact with an obstacle is detected during movement of the robot 12, the driving of the wheel 18 is promptly halted to bring the robot 12 into a sudden stop.
  • a height of the robot 12 is supposed to be about 100 cm in this embodiment not so as to give a coercive impression to persons, particularly children.
  • the height can be arbitrarily changed.
  • a sensor attachment panel 20 of polygonal column Provided on the truck 16 is a sensor attachment panel 20 of polygonal column. Each surface of the sensor-mount panel 20 is attached with an ultrasonic distance sensor 22.
  • the ultrasonic distance sensor 22 measures a distance from mainly a person in a circumference of the attachment panel 20, i.e., the robot 12.
  • a body of the robot 12 whose lower part is surrounded by the above stated attachment panel 20, in such a manner as to stand upright.
  • the body is composed of a lower body 24 and an upper body 26. These lower body 24 and upper body 26 are coupled to each other by a joint portion 28.
  • the joint portion 28 incorporates an elevating mechanism.
  • the elevating mechanism By using the elevating mechanism, it is possible to change a height of the upper body 26, that is, the height of the robot 12.
  • the elevating mechanism is driven by a waist motor (indicated by a reference numeral "72" in Figure 3 ), as described later.
  • the above mentioned height 100 cm of the robot 12 is a value with the upper body 26 in a lowest position. Therefore, the height of the robot 12 may be more than 100 cm.
  • the omni-directional camera 30 photographs a circumference of the robot 12, and is distinguished from an eye camera 52 described later.
  • the microphone 32 captures ambient sounds, in particular, persons' voices.
  • the upper body 26 has upper arms 36R and 36L attached on its both shoulders by shoulder joints 34R and 34L, respectively.
  • the shoulder joints 34R and 34L each have degrees of freedom for three axes. More specifically, the shoulder joint 34R can control an angle of the upper arm 36R around each of X axis, Y axis and Z axis.
  • the Y axis is an axis parallel to the upper 36R in a longitudinal direction (or the axis), and the X axis and the Z axis are axes orthogonal to the Y axis in different directions.
  • the shoulder joint 34L can control an angle of the upper arm 36L around each of A axis, B axis and C axis.
  • the B axis is an axis parallel to the upper arm 36L in a longitudinal direction (or the axis), and the A axis and the C axis are axes orthogonal to the B axis in different directions.
  • Front arms 40R and 40L are attached to respective ends of the upper arms 36R and 36L via elbow joints 38R and 38L.
  • the elbow joints 38R and 38L can control angles of the front arms 40R and 40L around each of a W axis and a D axis, respectively.
  • "0 degree” is a home position for the X, Y, Z and W axes and the A, B, C and D axes which control displacements of the upper arms 36R, 36L and the front arms 40R, 40L.
  • the upper arms 36R, 36L and the front arms 40R and 40L are directed in the downward direction.
  • a shoulder part of the upper body 26 including the shoulder joints 34R and 34L, the above described upper arms 36R, 36L and the front arms 40R and 40L are each provided with a touch sensor. These touch sensors detect whether or not the person 14 has touched these parts of the robot 12. These touch sensors are also collectively shown in Figure 3 by a reference numeral 80.
  • Balls 42R and 42L corresponding to hands are fixedly attached to respective ends of the front arms 40R and 40L are. Also, in case where some finger functions are required unlike the case with the robot 12 of this embodiment, it may be possible to use "hands" in the shape of a human's hands instead of the balls 42R and 42L.
  • the upper body 26 may include a front side, rear side, right side, left side, top side and bottom side, and the right side and left side may be formed in such a manner that their surfaces are obliquely directed forward in another embodiment. More specifically, it may be formed in such a manner that a width of the front side is shorter than that of the rear side, and an overhead view of the upper body 26 is a trapezoid in shape.
  • the shoulder joints 34 R and 34L are attached to the right side and the left side via right and left support portions whose surfaces are parallel to the right and left sides, respectively.
  • rotation ranges of the upper arm 36R and the upper arm 36L are restricted by the right and left sides or surfaces (attachment surfaces) of the support portions, and thus the upper arms 36R and 36L will never rotate beyond the attachment surfaces.
  • the upper arms 36R and 36L can rotate more inwards beyond the front. Accordingly, the arms of the robot 12 can be crossed at the front even if there is no degree of freedom in the arms due to the W axis and D axis. It is consequently possible to establish close communications with a person positioned in front such as giving a hug each other even with few degrees of freedom of the arm.
  • a head 46 is attached to a top center of the upper body 26 via a neck joint 44.
  • the neck joint 44 has 3 degrees of freedom and can be controlled in angle around each of an S axis, a T axis and a U axis.
  • the S axis is an axis extending upward directly from the neck, and the T axis and U axis are axes orthogonal to the S axis in different directions.
  • the head 46 is provided with a speaker 48 in a position corresponding to a human's mouth.
  • the speaker 48 is used by the robot 12 to communicate by sound or voice with a person around the same. Alternatively, the speaker 48 may be provided to another part of the robot 12, for example, the body.
  • the head 46 is provided with eyeballs 50R and 50L in positions corresponding to eyes.
  • the eyeballs 50R and 50L include eye cameras 52R and 52L, respectively. Additionally, in some cases, the right eyeball 50R and the left eyeball 50L are collectively called as an eyeball portion 50, and the right eye camera 52R and the left eye camera 52L are collectively referred to as an eye camera 52.
  • the eye camera 52 photographs a face or another part of a person or another object which comes close to the robot 12, and fetches an image signal of the same.
  • either of the above described omni-directional camera 30 and the eye camera 52 may be a camera using a solid-state imaging device such as a CCD and a CMOS.
  • the eye camera 52 is fixed within the eyeball 50, and the eyeball 50 is attached in a prescribed position within the head 46 via an eye ball support (not illustrated).
  • the eyeball support has 2 degrees of freedom and can be controlled in angle around each of an ⁇ axis and a ⁇ axis.
  • the ⁇ axis and ⁇ axis are axes set with respect to the head 46.
  • the ⁇ axis is an axis extending upward from the head 46
  • the ⁇ axis is an axis orthogonal to the ⁇ axis and also orthogonal to a direction in which a front side (face) of the head 46 points.
  • the ⁇ axis is set so as to be parallel to the S axis and the ⁇ axis is set so as to be parallel to the U axis.
  • the eyeball support are rotated around each of the ⁇ axis and the ⁇ axis and thus a tip end (front) side of the eyeball 50 or eye camera 52 is displaced, which moves a camera axis, i.e., a direction of gaze.
  • "0 degree” is a home position for the ⁇ axis and ⁇ axis which control displacement of the eye camera 52.
  • the camera axis of the eye camera 52 is pointed in a direction in which the front side (face) of the head 46 points, and the gaze is in a normal vision state, as shown in Figure 2 .
  • FIG. 3 shows a block diagram depicting an internal structure of the robot 12.
  • the robot 12 includes a microcomputer or a CPU 54 for entire control.
  • the CPU 54 is connected through a bus 56 with a memory 58, a motor control board 60, a sensor input/output board 62 and a voice input/output board 64.
  • the memory 58 includes a ROM, an HDD and a RAM, although not illustrated. Previously written into the ROM and the HDD are programs for entire control of the robot 12 and various data such as voice data for utterance.
  • the RAM is used as a temporary storage memory and also utilized as a working memory.
  • the motor control board 60 is composed of a DSP (Digital Signal Processor), for example, and controls each of axis motors of arms, head, eyeballs etc. More specifically, the motor control board 60 receives control data from the CPU 54, and adjusts rotation angles of total four motors (collectively shown as "right arm motor” in Figure 3 ) 66 which are composed of three motors controlling respective angles ⁇ x, ⁇ y and ⁇ z of the X, Y and Z axes of the right shoulder joint 34R, and one motor controlling angle ⁇ w of the axis W of the right elbow joint 38R.
  • DSP Digital Signal Processor
  • the motor control board 60 also adjusts rotation angles of total four motors (collectively shown as “left arm motor” in Figure 3 ) 68 which are composed of three motors controlling respective angles ⁇ a, ⁇ b and ⁇ c of the A, B and C axes of the left shoulder joint 34L, and one motor controlling angle ⁇ d of the D axis of the left elbow joint 38L.
  • the motor control board 60 adjusts rotation angles of three motors (collectively shown as “head motor” in Figure 3 ) 70 which control respective angles ⁇ s, ⁇ t and ⁇ u of the S, T and U axes of the head 46.
  • the motor control board 60 controls the waist motor 72 and two motors (collectively shown as "wheel motor” in Figure 3 ) 74 which drive the wheel 18.
  • the motor control board 60 adjusts rotation angles of two motors (collectively shown as “right eyeball motor” in Figure 3 ) 76 which control respective angles of the ⁇ axis and ⁇ axis of the right eyeball 50R, and adjusts rotation angles of two motors (collectively shown as “left eyeball motor” in Figure 3 ) 78 which control respective angles of the ⁇ axis and ⁇ axis of the left eyeball 50L.
  • the above mentioned motors of this embodiment are all stepping motors or pulse motors for simplification of control, but they may be direct current motors like the wheel motor 74.
  • the sensor input/output board 62 is similarly composed of a DSP, and fetches signals from each sensor and camera and provides them to the CPU 54. More specifically, data on reflex time from each of the ultrasonic distance sensors 22 is input into the CPU 54 through the sensor input/output board 62. Also, a video signal from the omni-directional camera 30 is subjected as necessary to a prescribed process by the sensor input/output board 62, and then input into the CPU 54. A video signal from the eye camera 52 is supplied to the CPU 54 in a same manner.
  • Figure 3 shows the touch sensors, described in relation to Figure 2 , collectively as "touch sensor 80". A signal from the touch sensor 80 is supplied to the CPU 54 via the sensor input/output board 62.
  • the CPU 54 provides the speaker 48 with synthesized voice data via the voice input/output board 64. In response to that, the speaker 48 outputs sound or voice according to the data. Then, the CPU 54 fetches voice input from the microphone 32 via the voice input/output board 64.
  • a communication LAN board 84 similarly composed of a DSP, provides transmission data sent from the CPU 54 to a wireless communication apparatus 86 and causes the wireless communication apparatus 86 to transmit transmission data. Also, the communication LAN board 84 receives data via the wireless communication apparatus 86, and provides the received data to the CPU 54. In this embodiment, the robot 12 communicates with the system 10 via a wireless LAN.
  • the CPU 54 is connected via the bus 56 with a spontaneous behavior database (DB) 88 and a reflex behavior database (DB) 90.
  • DB spontaneous behavior database
  • DB reflex behavior database
  • Reflective behaviors here mean reactive motions in response to behavior of the person 14 as a communication partner, such as directing the face toward the communication partner during communications and, when touched on some body part by the communication partner, looking at the touched part.
  • reflex motions are frequently made in communications between persons, and by making the robot 12 perform the reflex actions, it becomes possible to realize more natural, abundant communications with the person 14.
  • spontaneous behaviors mean motions with spontaneity such as giving a greeting first and looking around, and here denotes the actions other than the reflex behaviors.
  • a spontaneous behavior processing program storage area in which a plurality of programs for making the robot perform a spontaneous behavior are registered in advance. For example, stored are a program for a behavior of waving a hand in making a salutation, a program for a behavior of swinging arms as if making a march, a program for an action of looking around, a program for a behavior of making a bow, a program for a behavior of looking straight ahead at something, a program for a behavior of saying "Hello", a program for a behavior of saying "Bye-bye", a program for a behavior of moving to another place, etc.
  • a direct input execution processing program is stored as well. The direct input execution processing program is intended to make the robot 12 perform motions not contained in the registered programs. When the angle of each axis is directly input by the user into the system 10 as described later, this program causes the robot 12 to perform a motion according to the input value.
  • a reflex behavior processing program storage area in which a plurality programs for making the robot 12 perform a reflex behavior are registered in advance. For example, stored are a program for a behavior of saying "Get out of the way, please" when a person comes in front, a program for a behavior of, when touched on the body, looking at the touched part, a program for a behavior of, when a person's face is found, directing the eyes to the person's face, etc.
  • the reflex behaviors are reactive motions as stated above, and thus in each reflex behavior processing program, there established a precondition that a person's specific behavior is detected for execution of a reflex behavior.
  • the robot 12 performs interactive actions or communication actions with the person 14 by executing individual behavior programs registered in these spontaneous behavior DB 88 and reflex behavior DB 90.
  • the system 10 aims to support input of interactive actions to be performed by the robot 12 as described above and to which such computers as PCs and workstations are applicable.
  • the system 10 includes a CPU, a ROM, a RAM, an HDD, input apparatuses such as a mouse and a keyboard, a display apparatus such as an LCD, a wireless communication apparatus for communicating with the robot 12, for example.
  • the HDD stores some programs and data for support of action input, and according to the programs, the CPU executes a process while generating temporary data in the RAM. Needless to say, these programs, data and the like for support of action input may be stored into the HDD of the system 10 from various kinds of publicly known information storage media holding them.
  • the HDD of the system 10 also stores a behavior list table as shown in Figure 6 .
  • Registered on this behavior list table are individual behaviors or motions which form a series of interactive actions to be performed by the robot 12.
  • Each behavior in the behavior list table corresponds to each behavior program registered in the spontaneous behavior DB 88 and reflex behavior DB 90 of the robot 12.
  • interactive actions to be performed by the robot 12 are created by selecting and specifying some behaviors in sequence from a plurality of behaviors registered in the behavior list table.
  • Registered in the behavior list table are, in association with an identifier of each behavior, information on region by which the behavior is performed, a text for display, and information indicating an attribute of the behavior.
  • the region by which the behavior is performed is divided into right hand, left hand, head, and utterance.
  • utterance is one of the regions because they are here intended to classify the behaviors of the robot 12 in an easy-to-understand manner.
  • the attribute of a behavior indicates whether the behavior is a spontaneous behavior or a reflex behavior.
  • Each behavior is associated by its identifier with a program for performing each behavior registered in the spontaneous behavior DB 88 and the reflex behavior DB 90 of the robot 12.
  • the behaviors of the right arm (the upper arm 36R and the front arm 40R), including a behavior of waving a hand in giving a greeting and a behavior of swinging arms as if making a march, are registered, and also the directly input behavior is registered as well.
  • the directly input behavior is intended to operate a body part according to angle data on each of the axes (the X, Y, Z and W axes in case of the right hand) directly input by the user.
  • the directly input behavior is classified as spontaneous behavior.
  • no behavior is also registered, which is to be specified in case of involving no action of the body part.
  • the registered behaviors of the left arm (the upper arm 36L and the front arm 40L) are the same as those of the right arm.
  • angle data of the A, B, C and D axes are input.
  • reflex behaviors such as a behavior of looking at a person's face (eye contact) and a behavior of looking at the touched body part
  • spontaneous behaviors such as a behavior of looking around, a behavior of making a bow and a behavior of looking straight ahead at something.
  • angle data of the S, T and U axes are input.
  • reflex behaviors such as a behavior of saying "Get out of the way, please” when a person comes in front, and spontaneous behaviors such as a behavior of saying "Hello” and a behavior of saying "Bye-bye”.
  • Figure 7 shows one example of a selection input screen for selecting a behavior to be performed by the robot 12.
  • the selection input screen is provided with a behavior list section 92 for displaying a registered behavior list.
  • the behavior list section 92 provides a list box for each of the region including the right arm, left arm, head and utterance, for example.
  • Each list box presents as option items a plurality of behaviors in text form, which are classified by region, based on the behavior list table.
  • a button with the mouse or the like displays a plurality of behaviors for the related region, and the user thus can select one behavior from the list. More specifically, it is possible to select any of the registered spontaneous behaviors and reflex behaviors, or no behavior of the related region. It is also possible to select direct input for the right arm, left arm and head. In this manner, a plurality of behaviors are classified, listed and displayed by region, which offers the user the plurality of behaviors in an easy-to-understand manner. This allows the user to easily input a desired behavior.
  • a direct input section 94 for inputting an angle of each axis when direct input is specified in the list box.
  • the angle of each axis can be specified by selecting the direct input from the list box for the left arm in the behavior list section 92 and setting the sliders 94a for the A, B, C and D axes to respective appropriate positions in the direct input section 94.
  • the angle of each axis may be set by inputting a numerical value.
  • an image section 96 for displaying the motion of the robot 12 by images.
  • the image section 96 displays a front view, side view, top view, perspective view of the robot 12 which is drawn as a wire-frame model, for example.
  • the user can operate a decision button 98 from the input apparatus to decide the selected/specified behavior as a behavior to be actually performed by the robot 12.
  • a decision button 98 from the input apparatus to decide the selected/specified behavior as a behavior to be actually performed by the robot 12.
  • an execution instruction for the selected and decided behavior is transmitted to the robot 12 and the program for the behavior is processed in the robot 12. According to that, the robot 12 actually performs the behavior. Therefore, the user can check the decided behavior not only in the image section 96 but by the real robot 12 as well.
  • the behavior list section 92 of Figure 7 has an omission of a portion where a moving action is selected and set.
  • the image section 96 provides no display of the motions relating to the moving action from its nature.
  • the user creates interactive actions to be performed by the robot 12 by repeating the selection/specification and decision of a behavior in the selection input screen.
  • This creation activity can be carried out while checking an image in the image section 96 of the selection input screen and actually operating the robot 12 toward the person 14 as a partner.
  • the selected and decided behavior is temporarily stored in the RAM as an input behavior history for generation of reproductive motion information.
  • a series of decided behaviors are stored as input history information in the RAM of the system 10.
  • operating an input end button 100 generates the reproductive motion information for reproduction of the interactive actions based on the input history.
  • the generated reproductive motion information is saved in a reproductive motion DB 102 ( Figure 1 ).
  • the robot 12 by specifying some reproductive motion information to be carried out from the reproductive motion information registered on the reproductive motion DB 102 and instructing for its reproduction, the instruction for the execution of the reproductive motion information is transmitted to the robot 12. In response to that, the robot 12 reproduces the reproductive motion information, thereby realizing the interactive actions input by the user.
  • Reproductive motion information contains not only spontaneous behaviors but also reflex behaviors, and therefore the interactive actions to be performed by the robot 12 include some reflex actions in response to behavior of the person 14. That is, if the person 14 as a communication partner performs an action which meets the precondition for activation of a reflex behavior, toward the robot 12 performing the behavior according to the reproductive motion information, then the robot 12 performs the reflex behavior and presents the person 14 the reactive motion responsive to the action of the person 14. Therefore, the robot 12 can realize more natural and diverse interactive actions or communication actions.
  • FIG 8 shows one example of an operation of the system 10.
  • the CPU of the system 10 transmits an activation instruction to the robot 12 via a wireless communication apparatus based on the user's operation or the program in a step S1.
  • a prescribed program is activated according to the activation instruction, and the robot 12 is brought into a state of waiting for instruction data transmitted from the system 10.
  • step S3 it is determined in a step S3 whether or not to input an action. If "YES" in the step S3, that is, if an instruction for input of an action is given by the user's operation of the input apparatus, for example, the selection input screen as shown in Figure 7 is displayed on the display apparatus in a succeeding step S5.
  • a behavior to be performed by the robot 12 is specified in the behavior list section 92 or the direct input section 94 and decided by the decision button 98 according to the user's operation of the input apparatus.
  • the user can repeat the selection/specification and decision of a behavior for creation of a series of interactive actions, and can end action input by operating the input end button 100.
  • a step S7 it is determined whether or not any reflex behavior is selected. More specifically, it is determined whether or not the attribute of the decided behavior is reflex behavior, by referring to the behavior list table. If "YES" in the step S7, an execution instruction for the selected reflex behavior is transmitted to the robot 12.
  • the transmitted execution instruction data contains an identifier indicating the selected reflex behavior, for example.
  • a corresponding reflex behavior processing program is activated according to the execution instruction and the reflex behavior is performed. Additionally, although not illustrated, the decided reflex behavior is temporarily stored in the RAM as input history information.
  • step S11 it is determined in a step S11 whether or not any spontaneous behavior is selected. More specifically, it is determined whether or not the attribute of the decided behavior is spontaneous behavior, by reference to the behavior list table. If "YES" in the step S11, an execution instruction for the selected spontaneous behavior is transmitted to the robot 12 in a succeeding step S13.
  • the transmitted execution instruction data contains an identifier indicating the selected spontaneous behavior, for example.
  • a corresponding spontaneous behavior processing program is activated according to the execution instruction, and the spontaneous behavior is performed.
  • directly input behaviors are included in the spontaneous behaviors.
  • the transmitted execution instruction contains its identifier and input angle data of each of the axes.
  • the decided spontaneous behavior is temporarily stored in the RAM as input history information.
  • step S15 it is determined in a step S15 whether the input is ended or not. If “NO” in the step S15, the process is returned to the step S7 to repeat the process. On the other hand, if "YES” in the step S 15, that is, if the input end button 100, for example, is operated, reproductive motion data is generated based the input history information and stored in the reproductive motion DB 102 in a succeeding step S 17. Although detailed descriptions of the reproductive motion data will be given later. In the reproductive motion data, for example, each of the behaviors is described by its identifier in input order, information on the region and attributes, etc. is contained and, in case of a directly input behavior, the input angle data is contained as well.
  • step S17 If the step S17 is completed or if "NO" in the step S3, it is determined in a step S 19 whether or not the behavior is to be reproduced. If "YES” in the step S19, that is, if the reproduction of the behavior is specified by the user's operation of the input apparatus, for example, the reproductive motion data to be executed is read out from the reproductive motion DB 102 in a step S21. A list of reproductive motions stored in the reproductive motion DB 102, for example, is displayed for the user to select the reproductive motion data to be executed. Subsequently, an execution instruction for the read reproductive motion data is transmitted to the robot 12 in a step S23. In response to the execution instruction, the robot 12 reproduces the reproductive motion data and performs a series of communication actions.
  • step S23 If the step S23 is ended or if "NO" in the step S19, it is determined in a step S25 whether or not the end is specified by the user's operation of the input apparatus. If “NO”, the process is returned to the step S3. On the contrary, if "YES”, the process for input support is terminated.
  • Figure 9 shows one example of reproductive motion data generated by the system 10 and stored in the reproductive motion DB 102.
  • Reproductive motion data is like a scenario, and contains not only spontaneous behaviors but reflex behaviors as well.
  • Data of a reproductive motion 1 in Figure 9 aims to make the robot perform behavior of going up on stage from an audience seat and saying "Hello".
  • each behavior is described by its identifier in order of execution.
  • information on the region and attribute is included and, in case of the directly input behavior, the input angle data is included as well.
  • the actions until reaching the stage include a behavior of moving along the aisle through the door, a behavior of swinging the arms as if making a march, and a behavior of looking around, which are all spontaneous behaviors.
  • a behavior of saying "Get out of the way, please" when a person comes in front is set as a reflex behavior during the behavior of moving.
  • a behavior of, when touched by someone, looking at the touched part is set as a reflex behavior during the behavior of swinging the arms as if making a march.
  • a behavior of looking toward a person's face, if any is set as a reflex behavior during the behavior of looking around.
  • the program corresponding to each behavior is repeatedly processed until the robot reaches the stage. Therefore, the robot 12 performs not just spontaneous behaviors such as moving, swinging the arms and looking around but also, in case where a person's specific behavior satisfying the precondition is detected, performs reactive motions, i.e., reflex behaviors in response to the behavior.
  • the actions after arrival to the stage include giving a greeting and bidding farewell.
  • the giving a greeting includes a behavior of saying "Hello” and a gesture of making a bow.
  • the bidding farewell afterward includes a behavior of saying "Bye-bye” and a behavior of waving a hand. They are all spontaneous behaviors.
  • the actions of leaving the stage include a behavior of moving along the aisle from the stage, a behavior of swinging the arms, and a behavior of looking straight ahead at the exit. They are all spontaneous behaviors.
  • a behavior of saying "Get out of the way, please" when a person comes in front is set as a reflex behavior during the behavior of moving
  • a behavior of, when touched by someone, looking at the touched part is set as a reflex behavior during the behavior of swinging the arms as if making a march
  • a behavior of looking toward a person's face, if any is set as a reflex behavior during the behavior of looking straight ahead, as in case with the actions until reaching the stage.
  • the program corresponding to each of the actions of leaving the stage is repeatedly processed until the robot reaches the exit. Additionally, an end instruction is described to end the reproduction of the reproductive motion 1 when the robot finally reaches the position of the exit.
  • Execution of the action of the robot 12 based on the above described reproductive motion data and the execution instruction for each behavior at a time of action input is processed in accordance with a flowchart shown in Figure 10 .
  • the CPU 54 of the robot 12 obtains instruction data transmitted from the system 10 via the wireless communication apparatus 86 and the communication LAN board, in a first step S41 of Figure 10 .
  • the instruction data is analyzed in a step S43.
  • the instruction data is an execution instruction for reproductive motion data, for example, or an execution instruction for each individual behavior at a time of action input.
  • reproductive motion data a plurality of behaviors are executed in described order.
  • a step S45 it is determined in a step S45 whether or not the behavior to be performed is a reflex behavior based on the attribute information, etc. If "YES", a corresponding reflex behavior program is loaded on the basis of the identifier from the reflex behavior DB 90 into a working area of the memory 58, and the process is executed according to the reflex behavior program. Therefore, if the precondition for performing the reflex behavior is satisfied, the robot 12 performs the reflex behavior and presents the motion toward the person 14 as a communication partner. Upon completion of the process in the step S47, the process is returned to the step S45.
  • Figure 11 shows details on the execution process for a behavior of looking at a person's face.
  • the CPU 54 of the robot 12 reads an image from a CCD camera (the eye camera 52), and determines in a step S63 whether or not there exists any person's face in the image.
  • step S63 If "NO" in the step S63, that is, if it is determined that there is no person's face in the obtained image, values of preset angles ⁇ s and ⁇ u that correspond to an upward and frontward direction are sent as angle control data for the S axis and U axis to the motor control board 60 in a step S65, which controls a head motor 70 of the S axis and U axis to point the head 46 in a direction in which any person's face may exist.
  • step S63 that is, if any person's face is detected in the obtained image, the values of the angles ⁇ s and ⁇ u corresponding to the direction of the detected person's face are calculated and the calculated values of the angles ⁇ s and ⁇ u are sent as angle control data for the S axis and U axis to the motor control board 60 in a step S67, which controls the head motor 70 of the S axis and U axis to point the head 46 in the direction in which the person's face exists.
  • step S65 or the step S67 the execution process for the reflex behavior is ended, and the process is returned to the step S45 in Figure 10 in this case.
  • step S49 it is determined in a step S49 whether or not the behavior to be performed is a spontaneous behavior based on the attribute information, etc. If "YES” in the step S49, a corresponding spontaneous behavior program is loaded on the basis of the identifier from the spontaneous behavior DB 88 into a working area of the memory 58, and the process is executed according to the spontaneous behavior program in a step S51. Additionally, in case of the directly input behavior, the process is based on the direct input execution processing program, the input angle data, etc. Therefore, the robot 12 performs the spontaneous behavior and presents the motion toward the person 14 as a communication partner. Upon completion of the process in the step S51, the process is returned to the step S45.
  • step S49 it is determined in a step S53 whether or not this is an end instruction. If “NO” in the step S53, the process is returned to the step 45. If "YES”, all the behaviors to be performed have been performed and thus the process is terminated.
  • the list of a plurality of behaviors including reflex behaviors responsive to behavior of a person is displayed so that the user can make a selection.
  • This makes it easy to input each behavior forming interactive actions, and also input and generate the interactive actions of the robot 12. Since the generated interactive actions includes not only spontaneous behaviors but also reflex behaviors (reactive motions), if a person exhibits specific behavior during the reproduction, it is possible to activate a reflex behavior responsive to that, which allows the robot 12 to easily realize more natural and diverse interactive actions.
  • behavior list is displayed in text form in the above described embodiments, it may be possible to present the list using icons of designs indicating characteristics of the behaviors, etc.
  • the interactive actions are reproduced by transmitting the reproductive motion data from the system 10 to the robot 12.
  • the reproductive motion data generated by the system 10 in the reproductive motion DB provided within the robot 12 so as to make the robot 12 perform the interactive actions by the reproductive motion data.
  • each of the above stated embodiments is supposed to support the input of interactive actions including reflex behaviors responsive to a person's behavior.
  • Figure 12 shows one example of the selection input screen displayed on the display apparatus of the system 10 of the other embodiment.
  • the selection input screen displays emotional expressions to be added to the behavior of the robot 12 in a selectable manner.
  • the behavior list section 92 is provided with a list box for selecting an emotional expression, for example.
  • the list box for emotional expressions provides a plurality of emotional expressions in text form or the like as option items based on an emotional expression list ( Figure 13 ).
  • the storage apparatus including the HDD of the system 10 stores emotional expression list data as shown in Figure 13 as well as the behavior list data.
  • Registered in the emotional expression list data is information regarding a plurality of emotional expressions to be added to the behavior of the robot 12 in association with the identifiers.
  • basic emotions are prepared such as delight, sadness, anger, fear, disgust and curiosity, for example, and also an option "nothing" is prepared in which no emotional expression is added.
  • the user can select the emotional expression which is to be added to the behavior to be performed by the robot 12.
  • listing and displaying a plurality of emotional expressions makes it possible to offer the user the plurality of emotional expressions in an easy-to-understand manner. This allows the user to easily input a desired emotional behavior.
  • emotional expressions to be prepared are not limited to the above mentioned examples and may be further extended in emotional representation.
  • facial expressions for example, the six basic expressions defined by Ekman, that is, happiness, anger, surprise, sadness, fear, and disgust are well known.
  • following expressions are named in " The Managed Heart: Commercialization of Human Feeling" (written by A. R. Hochschild, translated by Jun Ishikawa and Aki Murofushi, Sekaishisosha Co., Ltd., 2000 ): loneliness, nostalgia, depression, frustration, anger, fear, indignation, disgust, contempt, guilt feeling, anguish, envy, ashamedy, affection, sympathy, compassion, embarrassment, sense of shame, anxiety etc.
  • Figure 14 shows one example of contents of the behavior list table of this embodiment. Further registered in the behavior list of this embodiment is information for determining whether or not to, in association with the identifier of each behavior, add any emotional expression to the behavior. In this embodiment, information indicating emotional expressions that are prohibited to be added to the behavior, that is, emotional expressions inappropriate for the behavior, is also registered. The information is provided in order to prevent a contradiction between an original emotional expression inherent in a behavior or motion and an emotional expression added by the user.
  • the prohibited emotions are supposed to be registered in an example of Figure 14 .
  • the emotional expression when some emotional expression is being selected by the user, it is determined whether or not the emotional expression can be added to the selected behavior based on the information indicating the prohibited emotional expressions. If it is determined that the selected emotional expression is not appropriate to the selected action, the addition of the emotional expression to the behavior is not permitted, and the user's selection of emotional expression is reset and the user is prompted to reselect an emotional expression.
  • the robot 12 further comprises an emotional expression behavior database (DB) 104 for execution of an emotional behavior.
  • the emotional expression behavior DB 104 is provided with an emotional expression behavior processing program storage area as shown in Figure 15 in which a plurality of programs for making the robot 12 perform an emotional expression behavior are registered in advance. These stored programs are broadly divided into programs for performing emotional reactive motions and programs for performing emotion correction/conversion.
  • the stored emotional reflex behavior processing programs includes a program for a behavior of agreeing willingly, a program for a behavior of agreeing unwillingly, a program for a behavior of getting scared, etc.
  • the robot 12 utters same word(s) as those said by the object of agreeing, i.e., the communication partner, at a pitch harmonized with the partner's, and nods the head.
  • the tuning of a pitch here means that the pitch varies in a same strain. For example, if the person 14 as a communication partner says “Let's have fun ⁇ " with a rising tone, the robot 12 also says “Let's have fun ⁇ " with a rising tone.
  • the robot 12 utters same word(s) as those said by the communication partner at a low pitch not turned with the partner's, and nods the head. By not tuning the pitch, an opposite nuance is produced and reluctance is expressed. If the person 14 says “Let's have fun ⁇ " with a rising tone, the robot 12 says “Let's have fun ⁇ ” with a descending tone. Additionally, in the behavior of "getting scared”, the robot 12 turns its eyes away from the object.
  • each of the emotional reactive motions in this embodiment is executed by each program (emotional reflex behavior processing program) prepared in advance for execution of a behavior involving an emotional expression.
  • the memory 58 of the robot 12 stores a list of emotional reflex behaviors as shown in Figure 16 .
  • information indicating the reflex behavior (without emotional expression) and emotional expression is stored in association with the identifier of the emotional reflex behavior.
  • Each of the identifiers of the emotional reflex behaviors is associated with each of the emotional reflex behavior processing programs stored in the emotional expression behavior DB 104.
  • the information indicating the reflex behavior is an identifier in the behavior list ( Figure 14 ), for example, and the information indicating the emotional expression is an identifier in the emotional expression list ( Figure 13 ), for example. Therefore, the emotional reflex behavior selected by the user can be identified on the basis of the emotional reflex behavior list.
  • the emotional expression action DB 104 stores programs for addition of emotional expressions to the behaviors other than the behaviors prepared as emotional reflex behavior processing programs, that is, the emotion correction/conversion programs. More specifically, the programs for addition of emotional expressions such as delight, sadness and anger, are stored. Each of the emotion correction/conversion programs is associated with each of the identifiers in the emotional expression list ( Figure 13 ).
  • the emotion correction/conversion program coverts control data for realization of a behavior into control data in accordance with the emotional expression. More specifically, in case ofa behavior involving a gesture, the angle control data of each axis motor of the robot 12 is converted. Also, in case of a behavior involving utterance, the voice control data for control of a pitch of voice to be output and speaking speed, etc. is converted.
  • the angle control data of each axis is converted in such a manner that the motion becomes larger (about 1.5 times, for example), and the voice control data is converted in such a manner that the pitch of voice is made higher.
  • the angle control data of each axis is converted in such a manner that the motion becomes smaller (about 0.8 time, for example) and the head 46 is directed downward (-15 degrees, for example), and the voice control data is converted in such a manner that the pitch of voice is made lower.
  • the angle control data of each axis is converted in such a manner that the motion becomes more rapid, and the voice control data is converted so that the pitch of voice is made lower and the speaking speed becomes higher.
  • each axis motor is driven or the voice modified in pitch and speaking speed, etc. is output from the speaker 48, which realizes the behavior with the emotional expression added.
  • the robot 12 says “Hello (at a high pitch)” and makes a bow livelily (with a large gesture).
  • the robot 12 says “Hello (at a low pitch)” and makes a small bow while looking down.
  • anger is added, the robot 12 says “Hello (at a low pitch and in a rapid manner)” and makes a quick bow.
  • the robot 12 walks with a large swing of its arms. In case where sadness is added, the robot 12 walks with a small swing of its arms and its head down.
  • the robot 12 walks swinging its arms at a high speed. Moreover, if delight is added to the behavior of "shaking hands", the robot 12 says “Let's shake hands (at a high pitch)” and offers its hand in a large way. If sadness is added, the robot 12 says “Let's shake hands (at a low pitch)” and offers its hand in a small way with its head down. In case where anger is added, the robot 12 says “Let's shake hands (at a low pitch and in a rapid manner)” and offers its hand quickly.
  • Figure 17 shows one example of an operation of the system 10 at a time of action input.
  • the CPU of the system 10 displays the selection input screen as shown in Figure 12 on the display apparatus in a first step S5 of Figure 17 .
  • step S5 of Figure 17 is the same as that in the step S5 of Figure 8 stated above, and thus is given same reference numerals.
  • same reference numerals are given in same process as those of the above stated flowcharts so as to omit overlapping detailed descriptions.
  • data on the behavior selected/input by the user is stored in a prescribed area of the RAM in a step S71.
  • the identifier of each of the behaviors selected/specified by the user is stored, and in case of direct input, angle data is stored as well.
  • a step S73 it is determined whether or not any emotional expression is selected. If “YES” in the step S73, it is determined in a step S75 whether or not the selected emotional expression is an emotional expression inappropriate to the selected behavior, based on the behavior list data ( Figure 14 ). If "NO” in the step S75, that is, if the selected emotional expression is not registered in the behavior list data as an emotional expression prohibited for the selected behavior, for example, it is possible to add the emotional expression to the behavior.
  • the identifier of the selected emotional expression is stored as input data in a prescribed area of the RAM in a step S77.
  • step S75 it is informed by text display or sound output that the emotional expression the user is attempting to select cannot be chosen with respect to the selected behavior. Then, in a step S81, the user's selection of emotional expression is reset and the emotional expression selection section in the selection input screen is returned to default (no emotional expression, etc.), for example. In this manner, the user is informed that the selection of the emotional expression is not appropriate to the selected behavior, and prompted to select another emotional expression.
  • step S83 determines whether or not any behavior to be performed by the robot 12 is decided according to the user's operation. If “YES” in the step S83, that is, if the decision button 98 is operated, it is determined in a succeeding step S85 whether or not the selected and decided behavior is a reflex behavior, based on the behavior list data.
  • step S85 determines whether or not the behavior is accompanied with any emotional expression. If "NO” in the step S87, that is, if the identifier indicative of no emotional expression is stored as information on the selected emotional expression in a prescribed area of the RAM, an execution instruction for the selected and decided reflex behavior is transmitted to the robot 12 in a step S89.
  • instruction data including the identifier indicative of the reflex behavior, information on the attribute and region related to the behavior, and the identifier indicative of no emotional expression, etc.
  • the corresponding processing program for the reflex behavior is activated in the robot 12.
  • an execution instruction for the selected and decided emotional reflex behavior is transmitted to the robot 12 in a step S91.
  • transmitted is instruction data including the identifier indicative of the reflex behavior, information on the attribute and region related to the behavior, and the identifier indicative of the emotional expression, etc.
  • the corresponding processing program for the emotional reflex behavior is activated in the robot 12.
  • an input history is stored in a prescribed area of the RAM in a step S93. More specifically, the identifier of the decided reflex behavior, the information on the attribute and region related to the behavior, and the identifier of the emotional expression, are recorded as an input history.
  • step S95 it is determined in a step S95 whether or not the selected and decided behavior is a spontaneous behavior based on the behavior list data. If "YES” in the step S95, that is, if the attribute of the behavior is spontaneous behavior, it is determined in a step S97 whether or not the behavior is accompanied with any emotional expression. If "NO” in the step S97, an execution instruction for the selected and decided spontaneous behavior is transmitted to the robot 12 in a step S99. For example, transmitted is the instruction data including the identifier indicative of the spontaneous behavior, the information on the attribute and region related to the behavior, the identifier indicative of no emotional expression, and the input angle data (in case of direct input), etc., for example. In addition, according to the execution instruction, the corresponding processing program for the spontaneous behavior is activated in the robot 12.
  • an execution instruction for the emotional spontaneous behavior is transmitted to the robot 12 in a step S 101.
  • transmitted is the instruction data including the identifier indicative of the spontaneous behavior, the information on the attribute and region related to the behavior, the identifier indicative of the emotional expression, and the input angle data (in case of direct input), etc.
  • the corresponding emotion correction/conversion program and spontaneous behavior processing program are activated in the robot 12.
  • the input history is stored in a prescribed area of the RAM in a step S103. More specifically, the identifier of the decided spontaneous behavior, the information on the attribute and region related to the behavior, the identifier of the emotional expression, the input angle data (in case of direct input), etc. are recorded as the input history.
  • step S93 upon completion of the step S93 or the step S103, the process moves to the step S15. Also, if "NO” in the step S95 or if "NO” in the step S83, the process goes on to the step S 15 as well.
  • step S 15 it is determined whether or not the input end button 100 is operated. If “NO”, the process returns to the step S71 to repeat the process for a next behavior input. On the other hand, if "YES” in the step S15, reproductive motion data is generated on the basis of the input history information stored in a prescribed area of the RAM, and stored in the reproductive motion DB 102.
  • the reproductive motion data generated in this embodiment is the same as that in the above described embodiment, and further includes information (identifier) on the emotional expression of each behavior.
  • Figure 18 shows one example of reproductive motion data N including some emotional expression stored in the reproductive motion DB 102.
  • This scenario is intended to perform a behavior of making a greeting and shaking hands, for example. Firstly, making a greeting is set and no emotional expression is set. In the behavior of making a greeting, the robot 12 says "Hello" and makes a bow. Also, an eye contact with no emotional expression is set as a reflex behavior. Next, shaking hands is set and delight is set as an emotional expression.
  • the robot 12 says “Let's shake hands (at a high pitch)" and offers its hand in a big way as described above. Moreover, agreeing is set as a reflex behavior and delight is set as an emotional expression. In the behavior of agreeing with delight added, if some word(s) are uttered by the object of agreeing, the robot 12 utters same word(s) at a harmonized pitch and nods its head, as described later. Also, an end instruction for ending reproduction of this scenario is finally described.
  • Figure 19 shows one example of an operation of the system 10 at a time of reproduction. If an instruction for reproduction of the scenario is given by the user's operation, the CPU of the system 10 displays on the display apparatus a list of reproductive motions stored in the reproductive motion DB 102 in a first step S111 of Figure 19 . Subsequently, the selected reproductive motion data is read out from the reproductive motion DB 102 according to the user's operation in a step S113. Then, in a step S23, an execution instruction for the read reproductive motion data is transmitted to the robot 12. In addition, the robot 12 reproduces the reproductive motion data according to the execution instruction, and performs a series of communication actions including reflex motions and emotional expressions.
  • Figure 20 shows one example of an operation of the robot 12.
  • the CPU 54 of the robot 12 obtains instruction data from the system 10 in a step S41, and analyzes the instruction data in a step S43. Then, it is determined in a step S45 whether or not the behavior to be performed is a reflex behavior.
  • step S45 it is determined in a step S121 whether or not the behavior involves any emotional expression.
  • the instruction data transmitted from the system 10 contains information (identifier) on emotional expression related to the behavior to be performed, and thus the determination is made on the basis of the information. If "NO” in the step S121, that is, if the identifier indicating no emotional expression is described with respect to the behavior, the corresponding processing program for the reflex behavior is read out from the reflex behavior DB 90 based on the identifier of the reflex behavior, and the process is executed according to the read program in a succeeding step S47. Upon completion of the step S47, the process is returned to the step S45.
  • step S121 that is, if an identifier indicating any emotional expression is described with respect to the behavior, for example, the corresponding processing program for the emotional reflex behavior is read out from the emotional expression behavior DB 104 and the process is executed according to the read program in a step S123.
  • the instruction data contains the identifier of the reflex behavior and the identifier of the emotional expression, and thus the processing program for the emotional reflex behavior to be performed is identified on the basis of the list data ( Figure 16 ) of the emotional reflex behaviors stored in the memory 58 or the emotional expression behavior DB 104. Therefore, in case where the precondition for performing the emotional reflex behavior is satisfied, the robot 12 performs the behavior and offers the emotional reflex motion to the person 14 as a communication partner.
  • the process is returned to the step S45.
  • Figure 21 shows details on the execution process of the behavior of "agreeing willingly".
  • the CPU 54 of the robot 12 reads voice information into the memory 58 via the microphone 32 and the voice input/output board 64. This acquisition of voice information from the communication partner is repeatedly carried out until a prescribed time has elapsed, for example.
  • the read voice information is subjected to voice recognition in a step S143.
  • the robot 12 is equipped with a voice recognition capability, and the memory 58 stores a dictionary (dictionary for voice recognition) for recognizing word(s) (voice) uttered by the communication partner. Sound data in the dictionary for voice recognition and the obtained voice information are compared by a publicly known method, for example, DP matching or HMM (Hidden Markov Model) to identify contents (word(s) or phrase(s)) of the utterance from the object of agreeing.
  • step S145 it is determined in a step S145 whether or not the word(s) uttered by the object of agreeing exist in the voice data for utterance in the memory 58. If "NO" in the step 145, that is, if no same word(s) as those uttered by the communication partner are stored, the execution process for the behavior is terminated.
  • the values of the angles ⁇ s and ⁇ u for pointing in a more downward direction than the current angles are sent to the S axis and the U axis in a step S 147.
  • the angle control data is provided to the motor control board 60 so as to control the S axis motor and the U axis motor of the head motor 70 and direct the head 46 downward for a nodding motion.
  • a pitch of the same voice as the word(s) uttered by the object of agreeing is converted and reproduced in a manner harmonized with the object of agreeing in a step S149.
  • the voice control data is converted in such a manner that the pitch of voice to be output varies in the same strain as the pitch of voice from the object of agreeing, based on the information on the pitch of utterance from the object of agreeing obtained by the voice recognition process in the step S 143, for example. Then, the converted voice control data and the voice data of same word(s) are provided to the voice input/output board 64, and the speaker 48 outputs the voice.
  • step S45 it is determined in the step S49 whether or not the behavior to be performed is a spontaneous behavior. If “YES” in the step S49, it is determined in a step S125 whether or not the behavior is accompanied with any emotional expression. If “NO” in the step S125, that is, if the identifier indicating no emotional expression is described with respect to the behavior, for example, the corresponding processing program for the spontaneous behavior is read out from the spontaneous behavior DB 88 based on the identifier of the spontaneous behavior, and the process is executed according to the read program in a succeeding step S51. Upon completion of the step S51, the process is returned to the step S45.
  • the emotion correction/conversion process is carried out in a step S127.
  • the corresponding emotion correction/conversion program is read out from the emotional expression behavior DB 104 based on the identifier of the emotional expression, and the emotion correction/conversion is processed on the basis of the read program. More specifically, the control data for realizing the behavior of the robot 12 is altered according to the selected emotional expression.
  • One example of operation of the emotion correction/conversion process is presented in detail in Figure 22 .
  • a first step S 161 of Figure 22 the CPU 54 of the robot 12 reads out the control data of the motion (gesture) of the behavior to be performed and the control data of the utterance into a working area of the memory 58 from a prescribed area of the memory 58 or the spontaneous behavior DB 88.
  • the angle control data directly input by the user, obtained in the step S41 is read out.
  • the control data of the motion includes the angle control data of each axis motor
  • the control data of the utterance includes voice control data for controlling the pitch of voice to be output and speaking speed, etc.
  • step S 163 it is determined in a step S 163 whether or not the identifier of the emotional expression is an identifier indicating delight. If “YES” in the step S163, the process is carried out according to the emotion correction/conversion program for delight read out from the emotional expression behavior DB 104. That is, it is determined in a step S 165 whether or not the behavior involves any motion based on the angle control data, for example. If “YES” in the step S165, the angle control data of each axis is converted for expression of delight in such a manner to make the motion larger (1.5 times, for example) in a step S167.
  • step S169 Upon completion of the step S167, or if "NO” in the step S165, it is determined in a step S169 whether or not the behavior involves any utterance based on the voice control data, for example. If “YES” in the step S169, the voice control data is converted for expression of delight in such a manner that the pitch of voice to be output is made higher. Upon completion of the step S171, or if "NO” in the step S169, the process is returned to the step S129 of Figure 20 .
  • step S 173 it is determined in a step S 173 whether or not the identifier of the emotional expression is an identifier indicating sadness. If “YES” in the step S173, the process is carried out according to the emotion correction/conversion program for sadness read out from the emotional expression behavior DB 104. That is, it is determined in a step S 175 whether or not the behavior is accompanied with any motion. If “YES” in the step S175, the angle control data of each axis is converted for expression of sadness in such a manner so as to make the motion smaller (0.8 time, for example) and direct the head 46 downward (-15 degrees, for example) in a step S177.
  • step S179 Upon completion of the step S 177, or if "NO” in the step S 175, it is determined in a step S179 whether or not the behavior is accompanied with any utterance. If "YES” in the step S179, the voice control data is converted for expression of sadness in such a manner that the pitch of voice to be output becomes lower. Upon completion of the step S 181, or if "NO” in the step S 179, the process is returned to the step S129 of Figure 20 .
  • step S 173 it is determined in a step S 183 whether or not the identifier of the emotional expression is an identifier indicating anger. If “YES” in the step S183, the process is executed according to the emotion correction/conversion program for anger read out from the emotional expression behavior DB 104. That is, it is determined in a step S 185 whether or not the behavior is accompanied with any motion. If “YES” in the step S185, the angle control data of each axis is converted for expression of anger in such a manner that the motion becomes more rapid.
  • step S189 Upon completion of the step S 187, or if "NO” in the step S 185, it is determined in a step S189 whether or not the behavior involves any utterance. If “YES” in the step S189, the voice control data is converted for expression of anger in such a manner so as to lower the pitch of voice to be output and increase the speaking speed. Upon completion of the step S191, or if "NO” in the step S189, the process is returned to the step S129 of Figure 20 .
  • step S183 if "NO" in the step S183, as in the above mentioned cases with emotional expressions of delight, sadness and anger, the motion and the utterance are altered according to another emotional expression in a succeeding step S193. That is, the angle control data and the voice control data are converted as necessary to add the emotional expression to the behavior according to the emotion correction/conversion program associated with the identifier of the emotional expression.
  • step S193 the process is returned to the step S129 of Figure 20 .
  • the emotional spontaneous behavior based on the correction/conversion is executed in the step S129. That is, the process is carried out according to the processing program for the spontaneous behavior associated with the identifier of the spontaneous behavior and the control data corrected/converted in the step S127. This offers the communication partner the spontaneous behavior with the emotional expression added. Upon completion of the step S129, the process is returned to the step S45.
  • step S49 it is determined in the step S53 whether or not an end instruction is given. If “NO”, the process returns to the step S45. If "YES”, all the behavior to be performed has been carried out and the process is terminated.
  • the list of behaviors and also the list of emotional expressions to be added to the behaviors are displayed for the user's selection, which makes it possible to easily input an emotional motion and also easily input and generate emotional interactive actions to be performed by the communication robot. Therefore, the generated interactive actions may include emotional reflex behaviors and emotional spontaneous behaviors, etc., and thus it is possible to easily make the robot 12 realize more diverse interactive actions.
  • information indicating inappropriate or appropriate emotional expressions with respect to each behavior is stored so as to determine whether or not the emotional expression the user attempts to select is appropriate, and thus it is possible to avoid a contradiction between the emotional expression inherent in the behavior selected by the user and the additional emotional expression. Therefore, this makes possible to easily input a natural emotional motion and also easily input and generate natural emotional interactive actions.
  • emotion correction/conversion is carried out by the robot 12 as shown in Figure 20 and Figure 22 .
  • the emotional expression behavior DB 104 (emotion correction/conversion program) may be provided in the system 10 so that the system 10 executes emotion correction/conversion in the case where a behavior involving any emotional expression is selected and decided.
  • the angle control data and the voice control data are converted as necessary in the step S101 of Figure 17 , and the instruction data including the converted control data is transmitted to the robot 12.
  • the reproductive motion data including the information on the behavior and the control data subjected to emotion correction/conversion, etc. is generated and stored in the reproductive motion DB 102.
  • emotion correction/conversion is executed only on spontaneous behaviors, and reflex behaviors are supposed to be performed by each individual program for performing each behavior in which some emotional expression is incorporated.
  • reflex behaviors are supposed to be performed by each individual program for performing each behavior in which some emotional expression is incorporated.

Landscapes

  • Engineering & Computer Science (AREA)
  • Robotics (AREA)
  • Mechanical Engineering (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • General Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Manipulator (AREA)

Claims (13)

  1. Steuersystem (10) für einen Kommunikationsroboter (12) zum Unterstützen der Eingabe von interaktiven Aktionen, die von einem Kommunikationsroboter durchzuführen sind, umfassend:
    ein Speichermittel (102) zum Speichern von Informationen über eine Vielzahl von Verhalten im Voraus, die mit einer Vielzahl von Verhaltensprogrammen assoziiert sind, einschließlich eines Spontanverhaltensprogramms zum Ausüben eines spontanen Verhaltens und eines Reflexverhaltensprogramms, das unter Einbeziehung der Ermittlung einer Vorbedingung und zum Ausüben eines Reflexverhaltens als Reaktion auf das Verhalten einer Person (14), wenn die Vorbedingung erfüllt wird, erstellt ist,
    ein Anzeigemittel zum Anzeigen einer Liste der genannten Vielzahl von Verhalten auf eine benutzerwählbare Weise auf der Basis der genannten Informationen, die in dem genannten Speichermittel (102) gespeichert werden,
    ein Verhaltensbestimmungsmittel zum Bestimmen eines von dem genannten Kommunikationsroboter (12) auszuübenden Verhaltens aus der genannten Liste von Verhalten, die von dem genannten Anzeigemittel angezeigt wird, gemäß der Bedienung durch einen Benutzer und
    ein Erzeugungsmittel zum Erzeugen reproduktiver Bewegungsinformationen für interaktive Aktionen, die von dem genannten Kommunikationsroboter durchzuführen sind, auf der Basis einer Historie des von dem genannten Verhaltensbestimmungsmittel bestimmten Verhaltens.
  2. Steuersystem (10) für einen Kommunikationsroboter (12) nach Anspruch 1, wobei
    die genannten Anzeigemittel ferner zum Anzeigen einer Liste einer Vielzahl von Gefühlsausdrücken auf eine benutzerwählbare Weise konfiguriert sind,
    die genannten Verhaltensbestimmungsmittel ferner zum Bestimmen eines Gefühlsausdrucks, der zu dem von dem genannten Kommunikationsroboter auszuübenden Verhalten hinzuzufügen ist, aus der genannten Liste von Gefühlsausdrücken gemäß der Bedienung durch den Benutzer konfiguriert sind und
    die genannten Erzeugungsmittel ferner zum Erzeugen der genannten reproduktiven Bewegungsinformationen auf der Basis der Historie des von dem genannten Verhaltensbestimmungsmittel bestimmten Verhaltens und Gefühlsausdrucks konfiguriert sind.
  3. Steuersystem (10) für einen Kommunikationsroboter (12) nach Anspruch 2, wobei das genannte Verhaltensbestimmungsmittel ferner ein Ermittlungsmittel aufweist zum Ermitteln, ob der vom Benutzer gewählte Gefühlsausdruck für das gewählte Verhalten angebracht ist oder nicht, wobei das Ermittlungsmittel konfiguriert ist, um das Hinzufügen des genannten Gefühlsausdrucks zu dem genannten Verhalten nicht zuzulassen, wenn es ermittelt, dass der genannte Gefühlsausdruck für das genannte Verhalten nicht angebracht ist.
  4. Steuersystem (10) für einen Kommunikationsroboter (12) nach einem der Ansprüche 1 bis 3, das ferner ein Übertragungsmittel aufweist zum Übertragen einer Ausführungsanweisung für das genannte Verhalten an den genannten Kommunikationsroboter, wenn das genannte Verhaltensbestimmungsmittel das von dem genannten Kommunikationsroboter auszuübende Verhalten bestimmt hat.
  5. Steuersystem (10) für einen Kommunikationsroboter (12) nach einem der Ansprüche 1 bis 4, wobei die genannten Anzeigemittel zum Anzeigen der genannten Liste von Verhalten, nach Region des genannten Kommunikationsroboters eingestuft, funktionell sind.
  6. Steuersystem (10) für einen Kommunikationsroboter (12) nach einem der Ansprüche 1 bis 5, wobei die genannten Anzeigemittel so konfiguriert sind, dass das genannte Anzeigemittel ein Bild eines Aussehens des das genannte Verhalten ausübenden genannten Kommunikationsroboters anzeigt, wenn das Verhalten durch die Bedienung durch den Benutzer aus der genannten Liste von Verhalten gewählt wird.
  7. Steuersystem für einen Kommunikationsroboter nach einem der Ansprüche 2 bis 6, wobei
    die genannten Erzeugungsmittel so konfiguriert sind, dass das genannte Erzeugungsmittel Steuerdaten für das Ausüben des Verhaltens gemäß dem Gefühlsausdruck korrigiert und reproduktive Bewegungsinformationen einschließlich der korrigierten Steuerdaten erzeugt, wenn der genannte Gefühlsausdruck zu dem genannten Verhalten hinzugefügt wird.
  8. Aktionseingabeunterstützungsverfahren zum Unterstützen der Eingabe interaktiver Aktionen, die von einem Kommunikationsroboter durchzuführen sind, in einem Steuersystem für einen Kommunikationsroboter, das ein Speichermittel aufweist zum Speichern von Informationen über eine Vielzahl von Verhalten im Voraus, die mit einer Vielzahl von Verhaltensprogrammen assoziiert sind, einschließlich eines Spontanverhaltensprogramms zum Ausüben eines spontanen Verhaltens und eines Reflexverhaltensprogramms, das unter Einbeziehung der Ermittlung einer Vorbedingung und zum Ausüben eines Reflexverhaltens als Reaktion auf das Verhalten einer Person, wenn die Vorbedingung erfüllt wird, erstellt ist, einschließlich:
    eines Anzeigeschritts des Anzeigens einer Liste der genannten Vielzahl von Verhalten auf eine benutzerwählbare Weise auf der Basis der genannten Informationen, die in dem genannten Speichermittel gespeichert werden,
    eines Verhaltensbestimmungsschritts zum Bestimmen eines von dem genannten Kommunikationsroboter auszuübenden Verhaltens aus der genannten Liste von Verhalten, die durch den genannten Anzeigeschritt angezeigt wird, gemäß der Bedienung durch einen Benutzer und
    eines Erzeugungsschritts zum Erzeugen reproduktiver Bewegungsinformationen für interaktive Aktionen, die von dem genannten Kommunikationsroboter durchzuführen sind, auf der Basis einer Historie des von dem genannten Verhaltensbestimmungsschritt bestimmten Verhaltens.
  9. Aktionseingabeunterstützungsverfahren nach Anspruch 8, wobei
    der genannte Anzeigeschritt ferner eine Liste einer Vielzahl von Gefühlsausdrücken auf eine benutzerwählbare Weise anzeigt,
    der genannte Verhaltensbestimmungsschritt ferner einen Gefühlsausdruck, der zu dem von dem genannten Kommunikationsroboter auszuübenden Verhalten hinzuzufügen ist, aus der genannten Liste von Gefühlsausdrücken gemäß der Bedienung durch den Benutzer bestimmt und
    der genannte Erzeugungsschritt die genannten reproduktiven Bewegungsinformationen auf der Basis der Historie des von dem genannten Verhaltensbestimmungsschritt bestimmten Verhaltens und Gefühlsausdrucks erzeugt.
  10. Aktionseingabeunterstützungsverfahren nach Anspruch 9, wobei der genannte Verhaltensbestimmungsschritt ferner einen Ermittlungsschritt beinhaltet zum Ermitteln, ob der vom Benutzer gewählte Gefühlsausdruck für das gewählte Verhalten angebracht ist oder nicht, und das Hinzufügen des genannten Gefühlsausdrucks zu dem genannten Verhalten nicht zulässt, wenn der genannte Ermittlungsschritt ermittelt, dass der genannte Gefühlsausdruck für das genannte Verhalten nicht angebracht ist.
  11. Aktionseingabeunterstützungsverfahren nach Anspruch 9 oder 10, wobei
    der genannte Erzeugungsschritt das Korrigieren von Steuerdaten für das Ausüben des Verhaltens gemäß dem Gefühlsausdruck und das Erzeugen reproduktiver Bewegungsinformationen einschließlich der korrigierten Steuerdaten beinhaltet, wenn der genannte Gefühlsausdruck zu dem genannten Verhalten hinzugefügt wird.
  12. Aktionseingabeunterstützungsprogramm für ein Steuersystem für einen Kommunikationsroboter, wobei das Programm einen Code aufweist, der, wenn er von einem Prozessor in dem Steuersystem ausgeführt wird, den Prozessor zum Ausführen der Verfahrensschritte gemäß einem der Ansprüche 8 bis 11 veranlasst.
  13. Speichermedium, das ein Programm gemäß Anspruch 12 speichert.
EP04771835.8A 2003-08-12 2004-08-12 Steuersystem für kommunikationsroboter Expired - Lifetime EP1669172B1 (de)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2003291939 2003-08-12
PCT/JP2004/011874 WO2005014242A1 (ja) 2003-08-12 2004-08-12 コミュニケーションロボット用制御システム

Publications (3)

Publication Number Publication Date
EP1669172A1 EP1669172A1 (de) 2006-06-14
EP1669172A4 EP1669172A4 (de) 2008-01-09
EP1669172B1 true EP1669172B1 (de) 2013-10-02

Family

ID=34131686

Family Applications (1)

Application Number Title Priority Date Filing Date
EP04771835.8A Expired - Lifetime EP1669172B1 (de) 2003-08-12 2004-08-12 Steuersystem für kommunikationsroboter

Country Status (6)

Country Link
US (1) US7689319B2 (de)
EP (1) EP1669172B1 (de)
JP (1) JP4271193B2 (de)
KR (1) KR100767170B1 (de)
CN (1) CN100436082C (de)
WO (1) WO2005014242A1 (de)

Families Citing this family (97)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8412377B2 (en) 2000-01-24 2013-04-02 Irobot Corporation Obstacle following sensor scheme for a mobile robot
US8788092B2 (en) 2000-01-24 2014-07-22 Irobot Corporation Obstacle following sensor scheme for a mobile robot
US6956348B2 (en) 2004-01-28 2005-10-18 Irobot Corporation Debris sensor for cleaning apparatus
US6690134B1 (en) 2001-01-24 2004-02-10 Irobot Corporation Method and system for robot localization and confinement
US7571511B2 (en) 2002-01-03 2009-08-11 Irobot Corporation Autonomous floor-cleaning robot
US7429843B2 (en) 2001-06-12 2008-09-30 Irobot Corporation Method and system for multi-mode coverage for an autonomous robot
US8396592B2 (en) 2001-06-12 2013-03-12 Irobot Corporation Method and system for multi-mode coverage for an autonomous robot
US9128486B2 (en) 2002-01-24 2015-09-08 Irobot Corporation Navigational control system for a robotic device
US8386081B2 (en) 2002-09-13 2013-02-26 Irobot Corporation Navigational control system for a robotic device
US8428778B2 (en) 2002-09-13 2013-04-23 Irobot Corporation Navigational control system for a robotic device
JP3875659B2 (ja) * 2003-07-25 2007-01-31 株式会社東芝 カメラ装置及びロボット装置
US7332890B2 (en) 2004-01-21 2008-02-19 Irobot Corporation Autonomous robot auto-docking and energy management systems and methods
US7720554B2 (en) 2004-03-29 2010-05-18 Evolution Robotics, Inc. Methods and apparatus for position estimation using reflected light sources
EP1776624A1 (de) 2004-06-24 2007-04-25 iRobot Corporation Programmier- und diagnosewerkzeug für einen beweglichen roboter
US7706917B1 (en) 2004-07-07 2010-04-27 Irobot Corporation Celestial navigation system for an autonomous robot
US8972052B2 (en) 2004-07-07 2015-03-03 Irobot Corporation Celestial navigation system for an autonomous vehicle
JP4564447B2 (ja) * 2004-12-14 2010-10-20 本田技研工業株式会社 自律移動ロボット
US7620476B2 (en) 2005-02-18 2009-11-17 Irobot Corporation Autonomous surface cleaning robot for dry cleaning
US8392021B2 (en) 2005-02-18 2013-03-05 Irobot Corporation Autonomous surface cleaning robot for wet cleaning
ATE468062T1 (de) 2005-02-18 2010-06-15 Irobot Corp Autonomer oberflächenreinigungsroboter für nass- und trockenreinigung
US8930023B2 (en) 2009-11-06 2015-01-06 Irobot Corporation Localization by learning of wave-signal distributions
JP2007044825A (ja) * 2005-08-10 2007-02-22 Toshiba Corp 行動管理装置、行動管理方法および行動管理プログラム
US7860614B1 (en) * 2005-09-13 2010-12-28 The United States Of America As Represented By The Secretary Of The Army Trainer for robotic vehicle
EP2270619B1 (de) 2005-12-02 2013-05-08 iRobot Corporation Modularer Roboter
US9144360B2 (en) 2005-12-02 2015-09-29 Irobot Corporation Autonomous coverage robot navigation system
EP2544065B1 (de) 2005-12-02 2017-02-08 iRobot Corporation Robotersystem
EP2120122B1 (de) 2005-12-02 2013-10-30 iRobot Corporation Abdeckungsrobotermobilität
EP2816434A3 (de) 2005-12-02 2015-01-28 iRobot Corporation Roboter mit autonomem Wirkungsbereich
US20070150104A1 (en) * 2005-12-08 2007-06-28 Jang Choul S Apparatus and method for controlling network-based robot
US20070135967A1 (en) * 2005-12-08 2007-06-14 Jung Seung W Apparatus and method of controlling network-based robot
EP2000889B1 (de) * 2006-03-15 2018-06-27 Omron Corporation Überwachungsgerät und überwachungsverfahren, steuerung und steuerverfahren, sowie programm
EP3031377B1 (de) 2006-05-19 2018-08-01 iRobot Corporation Müllentfernung aus reinigungsrobotern
US8417383B2 (en) 2006-05-31 2013-04-09 Irobot Corporation Detecting robot stasis
KR100883352B1 (ko) 2006-11-21 2009-02-11 한국전자통신연구원 원격대화에서 감정 및 의사의 표현 방법과 이를 위한 리얼이모티콘 시스템
KR101168481B1 (ko) 2007-05-09 2012-07-26 아이로보트 코퍼레이션 자동 커버리지 로봇
JP5181541B2 (ja) * 2007-06-15 2013-04-10 富士通株式会社 ロボットシステム、エディタ端末、およびエディタプログラム
FR2918477A1 (fr) * 2007-07-04 2009-01-09 Aldebaran Robotics Soc Par Act Procede d'edition de mouvements d'un robot
KR100880613B1 (ko) * 2007-08-10 2009-01-30 주식회사 쿼터뷰 지능형로봇의 감정표현지원시스템과 지원방법 및 이를이용한 지능형 로봇 시스템
KR100893758B1 (ko) * 2007-10-16 2009-04-20 한국전자통신연구원 로봇의 감성 표현 제어 시스템 및 방법
KR20090065212A (ko) * 2007-12-17 2009-06-22 한국전자통신연구원 로봇 채팅 시스템 및 방법
US8103382B2 (en) * 2008-04-24 2012-01-24 North End Technologies Method and system for sharing information through a mobile multimedia platform
WO2009158653A1 (en) * 2008-06-27 2009-12-30 Intuitive Automata, Inc. Apparatus and method for assisting in achieving desired behavior patterns
KR20110134814A (ko) * 2008-08-28 2011-12-15 (주)수연미디어 방송 컨텐츠와 연계되어 동작하는 컨텐츠 로봇 및 그 제어 방법
JP5187128B2 (ja) * 2008-10-16 2013-04-24 富士通株式会社 検索装置、検索方法、及び、プログラム
US8160747B1 (en) * 2008-10-24 2012-04-17 Anybots, Inc. Remotely controlled self-balancing robot including kinematic image stabilization
TW201019242A (en) * 2008-11-11 2010-05-16 Ind Tech Res Inst Personality-sensitive emotion representation system and method thereof
US8442661B1 (en) 2008-11-25 2013-05-14 Anybots 2.0, Inc. Remotely controlled self-balancing robot including a stabilized laser pointer
KR101493383B1 (ko) * 2009-01-23 2015-02-13 삼성전자 주식회사 로봇
US10163175B2 (en) * 2009-02-25 2018-12-25 Humana Inc. System and method for improving healthcare through social robotics
US9386139B2 (en) * 2009-03-20 2016-07-05 Nokia Technologies Oy Method and apparatus for providing an emotion-based user interface
FR2946160B1 (fr) * 2009-05-26 2014-05-09 Aldebaran Robotics Systeme et procede pour editer et commander des comportements d'un robot mobile.
JP5429461B2 (ja) * 2009-06-17 2014-02-26 株式会社国際電気通信基礎技術研究所 コミュニケーションロボット開発支援装置
FR2947923B1 (fr) * 2009-07-10 2016-02-05 Aldebaran Robotics Systeme et procede pour generer des comportements contextuels d'un robot mobile
KR101019655B1 (ko) * 2009-12-04 2011-03-07 (주) 퓨처로봇 사용 유도 장치 및 방법
KR20110080919A (ko) * 2010-01-07 2011-07-13 삼성전자주식회사 로봇
CN101808047A (zh) * 2010-02-10 2010-08-18 深圳先进技术研究院 即时通讯伴侣机器人及具有通讯伴侣的即时通讯方法
US8800107B2 (en) 2010-02-16 2014-08-12 Irobot Corporation Vacuum brush
US8306664B1 (en) * 2010-05-17 2012-11-06 Anybots 2.0, Inc. Self-balancing robot having a shaft-mounted head
US8788096B1 (en) * 2010-05-17 2014-07-22 Anybots 2.0, Inc. Self-balancing robot having a shaft-mounted head
US9092021B2 (en) * 2012-01-06 2015-07-28 J. T. Labs Limited Interactive apparatus
US9146560B2 (en) 2012-03-30 2015-09-29 Irobot Corporation System and method for implementing force field deterrent for robot
KR101190660B1 (ko) * 2012-07-23 2012-10-15 (주) 퓨처로봇 로봇 제어 시나리오 생성 방법 및 장치
JP5527747B1 (ja) * 2012-12-25 2014-06-25 コリア インスティチュート オブ インダストリアル テクノロジー 機械装置の持続的な情緒表現動作を選択するための装置及び方法
US9956687B2 (en) 2013-03-04 2018-05-01 Microsoft Technology Licensing, Llc Adapting robot behavior based upon human-robot interaction
US20140288704A1 (en) * 2013-03-14 2014-09-25 Hanson Robokind And Intelligent Bots, Llc System and Method for Controlling Behavior of a Robotic Character
CA2904359A1 (en) 2013-03-15 2014-09-25 JIBO, Inc. Apparatus and methods for providing a persistent companion device
US20150314454A1 (en) 2013-03-15 2015-11-05 JIBO, Inc. Apparatus and methods for providing a persistent companion device
US9211644B1 (en) 2013-10-25 2015-12-15 Vecna Technologies, Inc. System and method for instructing a device
US9724795B2 (en) * 2013-11-07 2017-08-08 Apex Brands, Inc. Tooling system with visual identification of attached component
EP2933071A1 (de) * 2014-04-17 2015-10-21 Aldebaran Robotics Verfahren und Systeme zur Verwaltung von Dialogen eines Roboters
USD746886S1 (en) 2014-05-23 2016-01-05 JIBO, Inc. Robot
US9370862B2 (en) * 2014-06-12 2016-06-21 Play-i, Inc. System and method for reinforcing programming education through robotic feedback
US10279470B2 (en) 2014-06-12 2019-05-07 Play-i, Inc. System and method for facilitating program sharing
WO2016011159A1 (en) * 2014-07-15 2016-01-21 JIBO, Inc. Apparatus and methods for providing a persistent companion device
US10152117B2 (en) * 2014-08-07 2018-12-11 Intel Corporation Context dependent reactions derived from observed human responses
JP6524631B2 (ja) * 2014-09-30 2019-06-05 セイコーエプソン株式会社 ロボット、制御装置およびロボットシステム
JP6323356B2 (ja) * 2015-02-03 2018-05-16 株式会社安川電機 処理システム、監視方法及びコンピュータプログラム
CN106926258B (zh) * 2015-12-31 2022-06-03 深圳光启合众科技有限公司 机器人情绪的控制方法和装置
JP6726388B2 (ja) * 2016-03-16 2020-07-22 富士ゼロックス株式会社 ロボット制御システム
CN107731225A (zh) * 2016-08-10 2018-02-23 松下知识产权经营株式会社 待客装置、待客方法以及待客系统
JP6819990B2 (ja) * 2016-08-16 2021-01-27 国立研究開発法人情報通信研究機構 対話システム及びそのためのコンピュータプログラム
JP7351745B2 (ja) * 2016-11-10 2023-09-27 ワーナー・ブラザース・エンターテイメント・インコーポレイテッド 環境制御機能を有する社会ロボット
CN107053191B (zh) 2016-12-31 2020-05-08 华为技术有限公司 一种机器人，服务器及人机互动方法
EP3568269A4 (de) * 2017-01-10 2020-08-19 Intuition Robotics, Ltd. Vorrichtung zur durchführung von emotionalen gesten zur interaktion mit einem benutzer
JP6855825B2 (ja) * 2017-02-13 2021-04-07 沖電気工業株式会社 情報処理装置、情報処理方法およびプログラム
JP2019005842A (ja) * 2017-06-23 2019-01-17 カシオ計算機株式会社 ロボット、ロボットの制御方法及びプログラム
JP6812325B2 (ja) * 2017-10-26 2021-01-13 株式会社日立ビルシステム ロボット管理システム
CN107756417A (zh) * 2017-11-21 2018-03-06 北方民族大学 智能双臂安全协作人机共融机器人系统
JP6747423B2 (ja) 2017-12-22 2020-08-26 カシオ計算機株式会社 ロボット、ロボット制御システム、ロボットの制御方法及びプログラム
JP2019123055A (ja) 2018-01-18 2019-07-25 株式会社ユピテル 装置
US11855932B2 (en) 2018-03-02 2023-12-26 Intuition Robotics, Ltd. Method for adjusting a device behavior based on privacy classes
KR20190141303A (ko) * 2018-06-14 2019-12-24 엘지전자 주식회사 이동 로봇의 동작 방법
WO2019246239A1 (en) 2018-06-19 2019-12-26 Ellipsis Health, Inc. Systems and methods for mental health assessment
US20190385711A1 (en) 2018-06-19 2019-12-19 Ellipsis Health, Inc. Systems and methods for mental health assessment
JP7186036B2 (ja) * 2018-08-14 2022-12-08 日本放送協会 ロボット操作装置及びロボット操作プログラム
US11376733B2 (en) * 2019-06-11 2022-07-05 Facebook Technologies, Llc Mechanical eyeball for animatronic devices
KR20210020312A (ko) * 2019-08-14 2021-02-24 엘지전자 주식회사 로봇 및 그의 제어 방법

Family Cites Families (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2001157976A (ja) * 1999-11-30 2001-06-12 Sony Corp ロボット制御装置およびロボット制御方法、並びに記録媒体
JP3797047B2 (ja) * 1999-12-08 2006-07-12 富士通株式会社 ロボット装置
JP2001191280A (ja) 1999-12-28 2001-07-17 Sony Corp ロボット装置及びその制御方法
JP2001191281A (ja) * 1999-12-29 2001-07-17 Sony Corp 編集装置、編集方法及び記録媒体
KR20020026165A (ko) * 2000-03-24 2002-04-06 이데이 노부유끼 로봇 장치의 행동 결정 방법 및 로봇 장치
JP2001277163A (ja) * 2000-04-03 2001-10-09 Sony Corp ロボットの制御装置及び制御方法
JP2001353678A (ja) * 2000-06-12 2001-12-25 Sony Corp オーサリング・システム及びオーサリング方法、並びに記憶媒体
JP4765155B2 (ja) 2000-09-28 2011-09-07 ソニー株式会社 オーサリング・システム及びオーサリング方法、並びに記憶媒体
CN1392826A (zh) * 2000-10-05 2003-01-22 索尼公司 机器人设备及其控制方法
JP4670136B2 (ja) * 2000-10-11 2011-04-13 ソニー株式会社 オーサリング・システム及びオーサリング方法、並びに記憶媒体
JP2002137182A (ja) * 2000-10-31 2002-05-14 Yamaha Motor Co Ltd 動作内容決定装置
JP4316819B2 (ja) * 2001-03-16 2009-08-19 株式会社バンダイ ロボットの制御装置、ロボットの制御方法、プログラム、及び記憶媒体
JP2003191187A (ja) 2001-12-21 2003-07-08 Sony Corp ロボット装置及びその制御方法

Also Published As

Publication number Publication date
CN100436082C (zh) 2008-11-26
KR20060073931A (ko) 2006-06-29
EP1669172A4 (de) 2008-01-09
WO2005014242B1 (ja) 2005-03-24
CN1835828A (zh) 2006-09-20
EP1669172A1 (de) 2006-06-14
JPWO2005014242A1 (ja) 2006-09-28
KR100767170B1 (ko) 2007-10-15
US20060293787A1 (en) 2006-12-28
WO2005014242A1 (ja) 2005-02-17
US7689319B2 (en) 2010-03-30
JP4271193B2 (ja) 2009-06-03

Similar Documents

Publication Publication Date Title
EP1669172B1 (de) Steuersystem für kommunikationsroboter
US7216082B2 (en) Action teaching apparatus and action teaching method for robot system, and storage medium
KR101880775B1 (ko) 자연스러운 대화 인터페이스가 장착된 인간형 로봇, 이러한 로봇의 제어 방법 및 대응 프로그램
US7076334B2 (en) Robot apparatus and method and system for controlling the action of the robot apparatus
JP7400923B2 (ja) 情報処理装置および情報処理方法
US6667593B2 (en) Robot apparatus
US6493606B2 (en) Articulated robot and method of controlling the motion of the same
US7251606B2 (en) Robot device with changing dialogue and control method therefor and storage medium
JP7260221B2 (ja) ロボット対話方法およびデバイス
JP2003266351A (ja) ロボット装置及びロボット装置の動作制御方法
US20060177802A1 (en) Audio conversation device, method, and robot device
JP2002049385A (ja) 音声合成装置、疑似感情表現装置及び音声合成方法
JP4617428B2 (ja) 動作生成システム
JP4059742B2 (ja) コミュニケーションロボット
JP2005059185A (ja) ロボット装置及びその制御方法
JP7156300B2 (ja) 情報処理装置、情報処理方法、およびプログラム
JP2006263858A (ja) コミュニケーションロボット
CN211806147U (zh) 一种基于模块化机器人的主控单元及模块化机器人
JP7024754B2 (ja) 制御装置、ロボット、制御方法およびプログラム
JP4735965B2 (ja) 遠隔地間対話システム
JP2001191279A (ja) 行動管理システム、行動管理方法及びロボット装置
JP2019150893A (ja) ロボット制御システム、ロボット制御方法およびロボット制御プログラム
JP2018161702A (ja) ロボット制御システムおよびロボット制御装置
JP2002239962A (ja) ロボット装置、ロボット装置の動作制御方法及びロボット装置の動作制御システム
JP2002120180A (ja) ロボット装置及びその制御方法

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20060209

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): DE FR GB SE

DAX Request for extension of the european patent (deleted)
RBV Designated contracting states (corrected)

Designated state(s): DE FR GB SE

A4 Supplementary search report drawn up and despatched

Effective date: 20071206

RIC1 Information provided on ipc code assigned before grant

Ipc: B25J 13/00 20060101AFI20050222BHEP

Ipc: B25J 9/16 20060101ALI20071130BHEP

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

INTG Intention to grant announced

Effective date: 20130528

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): DE FR GB SE

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: SE

Ref legal event code: TRGR

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602004043479

Country of ref document: DE

Effective date: 20131128

REG Reference to a national code

Ref country code: GB

Ref legal event code: 746

Effective date: 20140611

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602004043479

Country of ref document: DE

REG Reference to a national code

Ref country code: DE

Ref legal event code: R084

Ref document number: 602004043479

Country of ref document: DE

Effective date: 20140610

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

26N No opposition filed

Effective date: 20140703

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602004043479

Country of ref document: DE

Effective date: 20140703

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 13

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 14

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 15

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20230629

Year of fee payment: 20

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: SE

Payment date: 20230630

Year of fee payment: 20

Ref country code: FR

Payment date: 20230703

Year of fee payment: 20

Ref country code: DE

Payment date: 20230627

Year of fee payment: 20

REG Reference to a national code

Ref country code: DE

Ref legal event code: R071

Ref document number: 602004043479

Country of ref document: DE

REG Reference to a national code

Ref country code: GB

Ref legal event code: PE20

Expiry date: 20240811

REG Reference to a national code

Ref country code: SE

Ref legal event code: EUG

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GB

Free format text: LAPSE BECAUSE OF EXPIRATION OF PROTECTION

Effective date: 20240811