JP5223605B2 - Robot system, communication activation method and program - Google Patents

Description

  The present invention relates to a robot system that communicates with humans, a communication activation method using the robot system, and a program that is executed by a computer that controls the robot system.

  In general, in order to make communication through conversation natural, information other than voice (non-verbal information) including the gaze, gesture, and whisper of the other party is required in addition to voice information (verbal information). Both sense the rhythm, timing, distribution, strength, etc. related to speech and nonverbal information such as the other party's gaze, breathing, heartbeat, gestures, movement, whispering, competing, blinking, etc. By adjusting the rhythm, timing, distribution, and strength of the movement, it is only possible to share the physical rhythm (communication rhythm) with each other to make communication natural.

  It is called interaction behavior to sense such opponent's verbal information and non-verbal information with the five senses and take action in response to those reactions. Sharing communication rhythm through this interaction behavior is called communication tuning. In addition, the phenomenon of being drawn into a conversation by communication synchronization is called a pull-in phenomenon.

  In recent years, robot systems that can communicate with humans such as conversations have appeared. In communication between a human and a robot system, expressing this pull-in phenomenon is an important point for activating the communication. From such a background, various technologies that lead to the appearance of a pull-in phenomenon have been disclosed in communication between a robot and a user and communication between users through a robot (see, for example, Patent Documents 1 to 6). .

  The physical media communication system described in Patent Literature 1 displays both pseudo personality robots on the screens of the respective communication terminals in order to realize intimate communication with a communication partner separated in time or space. Let The pseudo personality robot operates based on both voice information and specific motion information. As a result, it is possible to give both the sense of sharing the space.

  The physical pull-in method and system described in Patent Document 2 simultaneously provide non-verbal information via the respective tactile sensations without requiring the speaker or the listener to switch the line of sight between the speaker and the listener. Thereby, it becomes possible to share each other's communication rhythm. Patent Document 3 discloses a method of applying the physical pull-in method and system disclosed in Patent Document 2 to a case where there are an unspecified number of listeners such as presentations.

  The automatic response toy described in Patent Document 4 determines the emotion of the toy based on external stimuli such as the size of the user's voice, the size of the user's face movement, and the timing of the user's whisper. For example, this toy interprets that the story is bouncing if the number of times the user's whispering is detected is high, and sets the emotion at that time to “happiness”. This toy performs a response action (interaction action) according to the determined emotion.

  The intention transmission device described in Patent Document 5 includes a voice transmission / reception unit, a shared robot, a listener control unit, and a speaker control unit. The voice transmission / reception unit transmits / receives a voice signal such as a conversation, and the shared robot responds to the voice signal by performing a head whispering action, a mouth opening / closing action, an eye blinking action, or a body gesture action. The listener control unit determines the behavior of the shared robot as a listener from the audio signal transmitted through the transmitter, and operates the shared robot. Then, the speaker control unit determines the behavior of the shared robot as a speaker from the voice signal received by the receiving unit, and operates the shared robot.

  The rhythm control dialogue apparatus described in Patent Document 6 includes a plurality of channels recognition means for recognizing a plurality of input data including time information of voice signals and gestures from a data input means, a time giving means for outputting time information, Rhythm detection means for processing a recognition result output from the recognition means to detect a rhythm of the user's dialogue, a cover history storage means for storing a rhythm history, and a dialog based on the rhythm recognized by the rhythm detection means Dialogue management means for proceeding and output means for outputting output data. The response content is transmitted to the user by the output means.

JP 2003-108502 A JP 2005-251133 A JP-A-2005-250421 JP 2002-239256 A JP 2000-349920 A Japanese Patent Laid-Open No. 10-11786

  All of the above six technologies detect the verbal information and non-verbal information of the communicating party (for example, detecting a silent section where neither the user nor the robot emits voice) and interacting with the robot based on the detected information By performing an action (interaction operation), it is expected that the user and the robot share a communication rhythm and develop a pull-in phenomenon.

  There are various methods of communication depending on the individual, but it is common for any person to slightly change the communication method according to the degree of synchronization with the other party. Therefore, in order to increase the probability of the pull-in phenomenon, a dynamic triggering strategy that changes the interaction behavior according to the development stage of communication is required. However, with the above six technologies, it is difficult to communicate under such a dynamic triggering strategy.

  The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a robot system, a communication activation method, and a program that can increase the probability of occurrence of a pull-in phenomenon and further activate communication.

In order to achieve the above object, a robot system according to a first aspect of the present invention includes:
An output unit for performing an interaction operation for a plurality of users;
A recognition unit for recognizing the communication rhythm of the user based on the verbal information and non-verbal information of each of the plurality of users;
Based on the communication rhythm, a synchronization level calculation unit that calculates a synchronization level between the users,
In accordance with the communication rhythm and the degree of synchronization, the output unit stores a rule constructed under a dynamic induction strategy of a pull-in phenomenon according to a communication development stage regarding an interaction operation to be performed on the user. A rule database;
An interaction control that refers to the rule database, searches for an operation command to be performed for the user using the communication rhythm and the degree of synchronization according to the rule, and controls the output unit based on the searched operation command A section.

The communication activation method according to the second aspect of the present invention includes:
A communication activation method using a robot system including an output unit that performs an interaction operation for a plurality of users,
A first step of recognizing the communication rhythm of the plurality of users based on the verbal information and non-verbal information of each of the users;
A second step of calculating a degree of synchronization between the users based on the communication rhythm;
In accordance with the communication rhythm and the degree of synchronization, the output unit stores a rule constructed under a dynamic induction strategy of a pull-in phenomenon according to a communication development stage regarding an interaction operation to be performed on the user. A third database for searching for an operation command to be performed on the user using the communication rhythm and the degree of synchronization according to the rule with reference to the rule database, and controlling the output unit based on the searched operation command And a process.

The program according to the third aspect of the present invention is:
In a computer that controls a robot system including an output unit that performs an interaction operation for a plurality of users,
A first procedure for recognizing the communication rhythm of the plurality of users based on the verbal information and non-verbal information of each of the users;
A second procedure for calculating the degree of synchronization between the users based on the communication rhythm;
In accordance with the communication rhythm and the degree of synchronization, the output unit stores a rule constructed under a dynamic induction strategy of a pull-in phenomenon according to a communication development stage regarding an interaction operation to be performed on the user. A third database for searching for an operation command to be performed on the user using the communication rhythm and the degree of synchronization according to the rule with reference to the rule database, and controlling the output unit based on the searched operation command And execute the procedure.

  According to the present invention, it is possible to increase the probability of the pull-in phenomenon and further activate communication.

  The best mode for carrying out the present invention will be described in detail with reference to the drawings. The robot system according to each embodiment of the present invention described below realizes communication synchronization between a robot and a user by recognizing a communication rhythm of a plurality of users and performing an interaction operation with respect to the plurality of users. It induces a phenomenon.

(First embodiment)
First, a first embodiment of the present invention will be described. FIG. 1 shows a schematic configuration of a robot system 100 according to the present embodiment. As shown in FIG. 1, the robot system 100 includes a communication field recognition unit 1, a communication synchronization degree calculation unit 2, an interaction control unit 3, and a social interaction rule database (hereinafter abbreviated as “SIRDB”) 4. And an output unit 5.

  The communication field recognition unit 1 has various sensors such as a microphone, a camera, and a biological sensor (not shown). The microphone inputs a voice of a user who is a communication partner of the robot system 100. The camera images the user. The biological sensor detects biological information such as the pulse of the user.

  Based on the voice information, image information, biological information, that is, verbal information and non-verbal information obtained from these sensors, the communication field recognition unit 1 utterance data such as the user's utterance power and its period, the user's gesture / motion -Recognize body movement data such as whispering and companion, and biological sensing data such as user's line of sight, breathing, heartbeat and blink. These utterance data, body movement data, and biological sensing data are referred to as communication rhythm (modal information). The recognized communication rhythm is output to the communication tuning degree calculation unit 2.

  The communication synchronization degree calculation unit 2 calculates the communication synchronization degree based on the communication rhythm (plural modal information) output from the communication field recognition unit 1. The degree of communication synchronization is an index value indicating the height of the shared state of the communication rhythm with the user. The larger this value, the more likely the user will be drawn. The degree of communication synchronization can be, for example, an average value of the user's speech power, a linear weighted sum of the number of user's line-of-sight exchanges, a number of whirlings, etc., that is, a linear weighted sum of various communication rhythms. The calculated communication synchronization degree is output to the interaction control unit 3.

  In addition to the communication rhythm calculated from the communication tuning degree calculation unit 2, the communication rhythm output from the communication field recognition unit 1 is also input to the interaction control unit 3. The interaction control unit 3 refers to the SIRDB 4 based on the input communication rhythm and communication tuning degree.

  The SIRDB 4 stores rules (interaction rules) related to interaction actions that the output unit 5 described later should perform on the user according to the communication rhythm and the communication synchronization degree. The interaction rule is usually constructed so as to take as much as possible the same action as the action taken by humans with respect to the verbal information and non-verbal information that humans feel in communication between humans. More specifically, this interaction rule is constructed under a dynamic triggering strategy of the pull-in phenomenon according to the development stage of communication. According to this interaction rule, even when the communication rhythm is the same, the interaction operation differs between the low and high communication synchronization levels.

  The interaction control unit 3 determines a robot action command, which is an operation command corresponding to an operation to be performed on the user, based on the input communication rhythm and communication synchronization degree according to the interaction rule. The determined robot action command is output to the output unit 5.

  The output unit 5 is a display or a humanoid robot body. In the case of a display, it is possible to adopt a display in which a person image (agent) created by CG (computer graphics) is displayed on the image. An agent or a humanoid robot has a face, a hand, a torso, etc., imitating an actual human being, and can move them. The face can move its eyes, nose and mouth. The output unit 5 realizes a robot action (interaction operation) by moving the face, hands, torso, and eyes, nose, mouth, and the like. Such interaction operations include, for example, line-of-sight change, breathing, heartbeat, blink, gesture, operation, whispering, and conflicting.

  The output unit 5 also has a speaker (not shown) for outputting the voice of the robot, and can speak by moving the displayed mouth and outputting the voice from the speaker. . As described above, it is desirable that the output unit 5 can perform various operations close to human operations.

  The output unit 5 actually performs an interaction operation according to the input robot action command under the control of the interaction control unit 3.

  As shown in the communication process of FIG. 2, the robot system 100 recognizes the communication rhythm by the communication field recognition unit 1 (step S10) → calculates the communication tuning degree by the communication tuning degree calculation unit 2 (step S12) → interaction. The determination of the robot action command by the control unit 3 (step S14) → the interaction operation by the output unit 5 (step S16) is performed in this order.

  While watching the robot action, the user further continues communication such as speech and gesture to the robot system 100. On the other hand, the robot system 100 recognizes the communication rhythm in the communication field recognition unit 1 (step S10), calculates the communication synchronization level in the communication synchronization level calculation unit 2 (step S12), and receives the robot action command in the interaction control unit 3. The determination (step S14) and the interaction operation (step S16) in the output unit 5 are repeated. While watching this interaction operation, the user further continues communication such as speech and gesture to the robot system 100.

  The user and the robot system 100 continue communication such as conversation while repeating such operations.

  As a result of the continuation of communication, the communication rhythm is shared between the user and the robot system 100, and the degree of communication synchronization is increased. As a result, a pull-in phenomenon is induced in the user.

  As described above, the robot system 100 according to the present embodiment recognizes the communication rhythm based on the verbal information and the non-verbal information (plural modal information), and directly obtains the communication synchronization degree based on the communication rhythm. Yes. In the robot system 100, an interaction operation is performed based on the degree of communication synchronization according to the interaction rule constructed based on the dynamic induction strategy of the pull-in phenomenon according to the communication development stage. As described above, since the robot system 100 can communicate with the user under the dynamic triggering strategy of the pull-in phenomenon according to the development stage of communication, the probability of the pull-in phenomenon is increased and the communication is further activated. Can be made.

(Second Embodiment)
A second embodiment of the present invention will be described. FIG. 3 shows a schematic configuration of the robot system 101 according to the present embodiment. As shown in FIG. 3, the robot system 101 according to the present embodiment further includes a communication mode determination unit 6 and a plurality of SIRDBs 41, 42, 43,... Instead of the SIRDB4. The operation of the interaction control unit 3 is different from the robot system 100 according to the first embodiment, and the other points are the same. Therefore, in the present embodiment, the same components as those in the first embodiment are denoted by the same reference numerals as those in FIG. 1, and detailed description thereof is omitted.

  The communication mode determination unit 6 determines a communication mode based on the communication tuning degree. The communication mode indicates the state of the communication place. As the communication mode, for example, various modes with different degrees of communication synchronization such as an initial meeting state, a topic providing state, and a topic excitement state can be set. By setting such a communication mode, the robot system 101 changes the interaction rule according to the communication mode in order to efficiently increase the degree of communication synchronization and easily induce the pull-in phenomenon, and according to the communication mode. A task for systematically changing the control state of the output unit 5 can be constructed. This makes it easier to develop a dynamic induction strategy for the pull-in phenomenon.

  As many SIRDBs 41, 42, 43... Are prepared as there are communication modes, and each of them corresponds to one of the communication modes.

  The communication mode determination unit 6 determines the current communication mode based on the communication tuning level output from the communication tuning level calculation unit 2 and outputs the communication mode to the interaction control unit 3. The interaction control unit 3 selects the SIRDB corresponding to the determined communication mode from the plurality of SIRDBs 41, 42, 43. Then, the interaction control unit 3 refers to the selected SIRDB, and outputs the communication rhythm output from the communication field recognition unit 1 and the communication synchronization degree calculation unit 2 according to the interaction rules stored in the SIRDB. A robot action command is determined based on the communication synchronization level. The output unit 5 performs an interaction operation according to the communication mode at that time in accordance with the robot action command.

  As described above, according to the present embodiment, the induction phenomenon induction strategy can be dynamically changed according to the state of the communication field, so that the probability of the induction phenomenon can be increased and communication can be further activated. it can.

(Third embodiment)
A third embodiment of the present invention will be described. FIG. 4 shows a schematic configuration of the robot system 102 according to the present embodiment. The robot system 102 further includes a user internal state estimation unit 7, a point provided with a plurality of SIRDBs 41, 42, 43... Instead of the SIRDB 4, and the operation of the interaction control unit 3 in the first embodiment. It is different from the robot system 100 according to the embodiment, and the other points are the same. Therefore, in this embodiment, the same components as those in the first embodiment are denoted by the same reference numerals as those in FIG. 1, and detailed description thereof is omitted.

  As shown in FIG. 4, the user internal state estimation unit 7 inputs the communication rhythm output from the communication field recognition unit 1. The user internal state estimation unit 7 estimates the user's internal state based on the communication rhythm. A user internal state is a user's mental state, such as a user's tension state and a pleasant state. The internal state of the user can be divided into, for example, (tension, pleasant), (tension, unpleasant), (relaxed, pleasant), (relaxed, unpleasant), and the like.

  As many SIRDBs 41, 42, 43... Are prepared as there are internal states of the user, and each of them corresponds to the internal state of one of the users. For example, one SIBDB can be prepared for each state such as (tension, pleasure), (tension, discomfort), and (relaxation, pleasure).

  The user internal state estimation unit 7 outputs the estimation result of the user internal state to the interaction control unit 3. The interaction control unit 3 selects the SIRDB corresponding to the internal state of the user from the plurality of SIRDBs 41, 42, 43. Then, the interaction control unit 3 refers to the selected SIRDB, and based on the communication rhythm output from the communication field recognition unit 1 and the communication synchronization level output from the communication synchronization level calculation unit 2 according to the interaction rule. To determine the robot action command. The output unit 5 performs an interaction operation according to the internal state of the user at that time in accordance with the robot action command.

  As described above, according to the present embodiment, the induction phenomenon induction strategy can be dynamically changed according to the internal state of the user, so that the probability of the induction phenomenon can be increased and communication can be further activated. it can.

(Fourth embodiment)
A fourth embodiment of the present invention will be described. FIG. 5 shows a schematic configuration of the robot system 103 according to the present embodiment. The robot system 103 is different from the robot system 102 according to the third embodiment in that it further includes an utterance mind estimation unit 8 and the operation of the interaction control unit 3, and the other points are the same. Therefore, in this embodiment, the same components as those in the third embodiment are denoted by the same reference numerals as those in FIG. 4, and detailed description thereof is omitted.

  As shown in FIG. 5, the utterance mind estimation unit 8 inputs the user's internal state output from the user internal state estimation unit 7. The utterance mind estimation unit 8 estimates an index value (hereinafter referred to as “utterance mind”) indicating whether or not the user has an intention to speak based on the internal state of the user.

  As many SIRDBs 41, 42, 43,... Are prepared as the number of values indicated by the utterance mind, and each corresponds to one of the utterance mind values.

  The utterance mind estimation unit 8 estimates the utterance mind based on the user internal state output from the user internal state estimation unit 7. The utterance mind is generally set so that the value becomes high when the user is focused on the robot system 103 (output unit 5) with a line of sight or when the tension is high. For example, the value can be set to 0 when it is considered that the user is not trying to speak, and the value can be set to 1 when the user is considered to be speaking.

  The estimation result of the utterance mind is output to the interaction control unit 3. The interaction control unit 3 selects the SIRDB corresponding to the user's utterance mind from the plurality of SIRDBs 41, 42, 43. Then, the interaction control unit 3 refers to the selected SIRDB, and based on the communication rhythm output from the communication field recognition unit 1 and the communication synchronization level output from the communication synchronization level calculation unit 2 according to the interaction rule. To determine the robot action command. The output unit 5 performs an interaction operation according to the utterance mind at that time in accordance with the robot action command.

  As described above, according to the present embodiment, the induction phenomenon induction strategy can be dynamically changed according to the user's utterance mind, so that the probability of the induction phenomenon can be increased and communication can be further activated. it can.

(Fifth embodiment)
A fifth embodiment of the present invention will be described. This system is particularly used for mediating communication between users for a plurality of users. FIG. 6 shows a schematic configuration of the robot system 104 according to the present embodiment. The robot system 104 is further provided with the inter-user information estimation unit 9 and the operation of the interaction control unit 3 is different from the robot system 102 according to the third embodiment, and the other points are the same. Therefore, in this embodiment, the same components as those in the third embodiment are denoted by the same reference numerals as those in FIG. 3, and detailed description thereof is omitted.

  As illustrated in FIG. 6, the inter-user information estimation unit 9 inputs the user internal state output from the user internal state estimation unit 7. The inter-user information estimation unit 9 estimates inter-user information indicating a social relationship between users based on the internal state of the user. Such inter-user information includes, for example, an index value that indicates whether or not the user has a close relationship. For example, when the user is very relaxed, it can be determined that the partner is close.

  As many SIRDBs 41, 42, 43... Are prepared as the number of user-to-user information, and each corresponds to the state of any one of the user-to-user information.

  The estimation result of the information between users is output to the interaction control unit 3. The interaction control unit 3 selects a SIRDB corresponding to the information between users from the plurality of SIRDBs 41, 42, 43. Then, the interaction control unit 3 refers to the selected SIRDB, and based on the communication rhythm output from the communication field recognition unit 1 and the communication synchronization level output from the communication synchronization level calculation unit 2 according to the interaction rule. To determine the interaction action. The output unit 5 performs an interaction operation according to the utterance mind at that time in accordance with the robot action command.

  As described above, according to the present embodiment, the induction phenomenon induction strategy can be dynamically changed according to the relationship between users, so that the occurrence probability of the induction phenomenon can be increased and communication can be further activated. it can.

(Sixth embodiment)
A sixth embodiment of the present invention will be described. FIG. 7 shows a schematic configuration of the robot system 105 according to the present embodiment. The robot system 105 further includes an episode storage unit 10, an episode storage database (hereinafter abbreviated as “ESDB”) 11, and an episode learning unit 12, and the operation of the interaction control unit 3 is the above. This is different from the robot system 100 according to the first embodiment, and the other points are the same. Therefore, in this embodiment, the same components as those in the first embodiment are denoted by the same reference numerals as those in FIG. 1, and detailed description thereof is omitted.

  As shown in FIG. 7, the episode accumulation unit 10 is output from the communication rhythm output from the communication field recognition unit 1, the communication synchronization level output from the communication synchronization level calculation unit 2, and the interaction control unit 3. Enter the robot action command. The episode accumulation unit 10 accumulates the communication rhythm, the communication synchronization degree, and the robot action command in the ESDB 11.

  The ESDB 11 is a database that stores the relationship between the communication rhythm and communication synchronization level and the robot action command. More specifically, the ESDB 11 stores the communication rhythm and the communication synchronization degree and the robot action commands searched based on them in association with each other. Further, the ESDB 11 stores the communication rhythm and the communication tuning degree as the user's reaction to the interaction operation of the output unit 5 performed under the interaction control unit 3 based on the operation command in association with each other.

  For example, consider an interaction operation at a certain time t (t is an arbitrary positive real number). As a premise, in the robot system 105, the interaction operation at the time t is based on the robot action command determined based on the communication rhythm and the communication synchronization degree at the time tb (b is a positive real number). Further, it is assumed that the user's reaction to the interaction operation at time t is recognized in the communication field at time t + a (a is a positive real number). In this case, the ESDB 11 stores the robot action command at time t, the communication rhythm and communication tuning degree at time t + a, and the communication rhythm and communication tuning degree at time t−b in association with each other.

  The episode learning unit 12 refers to the ESDB 11 and adjusts the interaction rules stored in the SIRDB 4. For example, when the communication synchronization level at time t + a is lower than the communication synchronization level at time t−b associated with the robot action command at time t, the episode learning unit 12 performs other interaction operations. Change the SIRDB4 interaction rules as determined.

  In this way, the episode learning unit 12 repeatedly changes the interaction rule of the SIRDB 4. As a result of this repetition, the relationship between the communication rhythm and the communication tuning degree and the interaction operation is learned, and the interaction rule in the SIRDB 4 is optimized so that the communication tuning degree is efficiently increased.

  If the user's tension state can be estimated, learning by the episode learning unit 12 may be performed based on whether or not the user's tension state is reduced.

  As described above, according to the present embodiment, the interaction rule is optimized based on the actual performance of communication, and communication is performed under the optimized interaction rule. Thereby, the onset phenomenon occurrence probability can be increased and communication can be further activated.

(Seventh embodiment)
A seventh embodiment of the present invention will be described. FIG. 8 shows a schematic configuration of the robot system 106 according to the present embodiment. The robot system 106 differs from the robot system 100 according to the first embodiment in that the robot system 106 further includes a user personality information database (hereinafter abbreviated as “UPIDB”) 13 and the operation of the interaction control unit 3. The other points are the same. Therefore, in this embodiment, the same components as those in the first embodiment are denoted by the same reference numerals as those in FIG. 1, and detailed description thereof is omitted.

  The UPIDB 13 stores user personal information. Such information includes personal information such as the name, birthplace, work history, and hobbies of the individual user, and information regarding the user's ability and personality such as the user's social skills and psychological analysis results. Information stored in the UPIDB 13 is referred to by the interaction control unit 3 and is used by the interaction control unit 3 to determine an interaction operation.

  The interaction rule in SIRDB4 is a rule in which the interaction operation differs depending on the personal information of the user. Even if the communication rhythm and the communication synchronization degree are the same, if the user is different, the resulting interaction is performed. The behavior can be different.

  As described above, according to the present embodiment, the induction phenomenon induction strategy can be dynamically changed according to the personal information of the user, so that the probability of the induction phenomenon can be increased and communication can be further activated. it can.

Next, further detailed embodiments of the present invention will be described with reference to the drawings.
(First embodiment)
First, a first embodiment of the present invention will be described. This example corresponds to the robot system 101 (see FIG. 3) according to the second embodiment.

  As a premise, a communication field to which the robot system 101 according to the present embodiment is applied will be described. As shown in FIG. 9A, in this communication field, two users H1 and H2 face each other across the table 30 and are in a state where they can talk. The robot system 101 according to the present embodiment provides support for smoothly promoting communication between the users H1 and H2.

  These users H1 and H2 are the first meeting. Therefore, in this embodiment, what is executed by the robot system 101 should be called an initial meeting introduction task.

  The output unit 5 of the robot system 101 is a display. On the screen of the output unit 5, an agent R that is a person image as shown in FIG. 9B is displayed. The agent R can perform various interaction operations by CG (computer graphics). The users H1 and H2 can see the interaction operation of the agent R displayed on the screen of the output unit 5.

  As shown in FIG. 9A, a microphone 31 is attached to each of the chests of the users H1 and H2, and an acceleration sensor 32 is attached to the top of the head. In addition, two cameras 33 and 34 for capturing images of the users H1 and H2 are installed on the table. The microphone 31, the acceleration sensor 32, and the cameras 33 and 34 constitute a part of the communication field recognition unit 1.

  In the present embodiment, the communication field recognition unit 1 detects the voice data of the users H1 and H2 based on the output of the microphone 31, detects the whispering of the users H1 and H2 based on the output of the acceleration sensor 32, Based on the output images of the cameras 33 and 34, the body movements of the users H1 and H2, such as the faces of the users H1 and H2 and the direction of the line of sight, are detected. The communication field recognition unit 1 recognizes the communication rhythm based on these sensing results.

  In the present embodiment, basic actions such as whispering, gaze, face orientation, and instructions of two users H1 and H2 are defined based on the following functions. The values of these functions are 1 if the operation described on the right side of the function (the operation detected from the sensing result) is performed, and 0 if no operation is performed. In this embodiment, the communication rhythm is recognized based on these functions.

・ Nod (H1, t): H1 goes at time t.
Utterance (H1, t): H1 speaks at time t.
Utterance (H1 → H2, t): H1 speaks to H2 at time t.
TerminateUtterance (H1, t): H1 ends utterance at time t.
Gaze (H1 → H2, t): H1 is looking at H2 at time t.
Face (H1 → H2, t): H1 turns his face to H2 at time t.
Gaze (H1⇔H2, t): H1 and H2 are simultaneously looking at time t (gaze matching state).
Face (H1⇔H2, t): H1 and H2 are facing their faces simultaneously at time t (face-to-face state).
・ TurnGaze (R, H1 → H2, t): R turns the line of sight from H1 to H2 at time t.
TurnUtterance (R, H1 → H2, t): R causes H1 to speak to H2 at time t.
Direct (H1 → H2, t): H1 indicates the direction of H2 at time t.
SilentTime (H1, t): Silent period at time t of H1 UtterancePower (H1, t): Volume of generated sound at time t of H1

  The argument of each function indicates the subject of the operation and the time when the operation was performed. Note that the interaction operation of the agent R can also be expressed by this function.

  The communication field recognition unit 1 obtains the value of each function based on the sensing result, and recognizes the communication rhythm based on these functions. The recognized communication rhythm is output to the communication tuning degree calculation unit 2 and the interaction control unit 3.

  The communication tuning degree calculation unit 2 calculates an evaluation function Eval (t) as the communication tuning degree at time t based on these communication rhythms. The communication synchronization degree Eval (t) is represented by a linear weighted sum of communication rhythms such as the average value of the speech powers of the users H1 and H2, the number of line-of-sight exchanges, and the number of strokes. Four evaluation functions Eval (t) [1] to Eval (t) [4] corresponding to the two communication modes are calculated.

ここで、γ、δは、正規化パラメータであり、式（１）と式（３）とで、γ、δの値は異なる。上記式（１）は、エージェントＲとユーザＨ１との間、エージェントＲとユーザＨ２との間で、それぞれの引込現象が発現されたか否かを評価するための評価関数である。上記式（２）は、ユーザＨ１、Ｈ２が向き合って対話を始めたか否かを評価するための評価関数である。上記式（３）、式（４）は、エージェントＲと２人のユーザＨ１、Ｈ２のスムーズな会話が確立されたか否かを評価するための評価関数である。

Here, γ and δ are normalization parameters, and the values of γ and δ are different between Expression (1) and Expression (3). The above equation (1) is an evaluation function for evaluating whether or not the respective pull-in phenomenon has occurred between the agent R and the user H1, and between the agent R and the user H2. The above equation (2) is an evaluation function for evaluating whether or not the users H1 and H2 have faced each other and started a conversation. The above formulas (3) and (4) are evaluation functions for evaluating whether a smooth conversation between the agent R and the two users H1 and H2 has been established.

  As will be described later, when the communication mode is in the topic provision state (the initial stage of conversation providing the topic), it is necessary to check the reaction of the users H1 and H2 in detail. In addition to the communication synchronization degree Eval (t) [2], the degree calculation unit 2 reacts when the agent R shown by the following equation notifies the user H1 of the topic information of the user H2 React (H2 → H1, t) Then, the reactivity React (H1 → H2, t) of the user H2 when the user H1 is notified of the topic information of the user H1 is also calculated as the communication synchronization degree.

ここで、α、βは、正規化パラメータである。
React(H2→H1,t)、React(H1→H2,t)は、エージェントＲによる話題提供が、ユーザＨ１、Ｈ２のコミュニケーションのきっかけとして成り得たか否かを評価するための評価関数である。

Here, α and β are normalization parameters.
React (H2 → H1, t) and React (H1 → H2, t) are evaluation functions for evaluating whether or not the topic provision by the agent R can be triggered by the communication between the users H1 and H2.

  The calculated communication synchronization degree Eval (t) is output to the communication mode determination unit 6.

  In this embodiment, five communication modes are prepared. FIG. 10 shows transition diagrams of five communication modes. This transition diagram represents the first meeting introduction task. As shown in FIG. 10, in this embodiment, in addition to the initial state, four communication modes are prepared: a greeting / first meeting state, a topic providing state, a topic digging state, and a topic excitement state.

  The initial state is a communication mode before the agent R and the two users H1 and H2 who meet for the first time gather in the same communication area.

  The greeting / first meeting state is a communication mode when two users H1 and H2 who are first meeting dodge each other and start a conversation. In this state, the degree of communication synchronization is low and is nearly zero.

  The topic providing state is a state in which a topic is provided for the purpose of allowing the two users H1 and H2 to face each other and have a conversation, thereby creating the beginning of the conversation. In this state, the degree of communication synchronization is slightly higher than that in the greeting / first meeting state.

  The topic digging state is a state when the provided topic is digged down in order to aim for a smooth conversation between the agent R and the two users H1 and H2. In this state, the degree of communication synchronization is higher than the topic provision state.

  The topic excitement state is a state in which the communication rhythm is shared and the communication rhythm is shared as a result of delving into the in-depth topic.

  As shown in FIG. 10, the communication mode changes from the initial state to the greeting / first meeting state and further to the topic providing state as the communication synchronization level increases. Thereafter, the communication mode transitions between a topic providing state, a topic digging state, and a topic excitement state according to the degree of communication synchronization.

  In the initial meeting introduction task, the most desirable flow is a flow in which the communication mode transitions from greeting / first meeting state → topic providing state → topic digging state → topic excitement state. When the topic is in a lively state and the task end condition is satisfied in this state, the robot system 101 ends the initial meeting introduction task, assuming that the role has been completed.

  The operation of the communication mode determination unit 6 will be described. When the two users H1 and H2 gather and the presence of both is detected by the cameras 33 and 34, the communication mode determination unit 6 changes the communication mode to the greeting / first meeting state.

  Thereafter, the communication mode determination unit 6 sets the communication tuning degrees Eval (t) [1] to Eval (t) [4] to predetermined threshold values Th_Eval [1], Th_Eval [2], Th_Eval [3], Th_Eval [4]. ] And the communication mode is determined based on the comparison result. As a result, the communication mode changes as shown in FIG. The relationship between the threshold values is Th_Eval [4]> Th_Eval [3]> Th_Eval [2]> Th_Eval [1].

The transition condition from the greeting / first meeting state to the topic providing state is expressed by the following expression.
Eval (t) [1] = 1 (= Th_Eval [1]) (7)
The fact that this transition condition is satisfied means that, as shown in the above formula (1), the respective pull-in phenomenon has occurred between the agent R and the user H1, and between the agent R and the user H2. Show.

The communication mode determination unit 6 compares React (H2 → H1) and React (H1 → H2) with a certain threshold value Th_React. The transition condition from the topic providing state to the topic digging state is as follows.
React (H2 → H1) ∧React (H1 → H2) ≧ Th_React and Eval (t) [2] ≧ Th_Eval [2] (8)

  The fact that the transition condition is satisfied means that the topic provision by the agent R succeeds and the two users H1 and H2 face each other as shown in the above formulas (2), (5), and (6). Indicates that the conversation has begun.

The transition condition from the topic digging state to the topic excitement state is as follows.
Eval (t) [3] ≧ Th_Eval [3] (9)

  The fact that the transition condition is satisfied indicates that a smooth conversation between the agent R and the two users H1 and H2 has been established as shown in the above equation (3).

The initial meeting introduction task end condition is as follows.
Eval (t) [4] ≧ Th_Eval [4] (10)
Thus, the communication mode determination unit 6 changes the communication mode by determining whether or not the transition condition is satisfied.

  Next, the operation of the interaction control unit 3 will be described. The interaction control unit 3 selects the SIRDB corresponding to the determined communication mode. Then, the interaction control unit 3 controls the interaction operation of the output unit 5 in accordance with the selected SIRDB interaction rule.

  In the SIRDB corresponding to the greeting / first meeting state, interaction rules are defined such that the agent R performs various interaction operations with the aim of smooth conversation between the agent R and the user H1, and between the agent R and the user H2. More specifically, the interaction rule is defined so that the agent R generates a conversation rhythm by performing utterance induction such as speaking to the users H1 and H2 voluntarily. Since the agent R takes such an action, a one-to-one pull-in phenomenon between the agent R and the user H1 and between the agent R and the user H2 is likely to occur, and the communication mode is easily shifted to the topic providing state.

  In the SIRDB corresponding to the topic providing state, an interaction rule is defined so that the agent R gives the information of the user H1 to the user H2 and transmits the information of the user H2 to the user H1. Furthermore, in this SIRDB, an interaction rule is defined in which both parties give their opinions on the same topic, or the agent R guides the line of sight to guide them to confront each other. By this guidance, it becomes possible to solve the “problem in which communication does not occur because of a conversation” that occurs between first-time users, and the communication mode can be easily shifted to the topic digging state.

  In the SIRDB corresponding to the topic digging state, an interaction rule is defined such that the agent R asks a question and enters the topic content with which the users H1 and H2 are interacting. Since the agent R takes such an action, the communication mode is easily changed to the topic excitement state.

  In the topic excitement state, an interaction rule is set so that the agent R tunes in on the spot as a listener so that he or she can speak appropriately or make a match. Thereby, consideration is given to prevent agent R from excessively interfering with both of the already popular topics.

FIG. 11 shows a basic example of an interaction rule. The interaction rule shown in FIG. 11 includes the following three rules.
-[Rule 1] whispering synchronization rule: If the opponent speaks, it will whisper immediately.
[Rule 2] Utterance timing rule: An utterance is made if a silent section lasts for a certain time (0.45 seconds) or longer and the last voice data is determined as the end of a sentence.
[Rule 3] Whisper / utterance timing rule according to the other party's utterance:
“If there is a silent period lasting for a certain time (0.45 seconds) and it is not the end of the sentence, it will be heard with a probability of 20%.” Speak with the probability of ".
Here, the determination of whether or not the sentence end is performed by performing morphological analysis on the last speech data and detecting whether or not the part of speech often appears at the end of the sentence, such as a particle or a final particle. Is possible.

  The agent R in the output unit 5 is in the neutral position shown in FIG. 12A when no robot action command is input. When a robot action command is output from the interaction control unit 3, the agent R performs any one of speech, whispering, and gesture interaction operations as shown in FIGS. 12 (B) to 12 (D).

  13A to 13E show the presence / absence of utterance, the amount of speech, the degree of line-of-sight coincidence, the degree of communication synchronization, and how the communication mode changes over time. (1) to (4) in FIG. 13E show a greeting / initial meeting state, a topic providing state, a topic digging state, and a topic excitement state, respectively. As comprehensively shown in FIGS. 13A to 13E, as the time elapses, the number of whispers increases, the speech power increases, and the line-of-sight frequency increases. In addition, the degree of communication synchronization gradually increases as they increase. As a result, the communication mode transitions from greeting / initial meeting state → topic providing state → topic digging state → topic excitement state.

  As described above, in the robot system 101 according to the present embodiment, the communication tuning degree is calculated based on the communication rhythm, and the communication mode is changed according to the communication tuning degree. H2 communication can be further activated.

(Second embodiment)
Next, a second embodiment of the present invention will be described. This example corresponds to the robot system 102 according to the third embodiment.

  This embodiment is also applied to the communication place shown in FIGS. 9A and 9B, as in the first embodiment.

  As described in the third embodiment, the user internal state estimation unit 7 included in the robot system 102 estimates the internal states of the users H1 and H2 based on the communication rhythm. In the present embodiment, the user internal state estimation unit 7 is an internal state of the users H1 and H2, that is, a tension state (a state of tension or relaxation) or a pleasant state (pleasant or uncomfortable). State). In general, sensing errors are stochastically included in the communication rhythm, that is, the sensing data regarding the eyes, blinks, and facial expressions of the users H1 and H2. Therefore, a probabilistic state transition model of the internal state of the user by a dynamic Bayesian network as shown in FIG. 14A is used for estimating the internal states of the users H1 and H2.

  First, the tension state estimation method will be described. Generally, if the line-of-sight matching frequency is low and the blinking frequency is increased, the tension state of the users H1 and H2 is estimated to increase with the passage of time. Therefore, in this embodiment, the user internal state estimation unit 7 has the detection probability p1 (detection probability p1 with gaze matching, detection probability 1-p1 without gaze matching) and the number of blinks. Based on the detection probability p2 of whether or not the threshold value is exceeded (detection probability p2 with blinking, detection probability 1-p2 without blinking), the time change of the user's tension state (tensity q1 and relaxation degree q2) is changed. , Calculated at predetermined time intervals (..., T-1, t,...).

  Next, a method for estimating a pleasant state will be described. The user internal state estimation unit 7 determines the pleasant state by using a detection probability p2 of whether or not the number of blinks is equal to or greater than a certain threshold (detection probability p2 with blink, detection probability 1-p2 without blink) and facial expression change. It estimates based on the feature-value to show. Here, as feature quantities indicating facial expression changes, feature quantities F1 to F6 calculated from the positional relationship between eyebrows, eyes, and mouth based on a FACS (Facial Expression Coding System) model as shown in FIG. Is used. In this embodiment, distances F1 to F6 are obtained from the imaging results of the cameras 33 and 34. For example, when a human laughs, it is considered that the distance F1 between the eyes and the mouth is shortened. Here, for example, it is assumed that the detection probability p3 that the distance F1 between the eyes and the mouth is equal to or less than the threshold Th and the detection probability 1-p3 that the distance F1 is greater than the threshold Th are obtained.

  The user internal state estimation unit 7 performs emotion recognition based on the coupling probability between the detection probability (p2, 1-p2) that there is blinking and the detection probability (p3, 1-p3) of the facial feature amount. Learning is performed, and the temporal change of the pleasant state (pleasant state degree q3, unpleasant state degree q4) is calculated at predetermined time intervals (..., T-1, t,.

  The line-of-sight coincidence detection probability p1 is used as an index indicating the internal state of the user in the estimation of the utterance mind in the fourth embodiment.

  In addition, in this embodiment, as a rule referred to by the interaction control unit 3, in addition to the interaction rule as shown in FIG. 11, when the agent R speaks to one of the two users H1 and H2, the user A rule of speaking to a user with a low degree of pleasure output from the internal state estimation unit 7 is added. In this rule, when the pleasure levels of the users H1 and H2 are the same value, the interaction control unit 3 controls the output unit 5 so that the agent R speaks to a user with a low degree of tension. If the pleasure levels of the users H1 and H2 are the same value and the tension levels of the users H1 and H2 are also the same value, it may be determined randomly to which user the agent R speaks.

  As described above, in this embodiment, the interaction operation is performed in consideration of the internal state of the users H1 and H2 such as the tension state and the pleasant state. Therefore, the operation is dynamically performed according to the internal state of the users H1 and H2. Will be adjusted. Communication of the users H1 and H2 who are first meeting can be further activated.

(Third embodiment)
Next, a third embodiment of the present invention will be described. This example corresponds to the robot system 103 according to the fourth embodiment.

  This embodiment is also applied to the communication place shown in FIGS. 9A and 9B, as in the first embodiment.

  The utterance mind estimation unit 8 is based on the tension state and the pleasant state of the users H1 and H2 estimated by the user internal state estimation unit 7 (the internal state estimated by the model shown in FIG. 14A). Is estimated.

  For example, when the users H1 and H2 are looking toward the agent R, it is assumed that there is an utterance mind with a probability of p1. When the user's degree of tension is equal to or greater than a certain threshold, it is determined that there is an utterance mind with a probability of q1. Further, when the user's degree of pleasure is equal to or greater than a certain threshold value, it is assumed that there is an utterance mind with a probability of q3. The final utterance mind is the combined probability of these probabilities. The utterance mind estimation unit 8 outputs the utterance mind estimated to the interaction control unit 3.

  Further, in this embodiment, as a rule referenced by the interaction control unit 3, in addition to the interaction rule as shown in FIG. 11, when the agent R speaks to either of the two users H1 and H2, the utterance mind A rule of speaking to a user who has a speech mind output from the estimation unit 8 is added. According to this rule, if both the users H1 and H2 have an utterance mind, or neither of them has an utterance mind, it is possible to randomly determine to which user H1 and H2 the agent R utters. Good.

  Thus, in this embodiment, the agent R speaks to the users H1 and H2 in consideration of the user's intention to speak. For this reason, it is possible to reduce the load felt by the users H1 and H2, and to activate the communication of the users H1 and H2 who are first meeting.

(Fourth embodiment)
Next, a fourth embodiment of the present invention will be described. The robot system according to the present example corresponds to the robot system 104 according to the fifth embodiment.

  This embodiment is also applied to the communication place shown in FIGS. 9A and 9B, as in the first embodiment.

  The inter-user information estimation unit 9 estimates inter-user information based on the estimated tension levels and pleasure levels of the users H1 and H2. In this embodiment, when the tension levels of the users H1 and H2 are both equal to or less than a certain threshold value, and the pleasure levels of the users H1 and H2 are both equal to or greater than a certain threshold value, the user relationship is assumed to be an affinity relationship. In other cases, it is assumed that the relationship is non-affinity. The inter-user information estimation unit 9 outputs this inter-user information to the interaction control unit 3. In addition, the inter-user information estimation unit 9 may define the inter-user information using the total communication time between a plurality of users.

  The interaction control unit 3 controls the output unit 5 in consideration of this user information. For example, when the robot system according to the present embodiment has a communication mode as in the robot system 101 according to the second embodiment, when the communication mode transitions from the topic providing state to the topic digging state. If the inter-user information output from the inter-user information estimation unit 9 is an affinity relationship, the communication mode determination unit 6 is shifted to the topic excitement state instead of the topic excavation state with a certain probability, and the topic excitement state When the transition is made, the output unit 5 may be controlled according to the interaction rule corresponding to the topic excitement state.

  In this way, in this embodiment, if the relationship between users is “affinity”, the agent R can avoid intervening in such a way as to flood the communication between the users H1 and H2, and thus more efficient. Can revitalize communication.

(Fifth embodiment)
Next, a fifth embodiment of the present invention will be described. This example is based on the robot system 105 according to the sixth embodiment, and in addition to those configurations, the user internal state estimation unit 7 and the user, which are components of the robot system 104 according to the fifth embodiment. An inter-information estimation unit 9 is further provided. That is, the robot system of this embodiment has a configuration in which the robot systems 104 and 105 are combined.

  This embodiment is also applied to the communication place shown in FIGS. 9A and 9B, as in the first embodiment.

  In the ESDB 11, for example, information as illustrated in FIG. 15 is stored in time series (0, 1, 2,...) By the episode accumulation unit 10. “User state” indicates the user internal state (degree of tension (high, medium, low)) at time t−b. “Inter-user information” indicates the relationship (non-affinity relationship, affinity relationship) between users at time t−b. “Action of R” is the type of action of agent R at time t (turn a line of sight to H1, ask H1 for a name, give a compliment to H1 and H2, etc). “Evaluation” is the total value of the users H1 and H2 of the decrease values (tensity degree decrease values) of the user H1 and H2 at the time t + a with respect to the tension levels of the users H1 and H2 at the time t−b.

  The episode learning unit 12 refers to the ESDB 11 to determine the relationship between the interaction rules stored in the SIRDB 4, that is, the communication rhythm and the communication tuning degree, and the interaction operation performed by the output unit 5 under the control based on them. The user H1 and H2 are repeatedly changed so as to reduce the degree of tension. In this way, the episode learning unit 12 learns the optimum relationship between the communication rhythm and communication synchronization degree and the interaction operation. Thereby, the interaction rule memorize | stored in ESDB11 is adjusted in the direction which raises a communication synchronization degree efficiently.

  In this learning, the episode learning unit 12 may change the learning rule based on the communication synchronization level output from the communication synchronization level calculation unit 2. For example, if the communication synchronization level is lower than a certain threshold for a certain period or longer, parameters such as the user's tension level and inter-user information used for learning are fixed, aiming at faster optimization convergence (higher learning speed). The number can be reduced.

(Sixth embodiment)
Next, a sixth embodiment of the present invention will be described. This example corresponds to the robot system 106 according to the seventh embodiment.

  This embodiment is also applied to the communication place shown in FIGS. 9A and 9B, as in the first embodiment.

  In the UPIDB 13, personal information such as the user's name, birthplace, work history, and hobby, and information regarding the user's ability and personality such as the user's social skills and psychological analysis results are stored in advance. The information stored in the UPIDB 13 is referred to by the interaction control unit 3 and is used by the interaction control unit 3 to determine the interaction operation.

  For example, when the agent R asks about the hobbies of the users H1 and H2, if the user personality information of the users H1 and H2 includes information on their hobbies, the contents are reflected in the utterance.

  The personal information of the user is also taken into account when determining the wording of the agent R to the users H1 and H2. One of the evaluation indexes of language skills is JICS. For example, in JICS, the wording of a robot can be adjusted by using the user's social skills regarding relationship adjustment (upper and lower relationship management).

  For example, it is assumed that the degree of relationship adjustment of users participating in communication is known from the result of questionnaire survey or the like, and that degree is stored in the UPIDB 13 in advance. It can be estimated that a user whose degree of relationship adjustment is higher than a certain threshold is conscious of the human vertical relationship. For this reason, for such a user, the interaction control unit 3 causes the agent R to use honorific words when speaking. On the other hand, since it can be estimated that a user whose degree of relationship adjustment is lower than a certain threshold is not so conscious about the human vertical relationship, the interaction control unit 3 performs agent R for such a user. To not use honorifics when speaking.

  In this way, the affinity between the agent R and the users H1 and H2 can be further increased.

  In addition to the social skill evaluation index, information related to the personality of the user estimated using the personality theory in the exchange analysis is stored in the UPIDB 13, and the utterance of the agent R to the users H1 and H2 is stored. Can be used for content adjustment.

  In such a theory, for example, the human personality (personality) is a critical kinship, a nurturing kinship, a rational adult soul, an innocent child soul, and an adapted child soul. It is roughly divided into two. In this theory, if a questionnaire survey on personality is conducted, it is possible to analyze the tendency of which of the five personalities is dominant among the personality. The personality of can be estimated to some extent.

  For example, as a result of a questionnaire survey on a certain user, if the critical kinship and the adapted child's mind are superior to the other three mind structures, the user's personality is of a logical type. Presumed. In this case, it is considered that the utterance content for the user belonging to this type should emphasize the reason. Therefore, for a user belonging to this type, the agent R takes an interactive strategy that prompts the user for an action with a reason. In this way, the conversation will proceed smoothly.

  As in the fifth example, a combination of the robot systems according to the above embodiments can be employed as the robot system. For example, the SIRDB may be prepared for at least some combinations of the communication mode, the user internal state, the utterance mind, and the inter-user information, and an interaction rule may be prepared for each combination. Further, in a system including a plurality of SIRDBs according to the combination thereof, episode learning may be performed, or the interaction operation may be changed based on the personal information of the user.

  Further, in each of the above embodiments, the CG model agent R performs an interaction operation. However, the output unit 5 includes various actuators and can be used to move the facial expression, arms, hands, feet, and body of the robot. You may make it use the robot of this. Even in this case, the output unit 5 effectively conveys emotions and attentions such as emotions and cautions of the robot to the user by changing the facial expression of the robot and moving the arms, hands, feet, and body. be able to.

  In this case, the output unit 5 may perform an interaction operation such as blinking of eyes, blinking, waving an arm, waving a neck, expanding / contracting the body, vibrating the body, or generating a beating sound. Further, the output unit 5 may execute an interaction operation such as tearing, rubbing against the user's feet, approaching the user, or jumping. Furthermore, as a method for transmitting the attention object, the output unit 5 may perform an interaction operation such as gazing at the attention object, pointing at the attention object, or approaching the attention object.

  As described above, the robot system may physically have an entity, or may be an agent type that does not have an entity and is displayed on the projection screen or display of the projector as in each of the above embodiments. However, it may be one that only displays characters on the screen or emits sound. In short, the robot system only needs to be capable of performing an operation including at least one of utterance, body movement, and character display.

  In addition, the components of each robot system such as the communication field recognition unit 1, the communication tuning degree 2, and the interaction control unit 3 may be realized only by hardware. In general, it is realized by cooperative operation. In the case of a coordinated operation of the software program and hardware, the CPU provided in the robot system executes the software program stored in a storage device such as a ROM in the same system, so that the function of each unit Is realized.

  In this case, a general-purpose computer can be used as the robot system. In this case, the software program stored in the storage device of the computer is a computer-readable recording such as a CD-ROM (Compact Disc Read Only Memory), a DVD (Digital Versatile Disc), an MO (Magneto Optical Disc), or a flexible disk. It may be stored in a medium and distributed, and installed in a robot system. Further, the program stored in a server device on a communication network such as the Internet may be downloaded to the computer and installed in the robot system.

1 is a block diagram showing a basic configuration of a robot system according to a first embodiment of the present invention. It is a flowchart which shows operation | movement of the robot system of FIG. It is a block diagram which shows the basic composition of the robot system which concerns on the 2nd Embodiment of this invention. It is a block diagram which shows the basic composition of the robot system which concerns on the 3rd Embodiment of this invention. It is a block diagram which shows the basic composition of the robot system which concerns on the 4th Embodiment of this invention. It is a block diagram which shows the basic composition of the robot system which concerns on the 5th Embodiment of this invention. It is a block diagram which shows the basic composition of the robot system which concerns on the 6th Embodiment of this invention. It is a block diagram which shows the basic composition of the robot system which concerns on the 7th Embodiment of this invention. FIG. 9A is a diagram illustrating an example of a communication field, and FIG. 9B is a diagram illustrating an example of an agent. It is a transition diagram of a communication mode. It is a figure which shows an example of an interaction rule. 12A is a diagram illustrating the neutral position of the agent, FIG. 12B is a diagram illustrating the speech operation, FIG. 12C is a diagram illustrating the whispering operation, and FIG. D) is a diagram illustrating a gesture operation. FIG. 13A is a graph showing temporal changes in presence / absence of whispering, FIG. 13B is a graph showing temporal changes in the amount of speech, and FIG. 13C shows temporal changes in line-of-sight matching. FIG. 13D is a graph showing the time change of the communication tuning degree, and FIG. 13E is a diagram showing the time change of the communication mode. FIG. 14A is a diagram showing an example of a probabilistic state transition model indicating the inner state of the user, and FIG. 14B is a feature calculated from the positional relationship between the eyebrows, eyes, and mouth based on the FACS model. It is a figure for demonstrating quantity. It is a figure which shows an example of the information memorize | stored in an episode memory database.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Communication field recognition part 2 Communication tuning degree calculation part 3 Interaction control part 4, 41, 42, 43 Social interaction rule database (SIRDB)
DESCRIPTION OF SYMBOLS 5 Output part 6 Communication mode determination part 7 User internal state estimation part 8 Utterance mind estimation part 9 Inter-user information estimation part 10 Episodic storage part 11 Episodic storage database (ESDB)
12 Episode Learning Department 13 User Personality Information Database (UPIDB)
30 Table 31 Microphone 32 Acceleration sensor 33, 34 Camera 100, 101, 102, 103, 104, 105, 106 Robot system H1, H2 User R agent

Claims (9)

An output unit for performing an interaction operation for a plurality of users;
A recognition unit for recognizing the communication rhythm of the user based on the verbal information and non-verbal information of each of the plurality of users;
Based on the communication rhythm, a synchronization level calculation unit that calculates a synchronization level between the users,
In accordance with the communication rhythm and the degree of synchronization, the output unit stores a rule constructed under a dynamic induction strategy of a pull-in phenomenon according to a communication development stage regarding an interaction operation to be performed on the user. A rule database;
An interaction control that refers to the rule database, searches for an operation command to be performed for the user using the communication rhythm and the degree of synchronization according to the rule, and controls the output unit based on the searched operation command A robot system. A mode determining unit for determining a communication mode which is a state of a communication field based on the degree of synchronization;
A plurality of the rule databases according to the communication mode,
The interaction control unit
From among the plurality of rule databases, select a rule database corresponding to the communication mode determined by the communication mode determination unit,
Referring to the selected rule database, search for an operation command to be performed on the user using the communication rhythm and the degree of synchronization according to the rule, and control the output unit based on the searched operation command The robot system according to claim 1, wherein: Based on the communication rhythm of each of the plurality of users, further comprising a user internal state estimation unit that estimates the internal state of the user,
A plurality of the rule databases are provided according to the internal state,
The interaction control unit
From among the plurality of rule databases, select a rule database corresponding to the internal state estimated by the user internal state estimation unit,
Referring to the selected rule database, search for an operation command to be performed for the user using the communication rhythm and the degree of synchronization according to the rule, and control the output unit based on the searched operation command The robot system according to claim 1 or 2, characterized in that An utterance mind estimating unit that estimates an index value indicating whether or not the user has an intention to speak based on an internal state of each of the plurality of users;
A plurality of the rule databases according to the index value,
The interaction control unit
From among the plurality of rule databases, select a rule database corresponding to the index value estimated by the utterance mind estimation unit,
Referring to the selected rule database, search for an operation command to be performed for the user using the communication rhythm and the degree of synchronization according to the rule, and control the output unit based on the searched operation command The robot system according to claim 3. Based on the internal state of each of the plurality of users, further comprising an inter-user information estimation unit that estimates inter-user information indicating a social relationship between the users,
A plurality of the rule databases are provided according to the information between users,
The interaction control unit
From among the plurality of rule databases, select a rule database corresponding to the inter-user information estimated by the inter-user information estimation unit,
Referring to the selected rule database, search for an operation command to be performed for the user using the communication rhythm and the degree of synchronization according to the rule, and control the output unit based on the searched operation command The robot system according to claim 3. Storing the communication rhythm and the degree of synchronization and the operation command searched based on them in association with each other;
Episodic memory for storing the operation command in association with the communication rhythm and the degree of tuning as a response of each of the users to the interaction operation of the output unit performed under the interaction control unit based on the operation command A database,
An episode storage unit for storing the communication rhythm, the degree of synchronization, and the operation command in the episode storage database;
An episode learning unit that adjusts a rule stored in the rule database by learning an optimal relationship between the communication rhythm and the degree of synchronization and the interaction action with reference to the episode storage database; The robot system according to any one of claims 1 to 5, further comprising: A user personality information database in which personal information of the user is stored;
The interaction control unit
The robot system according to claim 1, wherein the output unit is controlled based on information stored in the user personality information database. A communication activation method using a robot system including an output unit that performs an interaction operation for a plurality of users,
A first step of recognizing the communication rhythm of the plurality of users based on the verbal information and non-verbal information of each of the users;
A second step of calculating a degree of synchronization between the users based on the communication rhythm;
In accordance with the communication rhythm and the degree of synchronization, the output unit stores a rule constructed under a dynamic induction strategy of a pull-in phenomenon according to a communication development stage regarding an interaction operation to be performed on the user. A third database for searching for an operation command to be performed on the user using the communication rhythm and the degree of synchronization according to the rule with reference to the rule database, and controlling the output unit based on the searched operation command And a communication activation method including a process. In a computer that controls a robot system including an output unit that performs an interaction operation for a plurality of users,
A first procedure for recognizing the communication rhythm of the plurality of users based on the verbal information and non-verbal information of each of the users;
A second procedure for calculating the degree of synchronization between the users based on the communication rhythm;
In accordance with the communication rhythm and the degree of synchronization, the output unit stores a rule constructed under a dynamic induction strategy of a pull-in phenomenon according to a communication development stage regarding an interaction operation to be performed on the user. A third database for searching for an operation command to be performed on the user using the communication rhythm and the degree of synchronization according to the rule with reference to the rule database, and controlling the output unit based on the searched operation command A program that executes a procedure.

Images