JP2012181697A - Dialog system, interaction control method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To solve communication gaps in real time and perform a persistent and natural communication in a dialog system.SOLUTION: An interaction state amount calculation part 12 detects an interaction state amount including speaker switching pauses in utterance of a user P, a pitch of an utterance section, and power or mora. A communication synchronization control part 13 calculates a communication synchronization shift amount between the user P and a robot system 10, minimizes the communication synchronization shift amount by continuous pull-in control by a state equation expressing a synchronization model, and simultaneously makes an interaction state amount of the user P similar to that of a robot system 10 by discrete pull-in control of an interaction rule, or makes the interaction state amount of the user P similar to that of the robot system 10 while making the interaction state amount of the robot system 10 similar to that of the user P.

Description

  The present invention relates to a dialog system, a dialog control method, and a program. More specifically, the present invention relates to a dialogue system, a dialogue control method, and a program for performing conversation with a human at home to collect information such as an inquiry.

  Appropriate utterance control, prosodic control, and physical control such as whispering are necessary for natural communication through conversation. In order to smoothly collect information such as medical interviews, appropriate speech timing control, prosodic control, and physical control such as whispering are required according to the state of the user on that day.

  Non-Patent Document 1 and Non-Patent Document 2 relate to an interactive system that uses acoustic features and keywords of utterances as features, and performs interaction and response at appropriate utterance timings by machine learning of human interaction with a decision tree. is there. In this dialogue system, the timing of reciprocity and speaker change is generated in consideration of an average human state derived from offline dialogue learning. In the dialog system of Non-Patent Document 1 or 2, it is not possible to generate an utterance synchronously with the utterance of the other party in real time.

  Non-Patent Document 3 discloses a technique for developing the above-described dialog system and generating response timing in real time. However, in the spoken dialogue system of Non-Patent Document 3, it is shown how the system resolves the gap between the dialogue system and human prosodic information and utterances. Absent.

  Japanese Patent Application Laid-Open No. 2004-151867 discloses a voice dialogue apparatus for the purpose of performing a smooth dialogue in accordance with the sensitivity of a speaker. In the normal state, the speech dialogue apparatus of Patent Document 1 is controlled so that the pause time and speech rate until the response speech is spoken are in a state corresponding to the speech rate of the speaker. When an event occurs that the speaker wants the response voice to finish early during the response voice output period, the response voice is spoken more continuously than the previous speed without changing the pitch. Control to speed up.

  As a robot control technique, Patent Document 2 describes that the movement of a movable part of a robot is regarded as a periodic movement, and a global posture stability control is performed by adjusting the phase thereof. In patent document 2, the method of discretely controlling the phase is used for the purpose of rough terrain breakthrough.

JP 2008-26463 A Japanese Patent Laid-Open No. 2005-96068

Kitaoka, N., Takeuchi, M. Nishimura, R., Nakagawa, S. “Response Timing Detection Using Prosodic and Linguistic Information for Human-Friendly Spoken Dialog Systems, Journal of Japanese Society for Artificial Intelligence Vol.20 No.3, SP-E pp.220-228, 2005 Kitaoka, Nakagawa, et al., “Analysis and timing analysis of collaborative spoken dialogue and response generation based on it”, Grant-in-Aid for Scientific Research report No. 17300064, 2005 Nishimura, Nakagawa, “Spoken Dialogue System Considering Response Timing and Its Evaluation”, IPSJ SLP Research Report 2009-SLP-77 No.2, pp.1-6, 2009

  The speech dialogue apparatus of Patent Document 1 controls prosodic features (pause time and speech speed) of response speech utterance so as to be in a state according to the human speech speed. It is for speed conversion. However, a natural conversation cannot be realized only by adjusting the speaking speed, and a mechanism capable of adaptively controlling the utterance and tone is required.

  In order to collect daily information such as interviews at home without placing a burden on the user, we can grasp the amount of state related to communication resulting from changes in the user's physical condition and mental aspects in real time and collect information continuously. It needs to be done properly. Here, the issue is to collect accurate information through natural communication.

  In order to realize natural communication that does not impose a mental burden on the user, it is premised that the rhythm of the dialogue system and the rhythm of the user, as represented by the speaker change timing, are synchronized. The dialogue system has a configuration via a robot having an entity as hardware or via a robot (CG: computer graphics expression) having no entity. When the robot rhythm has a gap with the user's rhythm, the robot can search for the user's rhythm and bring the robot's rhythm closer to the user's rhythm. It is hard to say that bringing the communication control parameters of the robot closer to that of the user will realize natural communication. Communication is established temporarily, but it is unnatural communication when evaluated over the long term, destroying the impression unique to the robot, the user feels stressed about communication with the robot, and continuous communication May become impossible.

  In addition, there are many cases where communication synchronization is interrupted in the course of communication, and how to solve the communication gap in real time is an issue. To do this, it is necessary to design a communication control method considering the communication dynamics between the user and the robot. As a general methodology for modeling the communication dynamics, a method of expressing the dynamics of the behavior of the state quantity to be controlled by a state equation taking a continuous value is conceivable. The tuning phenomenon can be expressed using a state equation based on the principle of a vibrator used in a heart pacemaker or a resonance model of an electronic circuit. The communication state quantity in the present invention includes the state quantity of “between” (alternative latency) at the time of speaker change, the state quantity of prosodic features including the pitch, power, or mora of the utterance section, and / or whispering, etc. It is considered as a state quantity that indicates the physical behavior of the child. It is assumed that a synchronization phenomenon of these communication state quantities is expressed by a state equation and a pull-in control (continuous pull-in control) between the user and the robot is continuously performed.

  However, it is difficult to perform the pull-in control between the user and the robot only with such a physical tuning model. For example, when there is a gap between the robot's change latency and the user's change latency, the robot searches for the user's change latency and induces the synchronization phenomenon by bringing the robot change latency closer to the user's change latency. However, it is difficult to say that bringing the robot's alternation latency closer to that of the user realizes natural and continuous communication. Further, there is a possibility that the user gives up communication before the synchronization phenomenon appears, and it is necessary to cause the synchronization phenomenon to appear from an early stage of communication.

  Furthermore, in order to realize more natural communication, it is necessary to be conscious of the conversation structure according to the purpose. There are conversation structures according to the purpose, such as discussions at meetings, chats, and healing responses. There is no conversation structure that can handle all types of conversations, and conversations cannot be advanced naturally unless a model corresponding to the type of conversation is established.

  The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a dialog system, a dialog control method, and a program that solve a communication gap in real time and perform continuous and natural communication.

In order to achieve the above object, an interactive system according to the first aspect of the present invention provides:
An interactive system comprising speech recognition means for recognizing the utterance content of a speaker, and response control means for outputting an auditory response by voice and / or a visual response by expression of physical behavior according to the recognition result ,
Detection of state quantities of prosodic features including speaker alternation latency, pitch of speech section, power or mora, and / or detection of state quantities indicating physical behavior of the speaker in the speaker's utterance State quantity detection means to perform;
Means for calculating a communication synchronization deviation amount, which is a deviation amount of an interaction state amount between the speaker and the response control means, from an interaction state amount including the state amount of the prosodic feature or the state amount indicating the physical behavior;
Rule storage means for storing an interaction rule which is a rule for changing an interaction state quantity of the response control means;
Rule selection means for selecting the interaction rule from the rule storage means based on the communication synchronization deviation amount;
The amount of interaction state of the speaker is reduced by discrete pull-in control by the interaction rule selected by the rule selection means, while at the same time minimizing the communication tuning shift amount by continuous pull-in control by a state equation representing a tuning model. Approaching the interaction state quantity of the dialog system, or bringing the interaction state quantity of the dialog system close to the interaction state quantity of the speaker while bringing the interaction state quantity of the speaker close to the interaction state quantity of the dialog system Tuning control means to go,
It is characterized by providing.

The dialogue control method according to the second aspect of the present invention includes:
Dialogue performed by a dialogue system comprising speech recognition means for recognizing the utterance content of a speaker, and response control means for outputting an auditory response by voice and / or a visual response by expression of physical behavior according to the recognition result A control method,
Detection of state quantities of prosodic features including speaker alternation latency, pitch of speech section, power or mora, and / or detection of state quantities indicating physical behavior of the speaker in the speaker's utterance A state quantity detection step to be performed;
Calculating a communication synchronization deviation amount, which is a deviation amount of an interaction state amount between the speaker and the response control means, from an interaction state amount including the state amount of the prosodic feature or the state amount indicating the physical behavior;
A rule selection step of selecting the interaction rule from a rule storage unit that stores an interaction rule that is a rule for changing an interaction state amount of the response control unit based on the communication synchronization deviation amount;
The communication state deviation amount is minimized by continuous pull-in control by a state equation representing a tuning model, and at the same time, the interaction state amount of the speaker is determined by discrete pull-in control by the interaction rule selected in the rule selection step. Approaching the interaction state quantity of the dialog system, or bringing the interaction state quantity of the dialog system close to the interaction state quantity of the speaker while bringing the interaction state quantity of the speaker close to the interaction state quantity of the dialog system The tuning control step
It is characterized by providing.

The program according to the third aspect of the present invention is:
Controlling a dialogue system comprising voice recognition means for recognizing the utterance content of a speaker, and response control means for outputting an auditory response by voice and / or a visual response by expression of physical behavior according to the recognition result On the computer,
In the speaker's utterance, detection of the state of the prosody feature including the alternation latency of the speaker, the pitch of the utterance section, the power or the mora, and / or the state amount indicating the physical behavior of the speaker is performed. A state quantity detection step;
Calculating a communication synchronization deviation amount, which is a deviation amount of an interaction state amount between the speaker and the response control means, from an interaction state amount including the state amount of the prosodic feature or the state amount indicating the physical behavior;
A rule selection step of selecting the interaction rule from a rule storage unit that stores an interaction rule that is a rule for changing an interaction state amount of the response control unit based on the communication synchronization deviation amount;
The communication state deviation amount is minimized by continuous pull-in control by a state equation representing a tuning model, and at the same time, the interaction state amount of the speaker is determined by discrete pull-in control by the interaction rule selected in the rule selection step. Approaching the interaction state quantity of the dialog system, or bringing the interaction state quantity of the dialog system close to the interaction state quantity of the speaker while bringing the interaction state quantity of the speaker close to the interaction state quantity of the dialog system The tuning control step
Is executed.

  ADVANTAGE OF THE INVENTION According to this invention, a communication gap can be solved in real time, and a dialog system can be controlled so that daily information collection may be continuously performed through continuous and natural communication.

It is a block diagram which shows the structural example of the robot system which concerns on Embodiment 1 of this invention. It is a figure which shows the pattern of a drawing-in control system. It is a figure which shows the example of adaptive drawing-in control based on a user state quantity. It is a figure which shows acceleration of synchronous expression by interaction rule learning. It is a figure which shows the phase transition of entrainment control by an interaction rule. It is a figure which shows the example of pull-in control of alternation latency and prosodic features. It is a block diagram which shows the structural example of the robot system which concerns on Embodiment 2 of this invention. It is a block diagram which shows the structural example of the robot system which concerns on Embodiment 3 of this invention. It is a block diagram which shows the conversation structure model in an information collection (inquiry) task. It is a figure which shows the structural example of the continuous information collection (inquiry) system with an action. It is a figure which shows the multi-hierarchical model of drawing-in control. It is a figure which shows the example of prosodic information control based on an interaction rule, and the control between them. It is a figure which shows the example of operation | movement timing control regarding the speech by the decision tree learning, and whispering. It is a block diagram which shows the structural example of the robot system which concerns on Embodiment 4 of this invention. It is a block diagram which shows the physical structural example of the robot system which concerns on embodiment of this invention.

  In the present invention, the interaction rule selected by the learning model is discretely drawn in consideration of the adaptation timing in the process of minimizing the amount of communication tuning deviation in the framework of continuous pull-in control of the tuning model in the dialogue system. By controlling, the user's state quantity approaches the dialog system state quantity, or the user's state quantity approaches the dialog system state quantity while the dialog system state quantity approaches the user state quantity. Then, we propose a new dialogue control method that enables early communication synchronization between users and dialogue systems.

  Embodiments for carrying out the present invention will be described in detail with reference to the drawings. The dialogue system according to each embodiment of the present invention described below is capable of continuous communication control without applying stress to daily information collection through continuous and natural communication.

(Embodiment 1)
First, the configuration of the dialogue system according to the embodiment of the present invention will be described. In the embodiment, hereinafter, the dialogue system will be described by replacing it with a robot system. The robot system may be a character represented on a computer screen without a material entity. An interactive system may not visually have a so-called robot form. The dialogue system includes a case of a voice dialogue system using only voice.

  FIG. 1 shows a schematic configuration of the robot system according to the present embodiment. The robot system 10 includes a sensing unit 11, an interaction state amount calculation unit 12, a communication tuning control unit 13, an interaction rule learning history unit 19, a state equation parameter learning history unit 20, and a robot interaction control unit 14.

  The sensing unit 11 includes, for example, a microphone and a camera. The microphone converts voice uttered by the user P into an electrical signal and generates voice data. The sensing unit 11 captures the user P with a camera and generates moving image data thereof. The sensing unit 11 sends the generated audio data and moving image data to the interaction state amount calculation unit 12 as sensing data.

The interaction state quantity calculation unit 12 recognizes the utterance content from the voice of the user P and sends it to the robot interaction control unit 14. Based on the sensing data, the interaction state quantity calculation unit 12 calculates the interaction state quantity, for example, the change latency of the speaker change, the basic frequency (F ₀ ) of the utterance, the power / mora length, and the like, and the body such as whispering Detect motion events. The interaction state quantity calculation unit 12 sends these interaction state quantities (hereinafter also simply referred to as state quantities) to the communication tuning control unit 13.

  The communication tuning control unit 13 includes a continuous pull-in control unit 15, a state equation parameter storage unit 16, a discrete pull-in control unit 17, and an interaction rule storage unit 18. The interaction rule is a rule that changes the amount of interaction state of the robot system 10. The continuous pull-in control unit 15 receives the interaction state quantity of the user P and the interaction state quantity of the robot system 10 as inputs, and uses the state equation parameters in the state equation parameter storage unit 16 to perform continuous pulling by the state equations learned by parameters. Control. The discrete pull-in control unit 17 selects an interaction rule from the interaction rule storage unit 18 and performs discrete pull-in control using the learned interaction rule. The communication tuning control unit 13 generates interaction control information of the robot system 10 by these pull-in controls, and instructs the robot interaction control unit 14.

  Finally, the robot interaction control unit 14 receives the data recognized by the interaction state quantity calculation unit 12 and the interaction control information from the communication tuning control unit 13, and generates an interaction related to the speech and operation of the robot system 10. The robot interaction control unit 14 generates an interaction related to the speech and operation of the robot system 10 and outputs a voice by voice synthesis. In addition, according to the generated interaction, the arm, face, and body of the robot system 10 are moved.

  The interaction rule learning history unit 19 stores a learning result of an interaction rule described later. The state equation parameter learning history unit 20 stores a learning result of a state equation parameter described later. Hereinafter, a method in which the communication tuning control unit 13 generates the control information for the interaction when there is a gap in the interaction state quantity between the user P and the robot system 10 will be described.

FIG. 2 is a diagram illustrating a pattern of the pull-in control method. In FIG. 2, the state quantity is represented in one dimension for easy understanding. The state quantity is generally multidimensional, and the change in the state quantity is represented by a locus in the state quantity space. As shown in FIG. 2, when the gap between the state quantities of the user P and the robot system 10 is large, the tuning phenomenon does not occur, and it is necessary to search for the level of the state quantities at which the tuning occurs. The tuning phenomenon can be modeled by a nonlinear oscillator. In the present embodiment, formulation is performed using the Van der Pol equation. When the phase of the user P to be synchronized and the time t of the robot system 10 is x (t) and y (t) and the waveform displacement of the observed state quantity is F (x, y, t), the nonlinear oscillator is , X ′ (t) = y (t)
y ′ (t) = ε (1 −x ² (t)) y (t) + α × F (x, y, t) (1)
Is formulated by The orbit of the solution of this simultaneous ordinary differential equation is a limit cycle. Here, ε represents nonlinearity, and has a stable limit cycle when ε> 0. α is an influence parameter.

In the equation (1), the observed state quantity is single. However, when tuning is performed in a space composed of a plurality of observed state quantities, generally, α × F (x, y , T)
α × F (x, y, t) + β × G (x, y, t) +.
This is realized by linear combination as follows. Further, an equation may be established independently for each of the plurality of state quantities and controlled independently. When the independence between the state quantities is clear and there is no correlation between the state quantities, the independent control is more suitable. Whether it is linear combination or independent control is determined by the relationship between the state quantities.

  In order to perform communication control by the tuning model as shown in the equation (1), it is necessary to induce to a state quantity in which tuning is expressed. When there is a gap between the state quantity of the robot system 10 and the state quantity of the user P, the robot system 10 searches for the state quantity of the user P as shown in FIG. It is possible to induce the tuning phenomenon by approaching the state quantity of the user P. However, compliantly, bringing the communication control parameter of the robot system 10 close to that of the user P temporarily establishes communication, but also destroys the unique impression of the robot system 10, so that the user P can You may feel stressed and you may not be able to communicate continuously.

  As shown in FIG. 2B, there is also a strategy for starting the interaction rule of the robot system 10 so that the state quantity of the user P is forcibly brought close to the state quantity of the robot system 10. However, this strategy is not always successful. Therefore, for example, as shown in FIG. 2C, the robot system 10 is configured so that the state quantity of the robot system 10 is once brought close to the state quantity of the user P and then gradually pulled into the original state quantity of the robot system 10. Change the amount of state. Thus, the robot system 10 can reduce the load on the user P while adaptively bringing the state amount of the robot system 10 close to the state amount of the user P and accelerating the onset of synchronization while activating the interaction rule. The natural impression of 10 itself can be maintained.

  The interaction rule may have parameters such as the timing of starting it (gap conditions, etc.), the direction of the tuning operation and the speed of change, the timing of turning to pull-in, and the speed of change of pull-in. When the state quantity is multidimensional, the gap is also a vector, and the number of interaction rules is not necessarily one. Further, the change pattern of the interaction operation of the interaction rule is not necessarily one.

  The communication tuning control unit 13 selects an interaction rule according to the state quantity gap as described above. After activating the selected interaction rule, the interaction rule is evaluated based on, for example, the duration of synchronous communication or the amount of speech. That is, in correspondence with the selected interaction rule, the synchronized onset duration or the amount of utterance at that time is stored, and an interaction rule with a large average of past results (or the latest moving average, etc.) is selected. Alternatively, the parameters of the interaction rule are changed so as to converge to a parameter having a large synchronized expression duration and an evaluation value of the amount of speech. The communication tuning control unit 13 stores the interaction rule learning result in the interaction rule learning history unit 19 in this way.

  FIG. 3 shows an example of adaptive pull-in control based on user state quantities. As shown in FIG. 3, there are many cases in which communication synchronization is interrupted during the progress of communication synchronization. In such a case, what kind of interaction rule should be activated at which timing is important. Selection of such an interaction rule is performed based on the interaction rule learning history.

  FIG. 4 is a diagram illustrating acceleration of synchronized expression by interaction rule learning. The communication tuning control unit 13 selects an interaction rule that reduces the difference in the state quantity to the tuning expression level with the steepest slope (shortest time).

  FIG. 5 is a diagram illustrating a phase transition of the pull-in control based on the interaction rule. As shown in FIG. 5, once the limit cycle is entered, the trajectory converges to a phase stable point. However, in a multidimensional state quantity space, there are generally a plurality of limit cycles having stable convergence points. By applying an appropriate interaction rule, it is possible to make a phase transition to a more stable limit cycle.

  FIG. 6 shows an interaction state amount pull-in control process composed of the state amount during the speaker change and the state amount of the prosodic feature in the utterance section. Here, the change latency (time from the end of the utterance to the start of the other party's utterance) is taken up as the state quantity during the speaker change. An utterance interval (time from the start of utterance to the start of the other party's utterance) may be used instead of the alternate latency. The deviation amount between the user P and the robot system 10 in the time interval of the alternating latency is converged and minimized while oscillating in a stable limit cycle by the state equation of the tuning model expressed by the equation (1). .

A typical fundamental frequency F ₀ (pitch feature) will be described as a prosodic feature. The fundamental frequency (F ₀ ) is calculated at each time of the sampling interval, and an average F ₀ value averaged in the utterance interval is set as a fundamental frequency representing the utterance interval. The fundamental frequency representing the utterance interval may be the fundamental frequency of the utterance phrase ending mora. Here, from the viewpoint of tuning control, the policy is not to apply the control with the absolute value of the fundamental frequency but to apply the control according to the degree of change of the fundamental frequency. This is because the fundamental frequency depends on the voice quality and there are individual differences, so that tuning control is difficult with absolute amounts.

  The degree of change of the fundamental frequency is defined as a value of fluctuation of the fundamental frequency of the utterance section one turn before and the fundamental frequency of the utterance section of the current turn, and this fluctuation pattern is used as an input for tuning control. Although the tuning control may be performed based on the state equation of Expression (1), the tuning control may be performed by a state equation that performs linear prediction. Specifically, when the current turn is t and the previous turn is (t−1), the fundamental frequency of the utterance section of the robot system 10 of the current turn is the user of the (t−1) turn. The calculation may be performed by applying state control so that the same variation pattern is obtained from the fundamental frequency of the utterance section of P and the fundamental frequency of the utterance section of the robot system 10 of (t-1) turn.

  Similar to the interaction rule, the communication tuning control unit 13 evaluates the parameters of the state equation based on the duration of the synchronous expression or the amount of speech, and stores the learning result in the state equation parameter learning history unit 20. Then, with reference to the learning result, the parameter is set to a parameter that synchronizes and develops the earliest in accordance with the state quantity gap, and that the duration of the synchronized expression becomes longer.

  As described above, according to the dialog system of the present embodiment, the dialog system can be controlled so as to solve the communication gap in real time and perform continuous and natural communication.

(Embodiment 2)
Continuous pull-in control in which the user's state quantity is brought close to the dialog system state quantity, or the user's state quantity is brought close to the dialog system state quantity while the dialog system state quantity is brought close to the user's state quantity. If the user feels stress in the process of discrete pull-in control, continuous communication cannot be established. Stress can be measured by calculating an autonomic nerve index such as an exchange nerve index or an accessory nerve index from a change in an RR interval of an electrocardiogram, but a sense of resistance to wearing a sensor cannot be denied. .

  In the field of psychology, questionnaire items related to personality and social skills have been established, and the strategy for the dialogue system to approach the state quantity of the user can be determined based on the difference in personality and social skills. For example, if there is a tendency to depend on personality, even if the user's state quantity is close to the state quantity of the dialog system, no stress will be felt as a result. Rather than approaching the system state quantity, there is a tendency that stress is not felt when it is brought closer to the user state quantity and then drawn into the dialog system state quantity.

  These user personality information may be acquired in advance, but the daily personality of the user may change depending on the situation. In such a case, it is considered more effective that the interactive system acquires new personality information in the process of collecting information through conversation with the user.

  In the second embodiment, a strategy for the robot system 10 to approach the user's state quantity is determined in accordance with user personality information that is determined in advance from personality and social skills, and the user's stress during communication is within an allowable range. The dialogue control method is adopted that allows early synchronization in the communication between the user and the robot system 10 while suppressing the noise.

  FIG. 7 is a block diagram illustrating a configuration example of the robot system according to the second embodiment of the present invention. The robot system 10 of the second embodiment includes a user personality information database 21 in addition to the configuration of the first embodiment. The user personality information database 21 stores user personality information composed of the personality of the user P and / or social skills.

  The communication tuning control unit 13 acquires user personality information of the user P having a conversation from the user personality information database 21, and sets an interaction rule that causes the robot system 10 to approach the state quantity of the user P based on the user personality information. Determine for each P. By suppressing the stress of the user P at the time of communication within an allowable range, the synchronization in the communication between the user P and the robot system 10 can be expressed early in a state where the stress is reduced.

(Embodiment 3)
In this third embodiment, the goal is an information gathering robot for home-based interviews, etc., and the robot actively sympathizes with it, and by putting elements that act on episodes into the conversation structure model of information gathering, it is natural and sustainable. Realization of information collection. In the third embodiment, a transition model of three communication modes of a listening mode, a response / sympathy mode, and an episode development mode is assumed. For each mode, the natural communication between the user and the robot is maximized by adaptively changing the parameters of the state equation representing the tuning model and the interaction rules.

  The communication mode is how the speaker relates to the utterance including the listening mode, the response / sympathy mode, and the episode development mode. The conversation structure (model) is the order (model) in which the communication mode transitions.

  The state quantity of the change latency of the speaker change is tuned in common throughout the conversation. However, the state quantity of prosodic information is not subject to tuning control over the entire conversation, and it is necessary to selectively perform tuning control within the conversation structure.

  FIG. 8 is a block diagram illustrating a configuration example of the robot system according to the third embodiment of the present invention. The robot system 10 according to the third embodiment includes a conversation structure model description unit 22 and a conversation strategy description unit 23 in addition to the configuration of the first embodiment. The conversation structure model description unit 22 stores a state transition model of the conversation structure. The conversation strategy description unit 23 stores data that associates the purpose and state of the conversation with the conversation structure model. The purpose of the conversation includes, for example, an inquiry, information provision, encouragement, and change of mood. For example, in the case of an inquiry, the state of conversation refers to the elapsed time from the previous conversation, the duration of the previous conversation, the amount of speech, and the like for the first time, the second and subsequent times, and the second and subsequent times. The conversation strategy description unit 23 defines a conversation structure model to be adopted for each conversation purpose and state classification.

  FIG. 9 shows a conversation structure model of an information collection task such as an inquiry. In the inquiry, for example, the mode is changed from the listening mode to the response / sympathy mode, and information is collected through the episode development mode. FIG. 10 is a schematic diagram of the causal relationship leading to improvement of the degree of interest of the user and expression of motivation by the robot collecting information → sympathy → recalling the topic. As the conversation structure that the robot system 10 works on, the state transition of FIG. 9 is considered to be sufficient.

  It is assumed that prosody tuning is applied in an effective conversation state in such a conversation structure. When making a request such as a listening mode, prosody tuning does not work effectively because the responsiveness required for tuning is sparse. In contrast, in the response / sympathy mode, conflicts and responsiveness frequently occur. Therefore, prosody tuning is expected to work effectively. As described above, in the transition based on the conversation structure model, the tuning of prosody is selectively performed as shown in FIG. Interaction rules also need to be selectively applied during state transitions based on the conversation structure model.

FIG. 11 shows that the discrete pull-in control by the fundamental frequency (F ₀ ) increase rule of the phrase end mora is applied in the listening mode, and the continuous pull-in control by the state equation of prosodic tuning is performed in the response / sympathy mode. ing. In each mode, discrete pull-in control based on common interaction rules such as a phrase end mora long sound addition rule and a whispering generation rule in response to the conflict is applied. In addition, it is shown that the continuous pull-in by the state equation of the tuning at the alternation latency is applied in common to each mode.

FIG. 12A shows an interaction rule for increasing the fundamental frequency (F ₀ ) of the phrase end mora. This rule is suitable for use in the listening mode, and has the effect of increasing the prosodic level of the subsequent user's utterance.

  There are interaction rules suitable for such selective application, and rules suitable for common application. FIG. 12B shows the effect of applying the long sound addition rule that extends the phrase end mora. This rule has the effect of extending “between” (alternative latency), and is applied when accelerating the synchronization of the utterance timing of the robot system 10 when the utterance timing of the user is late. In addition, a whispering generation rule that responds to the interaction is an interaction rule that works in common regardless of the conversation structure model. Which timing is applied at what timing is determined by a learning technique such as a decision tree based on the rule application history in the conversation structure model. For example, a decision tree as shown in FIG.

  As described above, the pull-in control method of the third embodiment has a two-layer structure of continuous pull-in control based on the state equation and discrete pull-in control based on the interaction rule, and is common or selective in the conversation structure model. Thus, it is possible to maximize natural communication between the user and the robot system in the information collection task.

(Embodiment 4)
Continuous pull-in control and discrete operation to bring the user's state quantity close to the dialog system state quantity, or to bring the user's state quantity close to the dialog system state quantity while bringing the dialog system state quantity close to the user's state quantity In the process of automatic pull-in control, by combining the first embodiment, the second embodiment, and the third embodiment, the communication between the user and the dialogue system is expressed early, and the natural communication between the user and the dialogue system is performed. It is possible to realize a dialogue control method of a dialogue system characterized by maximizing the value of the dialogue system.

  FIG. 14 is a block diagram showing a configuration example of a robot system according to Embodiment 4 of the present invention. The robot system 10 according to the fourth embodiment has a configuration in which the configurations of the second and third embodiments are combined. That is, the user personality information database 21, the conversation structure model description unit 22, and the conversation strategy description unit 23 are added to the configuration of the first embodiment.

  Specifically, in the fourth embodiment, the strategy in which the robot system 10 approaches the state quantity of the user P in the communication tuning control unit 13 in accordance with the contents of the user personality information database 21 composed of personality and social skills determined in advance. To decide. Accordingly, the current conversation state is compared with the conversation structure model to determine which conversation state, and the state equation parameters and interaction rules that represent the tuning model are applied according to the characteristics of the listening mode, response / sympathy mode, and episode development mode. Change. In this way, synchronization in communication between the user P and the robot system 10 can be expressed early, and natural communication between the user P and the robot system 10 can be maximized.

(Modification of the embodiment)
The dialog control method of the above embodiment assumes a dialog system versus one user. The robot system 10 according to the first to fourth embodiments can be easily extended to a robot system versus a plurality of users. For example, an average state quantity of a plurality of users is defined as a representative state quantity of a user group, or a state quantity of a worst case user (for example, a user with the longest alternation latency) is represented by a user group. It can be defined as a state quantity. By defining state quantities that represent multiple users, the group user's representative state quantities are brought close to the robot system state quantities, or the robot system state quantities are representative of group user state quantities. It is possible to extend the process to a continuous pull-in control and a discrete pull-in control in which the representative state quantity of the group user is brought close to the state quantity of the robot system.

  FIG. 15 is a block diagram illustrating a physical configuration example of the robot system according to the embodiment of the present invention.

  As shown in FIG. 15, the robot system 10 includes a control unit 31, a main storage unit 32, an external storage unit 33, an operation unit 34, a display unit 35, an input / output unit 36, and a transmission / reception unit 37. The main storage unit 32, the external storage unit 33, the operation unit 34, the display unit 35, the input / output unit 36 and the transmission / reception unit 37 are all connected to the control unit 31 through the internal bus 30.

  The control unit 31 includes a CPU (Central Processing Unit) and the like, and executes processing for voice conversation interaction control according to a control program 39 stored in the external storage unit 33.

  The main storage unit 32 is composed of a RAM (Random-Access Memory) or the like, loads a control program 39 stored in the external storage unit 33, and is used as a work area for the control unit 31.

  The external storage unit 33 includes a non-volatile memory such as a flash memory, a hard disk, a DVD-RAM (Digital Versatile Disc Random-Access Memory), a DVD-RW (Digital Versatile Disc ReWritable), and the above processing is performed by the control unit 31. A control program 39 to be executed is stored in advance, and data stored in the control program 39 is supplied to the control unit 31 in accordance with an instruction from the control unit 31, and the data supplied from the control unit 31 is stored.

  The operation unit 34 includes a pointing device such as a keyboard and a mouse, and an interface device that connects the keyboard and the pointing device to the internal bus 30. User information, state equation parameters, interaction rules, user personality information, conversation structure model, conversation strategy, various determination conditions, and the like are input via the operation unit 34 and supplied to the control unit 31.

  The display unit 35 includes a CRT (Cathode Ray Tube) or LCD (Liquid Crystal Display), a speaker, and the like, and outputs the utterance of the robot system 10. When the robot is represented by a character on the computer screen, the robot character is displayed. In addition, user information, state equation parameters, interaction rules, user personality information, conversation structure model, conversation strategy, and the like are displayed.

  The input / output unit 36 includes a serial interface or a parallel interface. A position sensor, an imaging device, a microphone (all not shown), and the like are connected to the input / output unit 36. Further, when the robot has a substantial face, arms and legs and operates them, the control unit instructs the actuators to control signals via the input / output unit, and the data detected by the sensors. Enter.

  The transmission / reception unit 37 includes a wireless transmitter / receiver, a wireless modem or a network termination device, and a serial interface or a LAN (Local Area Network) interface connected thereto. Data that recognizes the user's utterance is transmitted via the transmission / reception unit 37. User personality information is also collected via the network.

  Sensing unit 11, interaction state quantity calculating unit 12, communication tuning control unit 13, robot interaction control unit 14, interaction rule learning history unit 19, state equation parameter learning history unit 20, user personality information database 21, conversation structure of robot system 10 In the processes such as the model description unit 22 and the conversation strategy description unit 23, the control program 39 is controlled by the control unit 31, the main storage unit 32, the external storage unit 33, the operation unit 34, the display unit 35, the input / output unit 36, and the transmission / reception unit 37. And so on as a resource.

  In addition, the above-described hardware configuration and flowchart are examples, and can be arbitrarily changed and modified.

  The central part that performs control processing including the control unit 31, the main storage unit 32, the external storage unit 33, the operation unit 34, the internal bus 30 and the like is not based on a dedicated system, but using a normal computer system. It is feasible. For example, a computer program for executing the above operation is stored and distributed in a computer-readable recording medium (flexible disk, CD-ROM, DVD-ROM, etc.), and the computer program is installed in the computer. Thus, the robot system 10 that executes the above-described processing may be configured. Alternatively, the robot system 10 may be configured by storing the computer program in a storage device included in a server device on a communication network such as the Internet and downloading it by a normal computer system.

  Further, when the functions of the robot system 10 are realized by sharing an OS (operating system) and an application program, or by cooperation between the OS and the application program, only the application program portion is stored in a recording medium or a storage device. May be.

  It is also possible to superimpose a computer program on a carrier wave and distribute it via a communication network. For example, the computer program may be posted on a bulletin board (BBS: Bulletin Board System) on a communication network, and the computer program may be distributed via the network. The computer program may be started and executed in the same manner as other application programs under the control of the OS, so that the above-described processing may be executed.

  A part or all of the above-described embodiment can be described as in the following supplementary notes, but is not limited thereto.

(Appendix 1)
An interactive system comprising speech recognition means for recognizing the utterance content of a speaker, and response control means for outputting an auditory response by voice and / or a visual response by expression of physical behavior according to the recognition result ,
Detection of state quantities of prosodic features including speaker alternation latency, pitch of speech section, power or mora, and / or detection of state quantities indicating physical behavior of the speaker in the speaker's utterance State quantity detection means to perform;
Means for calculating a communication synchronization deviation amount, which is a deviation amount of an interaction state amount between the speaker and the response control means, from an interaction state amount including the state amount of the prosodic feature or the state amount indicating the physical behavior;
Rule storage means for storing an interaction rule which is a rule for changing an interaction state quantity of the response control means;
Rule selection means for selecting the interaction rule from the rule storage means based on the communication synchronization deviation amount;
The amount of interaction state of the speaker is reduced by discrete pull-in control by the interaction rule selected by the rule selection means, while at the same time minimizing the communication tuning shift amount by continuous pull-in control by a state equation representing a tuning model. Approaching the interaction state quantity of the dialog system, or bringing the interaction state quantity of the dialog system close to the interaction state quantity of the speaker while bringing the interaction state quantity of the speaker close to the interaction state quantity of the dialog system Tuning control means to go,
A dialogue system characterized by comprising:

(Appendix 2)
Means for obtaining user personality information representing the personality and / or social skills of the speaker;
The dialog system according to claim 1, wherein the rule selection unit determines the order of the interaction rules to be selected in accordance with the progress of conversation based on the user personality information.

(Appendix 3)
A structure model storage means for storing a conversation structure model that defines a transition order of communication modes including a conversation listening mode, a response / sympathy mode, and an episode development mode;
Adaptive control means for changing a parameter of a state equation representing the tuning model and / or the interaction rule based on the communication mode transitioning according to the conversation structure model;
The interactive system according to appendix 1 or 2, characterized by comprising:

(Appendix 4)
Dialogue performed by a dialogue system comprising speech recognition means for recognizing the utterance content of a speaker, and response control means for outputting an auditory response by voice and / or a visual response by expression of physical behavior according to the recognition result A control method,
Detection of state quantities of prosodic features including speaker alternation latency, pitch of speech section, power or mora, and / or detection of state quantities indicating physical behavior of the speaker in the speaker's utterance A state quantity detection step to be performed;
Calculating a communication synchronization deviation amount, which is a deviation amount of an interaction state amount between the speaker and the response control means, from an interaction state amount including the state amount of the prosodic feature or the state amount indicating the physical behavior;
A rule selection step of selecting the interaction rule from a rule storage unit that stores an interaction rule that is a rule for changing an interaction state amount of the response control unit based on the communication synchronization deviation amount;
The communication state deviation amount is minimized by continuous pull-in control by a state equation representing a tuning model, and at the same time, the interaction state amount of the speaker is determined by discrete pull-in control by the interaction rule selected in the rule selection step. Approaching the interaction state quantity of the dialog system, or bringing the interaction state quantity of the dialog system close to the interaction state quantity of the speaker while bringing the interaction state quantity of the speaker close to the interaction state quantity of the dialog system The tuning control step
A dialogue control method comprising:

(Appendix 5)
Obtaining user personality information representing the personality and / or social skills of the speaker,
The dialog control method according to appendix 4, wherein the rule selection step determines the order of the interaction rules to be selected in accordance with the progress of the conversation based on the user personality information.

(Appendix 6)
A structure model storage step for setting a conversation structure model that defines a transition order of communication modes including a conversation listening mode, a response / sympathy mode, and an episode development mode;
An adaptive control step of changing a parameter of a state equation representing the tuning model and / or the interaction rule based on the communication mode transitioning according to the conversation structure model;
The dialog control method according to appendix 4 or 5, characterized by comprising:

(Appendix 7)
Controlling a dialogue system comprising voice recognition means for recognizing the utterance content of a speaker, and response control means for outputting an auditory response by voice and / or a visual response by expression of physical behavior according to the recognition result On the computer,
In the speaker's utterance, detection of the state of the prosody feature including the alternation latency of the speaker, the pitch of the utterance section, the power or the mora, and / or the state amount indicating the physical behavior of the speaker is performed. A state quantity detection step;
Calculating a communication synchronization deviation amount, which is a deviation amount of an interaction state amount between the speaker and the response control means, from an interaction state amount including the state amount of the prosodic feature or the state amount indicating the physical behavior;
A rule selection step of selecting the interaction rule from a rule storage unit that stores an interaction rule that is a rule for changing an interaction state amount of the response control unit based on the communication synchronization deviation amount;
The communication state deviation amount is minimized by continuous pull-in control by a state equation representing a tuning model, and at the same time, the interaction state amount of the speaker is determined by discrete pull-in control by the interaction rule selected in the rule selection step. Approaching the interaction state quantity of the dialog system, or bringing the interaction state quantity of the dialog system close to the interaction state quantity of the speaker while bringing the interaction state quantity of the speaker close to the interaction state quantity of the dialog system The tuning control step
A program characterized by having executed.

10 Robot system (dialogue system)
DESCRIPTION OF SYMBOLS 11 Sensing part 12 Interaction state quantity calculation part 13 Communication tuning control part 14 Robot interaction control part 15 Continuous drawing control part 16 State equation parameter storage part 17 Discrete drawing control part 18 Interaction rule storage part 19 Interaction rule learning history part 20 State Equation parameter learning history part 21 User personality information database 22 Conversation structure model description part 23 Conversation strategy description part 30 Internal bus 31 Control part 32 Main storage part 33 External storage part 34 Operation part 35 Display part 36 Input / output part 37 Transmission / reception part 39 Control program

Claims (5)

An interactive system comprising speech recognition means for recognizing the utterance content of a speaker, and response control means for outputting an auditory response by voice and / or a visual response by expression of physical behavior according to the recognition result ,
Detection of state quantities of prosodic features including speaker alternation latency, pitch of speech section, power or mora, and / or detection of state quantities indicating physical behavior of the speaker in the speaker's utterance State quantity detection means to perform;
Means for calculating a communication synchronization deviation amount, which is a deviation amount of an interaction state amount between the speaker and the response control means, from an interaction state amount including the state amount of the prosodic feature or the state amount indicating the physical behavior;
Rule storage means for storing an interaction rule which is a rule for changing an interaction state quantity of the response control means;
Rule selection means for selecting the interaction rule from the rule storage means based on the communication synchronization deviation amount;
The amount of interaction state of the speaker is reduced by discrete pull-in control by the interaction rule selected by the rule selection means, while at the same time minimizing the communication tuning shift amount by continuous pull-in control by a state equation representing a tuning model. Approaching the interaction state quantity of the dialog system, or bringing the interaction state quantity of the dialog system close to the interaction state quantity of the speaker while bringing the interaction state quantity of the speaker close to the interaction state quantity of the dialog system Tuning control means to go,
A dialogue system characterized by comprising: Means for obtaining user personality information representing the personality and / or social skills of the speaker;
The dialog system according to claim 1, wherein the rule selection unit determines an order of the interaction rules to be selected in accordance with the progress of the conversation based on the user personality information. A structure model storage means for storing a conversation structure model that defines a transition order of communication modes including a conversation listening mode, a response / sympathy mode, and an episode development mode;
Adaptive control means for changing a parameter of a state equation representing the tuning model and / or the interaction rule based on the communication mode transitioning according to the conversation structure model;
The dialogue system according to claim 1, further comprising: Dialogue performed by a dialogue system comprising speech recognition means for recognizing the utterance content of a speaker, and response control means for outputting an auditory response by voice and / or a visual response by expression of physical behavior according to the recognition result A control method,
Detection of state quantities of prosodic features including speaker alternation latency, pitch of speech section, power or mora, and / or detection of state quantities indicating physical behavior of the speaker in the speaker's utterance A state quantity detection step to be performed;
Calculating a communication synchronization deviation amount, which is a deviation amount of an interaction state amount between the speaker and the response control means, from an interaction state amount including the state amount of the prosodic feature or the state amount indicating the physical behavior;
A rule selection step of selecting the interaction rule from a rule storage unit that stores an interaction rule that is a rule for changing an interaction state amount of the response control unit based on the communication synchronization deviation amount;
The communication state deviation amount is minimized by continuous pull-in control by a state equation representing a tuning model, and at the same time, the interaction state amount of the speaker is determined by discrete pull-in control by the interaction rule selected in the rule selection step. Approaching the interaction state quantity of the dialog system, or bringing the interaction state quantity of the dialog system close to the interaction state quantity of the speaker while bringing the interaction state quantity of the speaker close to the interaction state quantity of the dialog system The tuning control step
A dialogue control method comprising: Controlling a dialogue system comprising voice recognition means for recognizing the utterance content of a speaker, and response control means for outputting an auditory response by voice and / or a visual response by expression of physical behavior according to the recognition result On the computer,
In the speaker's utterance, detection of the state of the prosody feature including the alternation latency of the speaker, the pitch of the utterance section, the power or the mora, and / or the state amount indicating the physical behavior of the speaker is performed. A state quantity detection step;
Calculating a communication synchronization deviation amount, which is a deviation amount of an interaction state amount between the speaker and the response control means, from an interaction state amount including the state amount of the prosodic feature or the state amount indicating the physical behavior;
A rule selection step of selecting the interaction rule from a rule storage unit that stores an interaction rule that is a rule for changing an interaction state amount of the response control unit based on the communication synchronization deviation amount;
The communication state deviation amount is minimized by continuous pull-in control by a state equation representing a tuning model, and at the same time, the interaction state amount of the speaker is determined by discrete pull-in control by the interaction rule selected in the rule selection step. Approaching the interaction state quantity of the dialog system, or bringing the interaction state quantity of the dialog system close to the interaction state quantity of the speaker while bringing the interaction state quantity of the speaker close to the interaction state quantity of the dialog system The tuning control step
A program characterized by having executed.

Images