JP2005258235A - Interaction controller with interaction correcting function by feeling utterance detection - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an interaction control method in which an interaction currently in process between a human and a machine is corrected into an interaction that a user desires more. <P>SOLUTION: An interaction controller always measures feelings from utterance of a person during the interaction between the human and machine. Once some feeling is detected, the interaction currently in process is corrected to provide a smoother interaction with the machine. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to dialog control that enables a user to control a current dialog to a dialog requested by the user by grasping a psychological state of the current user in the dialog with a machine.

  At present, dialogue between people and machines is performed using speech recognition from speech and linguistic processing, which is processing using words, so dialogue tends to be inorganic. become. Research has been reported with the aim of making conversations richer and smoother. Techniques for discriminating positive or negative from head gestures such as nodding (see Non-Patent Document 1, for example), continuation of speech There is a technique (for example, see Non-Patent Document 2) for determining whether the length is positive or negative. However, the currently reported technology only reflects in the dialogue the judgment that a certain psychological state of the user has been detected or not detected, and reflects the degree of the detected psychological state in the dialogue. Not.

Yasushi Ejiri Yoza Matsuzaka Tetsunori Kobayashi, “Recognition of Head Gestures During Dialogue”, IEICE Technical Report, Vol. 102, no. 218, pp31-36, PRMU2002-61, 2002

Shinya Fujie, Daizo Yagi, Hideaki Kikuchi, Tetsunori Kobayashi, “Spoken Dialogue System Using Paralinguistic Information”, Proceedings of the Acoustical Society of Japan Autumn Meeting, Vol. I, pp39-40, Sep. , 2003

  In a voice dialogue between a human and a machine, the machine creates a response to the human speech, converts the generated words into voice, and presents it to the user. By repeating this several times, a dialogue is performed. Currently, as a technique used for dialogue, a technique is used in which speech is converted from speech into speech using speech recognition and linguistic processing that means words is used. However, with this technology, the dialogue becomes inorganic, which means that humans are giving instructions to the machine. Therefore, we humans cannot feel close to the machine. Therefore, in the present invention, in order to allow humans to feel the machine more closely, the user's psychological state is determined, and the degree of the psychological state is measured, thereby further increasing the interaction between the human and the machine. It is an object of the present invention to provide a dialog control method and apparatus that can be enriched.

  In order to solve the above problems, an outline of a typical invention disclosed in the present application is as follows. The dialogue control device of the present invention is necessary for dialogue with a voice recognition program, a voice synthesis program, an emotion recognition program and a recording unit storing dialogue contents, an input unit for inputting signals such as voice, sound, and images. An output unit that outputs a signal such as a simple sound or an image, and a control unit that controls the output unit. Based on the signal sent to the input unit, the control unit selects one response from a plurality of responses, and outputs a signal of the selection result from the output unit to perform a dialogue. When emotions are detected from signals such as voice, sound and images input from the input unit, the emotion level is detected, and a response different from the currently performed response is selected and transmitted from the output unit. To do.

  In the dialogue between human and machine, in addition to the response selection using linguistic information using voice recognition, in order to grasp the human psychological state, the emotion and the degree of the emotion are analyzed from the human voice. And it becomes possible to consider an emotion and the degree of the emotion when selecting a response. As a result, the dialogue between humans and machines becomes smoother and richer.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a schematic diagram for explaining a system for realizing the dialogue control method of the present invention. Reference numeral 10 denotes a dialogue control device.
The dialogue control apparatus 10 includes a control unit 11 having emotion analysis and data processing functions, a recording unit 12 capable of storing various data such as a hard disk or a memory, a microphone for inputting user's voice, and a user's image. It consists of a camera, keyboard, mouse, touch panel, remote control or connection terminal for external devices and various other data such as an input unit 13, a liquid crystal display, a speaker, etc. The output unit 14 is capable of outputting sound. The recording unit 12 records a main program and the like for executing various processes in the dialog control apparatus 10, and the main program is read by the control unit 11 to execute various processes. The dialogue control device is a device that accepts a user input including a voice utterance by a user, generates and selects a response suitable for the input information, and outputs the response to the user using voice or the like. Control that outputs a response according to input information is called dialog control.

  Next, the drawings used to explain the present invention will be described. FIG. 2 is a diagram showing a dialogue processing configuration and its flow used in the present invention. FIG. 3 is a diagram illustrating a processing configuration and a flow of a sub dialog during a dialog. FIG. 4 is a diagram showing a normal input process and its flow, which is one of the questions from the machine during the sub-dialogue. FIG. 5 is a diagram showing a simple input process, which is one of the questions from the machine during the sub-dialogue, and its flow. FIG. 6 is a diagram showing a flow of dialogue when emotion is taken into consideration in normal / simple input. FIG. 7 is a diagram showing a dialogue control process and its flow when emotion is taken into consideration. FIG. 8 is a database describing the content of stories used in the sub dialog. FIG. 9 is a database in which processing necessary for conducting a dialogue is described. FIG. 10 is a database describing processing necessary for switching between normal and simple input. FIG. 11 is a database describing the current dialog state. FIG. 12 is a diagram showing a process necessary for the exchange between the emotion determination process and the dialogue control system and the flow thereof. FIG. 13, FIG. 14 and FIG. 15 are databases describing the contents spoken by the machine. FIG. 16 is a diagram illustrating a process for determining an emotion and the process. 17 and 18 are databases showing variables necessary for specifying emotions. The database and the program for realizing the processing configuration shown in these figures are recorded in the recording unit 12.

  In the present invention, when performing dialogue control, the control unit 11 analyzes the emotion of the signal input from the input unit 13. When a specific emotion is detected, a response corresponding to the emotion and the degree of the emotion is selected, and the response is sent to the user through the output unit 14. As a result, a smooth dialogue can be realized.

  In the present invention, if a human anger utterance is detected during a dialogue between a human and a machine, it is determined that the current dialogue is not established, and dialogue control for correcting the dialogue is performed. This is a conversation that improves the recognition rate by detecting human angry utterances when the speech recognition used for the conversation is reduced due to environmental noise, etc., and the conversation is no longer established. The purpose is to do. For example, in the case of a station name, the dialogue for improving the recognition rate is to directly obtain the station name, but after the change, the dialogue for obtaining the prefecture name first and then the station name is performed. The former is a method for directly obtaining a keyword, whereas the latter is divided into several times by narrowing down to a method for obtaining a keyword. The latter method aims to increase the recognition rate because words to be recognized can be narrowed down in advance in speech recognition. This makes the dialogue a little complicated, but since the task of obtaining the station name can be achieved, it is possible to avoid that the dialogue is not established.

  A first embodiment using the system configuration shown in FIG. 1 will be described below with reference to FIG. FIG. 2 is a dialogue story flowchart 200 showing a dialogue between a human and a machine. In order to perform dialogue control, the dialogue is basically divided into small sub-dialogues of 1-question 1-answer format. A dialog is composed by combining these sub dialogs. The dialogue story flowchart 200 can be used in various stories, and can be applied to stories such as ticket purchase and weather forecast, for example. Next, the sub dialog refers to one item necessary for purchasing a ticket, such as a destination and a date, in the case of a sub dialog for purchasing a ticket. In addition, the method of proceeding the dialogue may be a form in which a human answers a question from a machine or vice versa.

  Next, FIG. 2 is composed of five sub dialogs, and the number of sub dialogs in the dialog story can be arbitrarily increased or decreased. 201 is a dialog start, 207 is a dialog end, and 202 to 206 are sub-dialogs in a dialog story. Each dialogue is composed of a set of a user's utterance and a computer utterance (which may be speech or non-speech such as a display). The output of the sub dialog (202 to 206) is, for example, continuation and forced termination, and any other output may be used as long as it is necessary for controlling the human-machine dialog, and can be added separately. . Next, continuation and forced termination that are outputs will be described. Continuation indicates that the user has properly responded to the computer utterance (or that the computer has responded properly to the user utterance) and that the sub-interaction has been established Output and go to next sub-dialog. Thus, the continuation output is connected to the next sub-dialog, and the connection of the final sub-dialog is connected to the end 207. The forced termination is an output indicating that the dialogue has not been established, and is terminated even if the dialogue is in progress. Thus, all forced terminations are connected to forced termination 208. In the procedure of FIG. 2, the control unit 11 controls the dialogue according to the main program stored in the recording unit 12 of the dialogue control device 10, and the control unit 11 uses an input unit 13 such as a microphone, a speaker and the like. The output unit 14 is controlled.

  FIG. 3 is a flowchart showing an example of the processing and flow of the sub dialog (202 to 206) shown in FIG. This is referred to as a flowchart 300 of a sub dialog input method. Before explaining the flowchart of FIG. 3, the values necessary for this flowchart will be explained. This is because it is preferable to use the result of measuring the current state of dialogue for the execution of this flowchart. Therefore, in order to manage the current dialog state, for example, a dialog state analysis database 1100 that is a database of measurement results of the current dialog state may be used.

  FIG. 11 shows an example of the dialog state analysis database 1100. FIG. 11 is a database for storing the current dialog status, and describes and stores the result of recognition, the time used, the psychological state of the user, and the like for each item. The dialog state analysis database 1100 is intended to be used to know the current state and to be referred to in another state. In FIG. 11, a state 1101 indicates the current state. The previous state 1102 indicates the previous state. Time 1103 indicates the time when the current state is reached. The question recognition result 1104 stores the recognition result when the recognition is used at the time of the question. The confirmation recognition result 1105 stores a recognition result when recognition is used at the time of confirmation. The latest emotion determination 1106 indicates the latest emotion determination. The latest emotion analysis result 1107 stores emotion determination results. The latest emotion detection time 1108 stores the determined time.

  The latest mode 1109 indicates the current mode. The latest mode change time 1110 indicates the time when the current mode is changed. The latest input method 1111 shows the current input method. The latest input method change time 1112 indicates the time when the current input method is selected. The next state 1113 stores the state to proceed to the next. An item 1114 indicates each item. A value 1115 stores a value for each item. The items stored in the dialog state analysis database 1100 are not limited to these items, and can be deleted and added as necessary. The dialog state analysis database 1100 is preferably stored as data for referring to the past. For example, the dialog state analysis database 1100 is preferably stored in the recording unit 12 for each exchange between a human and a machine.

  Next, the configuration of the flowchart 300 of the sub dialog input method shown in FIG. 3 will be described below. The input branch 302 is a branch in the input method. The sub-dialogue is expressed by, for example, one item necessary for purchasing a ticket, which is a dialogue, such as a destination and date / time when the dialogue is a ticket purchase. The input of the sub dialog is, for example, one item necessary for establishing the dialog, such as a destination. The input method refers to a method for obtaining one item. For example, there are a method for directly obtaining an input and a method for obtaining the input in a stepwise manner. The number of branches in the input branch 302 is preferably the same as the number of input methods to be used. For example, in FIG. 3, the number of branches is set to 2 because it is configured with two input methods of a normal input 303 and a simple input 304. . The branching method is the latest input method 1111 of the dialog state analysis database 1100. If “normal” is described in the latest input method 1111, the process proceeds to the normal input 303, and if “confirmation” is described, the process proceeds to the confirmation input 304. It is also preferable to prioritize these input methods. For example, the flowchart 300 of the sub-dialog input method in FIG. 3 is composed of two types of input methods, and the normal input is performed in the order of priority of the input methods. 303 or simple input 304 may be used.

  The configurations of the normal input 303 and the confirmation input 304 will be described with reference to FIGS. 4 and 5, respectively. The outputs of the normal input 303 and the confirmation input 304 are, for example, restart, input switching, forced termination, and continuation, and any other output may be used as long as it is necessary or necessary for the next dialog. Next, restart, input switching, forced termination and continuation that are these outputs will be described. Restart returns to the beginning of the dialog to restart from the beginning of the dialog. Therefore, the destination connected by the restart is the input branch 302. Input switching is used as the next input method when input cannot be completed with the currently used input method. Therefore, in the case of the normal input 303, the simple input 304 is used as the next input method. In the case of the simple input 304, since there is no next input method, it is forced to end. The forced termination is an output indicating that the dialogue has not been established, and the dialogue is terminated even when the dialogue is in progress. Therefore, this is connected to the forced termination 307. Continuation is an output indicating that a dialogue has been established, and proceeds to the next dialogue. Thus, the continuation of each dialog is connected to the next dialog and is connected to the continuation 306.

  Next, an example of the configuration of the normal input 303 and the simple input 304, which are the respective input methods, will be described with reference to FIGS. 4 is a flowchart 400 for normal input, and FIG. 5 is a flowchart 500 for simple input. The difference between FIG. 4 and FIG. 5 is the difference in the method of obtaining one item. In FIG. 4, a method for directly obtaining an input is used, and in FIG. One item can be obtained by narrowing down several times. For example, it is configured once in FIG. 4 and twice in FIG. First, the normal input shown in FIG. 4 will be described. In FIG. 4, the normal input flowchart 400 is composed of the words A 402 since this is a method of obtaining directly at one time. The output of the word A 402 is, for example, restart, input switching, forced termination and continuation, and any other output may be used as long as it is an output necessary for the next dialog. Also, the output of FIG. 4 is preferably the same as the output of the ordinary input 303 of the flowchart 300 of the sub-dialog input method of FIG.

Next, the simple input shown in FIG. 5 will be described. In FIG. 5, the simple input flowchart 500 is a method of narrowing down and obtaining step-by-step two times, and thus includes a word B 502 and a word C 503. The outputs of the words B 502 and C 503 are, for example, restart, input switching, forced termination and continuation, and any other output may be used as long as it is an output necessary for the next dialogue. Further, the output of FIG. 5 is preferably the same as the output of the simple input 304 of the flowchart 300 of the sub-dialog input method of FIG.
Next, an example of the configuration of the word A 402 in FIG. 4 and the word B 502 and the word C 503 in FIG. 5 will be described with reference to FIG. FIG. 6 is a flow chart 600 in the word Iloha. The word A 402, the word B 502, and the word C 503 are processed by this flowchart.

  Before explaining the flow chart 600 in the word Iroha, it is preferable to first specify what kind of words are used for the words I, B, and C. It is also possible to use the dialogue association database 800 which is designated as B and the word C and is compiled as a database. A word is a content that a machine speaks at a time, and is preferably managed in units of words. FIG. 8 is an example of the dialogue association database 800. Since the utterance contents differ for each dialogue, it is preferable that words suitable for the dialogue to be used can be selected. For example, FIG. 8 is a correspondence table of words a, b, and c in each dialogue. Word 1 (801) in Dialogue 1 (804) is Word 1-1-1, Word B (801) is Word 1-2-1, Word Ha (801) is Word 1-2-2, and Dialog 2 ( 805) is the word 2-1-1, the word b (801) is the word 2-2-1, and the word ha (801) is the word 2-2-2. Note that the data items stored in the dialogue association database 800 are not limited to these, and can be deleted and added as necessary.

  In addition, values described in the dialogue association database 800 of FIG. 8 (such as words 1-1-1) will be described. This is for preliminarily designating information necessary for the dialogue, and the word configuration database 900 that is compiled as a database may be used. FIG. 9 is an example of the word configuration database 900. For example, it is possible to store question contents for dialogue, detailed parameters for use in speech synthesis and speech recognition, and detailed parameters necessary for the machine to act.

  In FIG. 9, the question use number 901 indicates the number of the question content. The dialog continuation confirmation number 902 indicates the contents for indicating the current dialog state when used when the dialog is continued. The input switching confirmation number 903 indicates the contents of the question used when switching the input. A question substitution ($ INPUT) 904 is used when a changeable word such as a place name is put in the question content, and indicates a word to be inserted. If this value is preceded by $, the text included in the item of the dialog state analysis database 1100 is inserted. Question recognition use 905 indicates whether or not to recognize the question. When using question recognition 906 indicates whether speech recognition is activated before or after the utterance from the machine. The question recognition DB name 907 indicates a database name used for recognition. The question recognition result substitution destination 908 indicates a destination to which the recognized result is substituted.

  The confirmation usage number 909 indicates the contents used for confirmation. The confirmation state content confirmation number 910 indicates the current state. Confirmation substitution ($ INPUT) 911 is used when a changeable word is put in a part of the question content, and indicates a word to be inserted. If this value is preceded by $, the text included in the item of the dialog state analysis database 1100 is inserted. Use of confirmation recognition 912 indicates whether or not voice recognition is performed at the time of a question. The use of confirmation recognition 913 indicates whether voice recognition is activated before or after the utterance from the machine. The confirmation recognition DB name 914 indicates a database name used for recognition. The confirmation recognition result substitution destination 915 indicates a destination to which the recognized result is substituted. The data items stored in these word configuration databases 900 are not limited to these, and can be deleted and added as necessary.

  Next, each step of the flowchart 600 in the word Iloha will be described. The content utterance 602 uses speech synthesis and speech recognition for dialogue utterances and questions, if any. In the file describing the content to be uttered, a number is assigned to each word in the dialogue association database 800, and the number varies depending on the dialogue. And based on the selected number, it selects from the dialogue sentence database 1300 using the number described in the question use number 901 of the word composition database 900. The recognized result is described in the question recognition result 1104 of the dialog state analysis database 1100. For example, in the dialogue association database 800, the word 1-1-1 is selected as the word 1 of the dialogue 1. Then, 1-1-1 that is the item 1-1-1 of the question usage number 901 in the word configuration database 900 is selected. Finally, “Please say the station name?”, Which is the 1-1-11th stored word, is selected from the dialogue text database 1300. It is possible to use a technique used for speech synthesis / recognition and various known techniques, and a program for executing the technique is stored in the recording unit 12 in advance. Further, instead of voice recognition, it is possible to perform operations by pressing a key on the keyboard of the input unit 13, pressing a button on the mouse, touching on the touch panel, pressing a button on the remote control, or the like. Furthermore, it is also possible to display using the liquid crystal display of the output unit 14 instead of voice synthesis.

  The dialogue correction 603 detects a person's emotion when speaking, and estimates what the person wants from the state of the emotion. The configuration of emotion correction to dialogue correction 603 will be described with reference to FIG. The output from the emotion detection to the dialogue correction 603 is, for example, restart, input switching, forced termination and continuation, and any other output may be used as long as it is necessary or necessary for the next dialogue. Next, restart, input switching, forced termination and continuation will be described. Restart returns to the beginning of the dialog to restart from the beginning of the dialog. Therefore, the destination connected by the restart is the content utterance 602. Input switching is used as the next input method when input cannot be completed with the currently used input method. Therefore, the input switching is connected to the input switching 609. The forced termination is an output indicating that the dialogue has not been established, and the dialogue is terminated even in the middle. Therefore, the forced terminations 610 and 611 are connected. Continuation is an output indicating that a dialogue has been established, and proceeds to the next dialogue. Thus, the continuation of the dialog is connected to the next dialog, and when continuing to the next dialog, it is connected to the continuation 607. The confirmation branch 604 branches depending on whether or not the content of a person uttered is confirmed. The selection method is the latest mode 1109 of the dialog state analysis database 1100.

  If “confirmation” is described in the latest mode 1109, the process proceeds to the confirmation utterance 605, and if “normal” is described, the process proceeds to continuation 607. The confirmation utterance 605 carries out a dialogue for confirmation using speech synthesis and speech recognition if there is a confirmation item, and an utterance for asking confirmation. The file describing the content to be uttered is selected from the dialogue sentence database 1300 using the number described in the confirmation usage number 909 of the word configuration database 900. The recognized result is described in the confirmation identification result 1105 of the dialog state analysis database 1100. For speech recognition and speech synthesis, the same speech utterance 602 of the flowchart 600 in the word Iloha may be used. Also, the emotion detection to dialogue correction 606 may be the same as the emotion detection to dialogue correction 603. In addition, it is preferable to use the result of measuring the state so far when determining the output in the dialogue correction 606 based on the detection of the content utterance 602 or emotion, for example, the dialogue state in which this is collected as a database An analysis database 1100 may be used. Further, it is preferable to describe the newly determined result in the dialog state analysis database 1100.

  The configuration of the dialogue correction 603 from the detection of emotion will be described. This is a smoother dialogue by correcting the current dialogue state after extracting the emotion. It is preferable to use emotion for correcting the dialogue, and an example of emotion correction from the detection of emotion will be described with reference to the flowchart of FIG. The emotion analysis result 702 determines the emotion of the current speaker's utterance. It is also preferable to determine the degree of emotion. As for the result 702 of emotion analysis, it is preferable to determine the emotion and its degree, which will be described with reference to FIG. A mode check 703 confirms the current mode, and performs a mode conversion 704 if a change is necessary. The mode conversion 704 performs mode conversion based on the result of the mode check 703. The conversion method is the latest mode 1109 and the latest mode change time 1110 of the dialog state analysis database 1100. The current mode is determined from the latest mode 1109 and the latest mode change time 1110. When the mode is converted, the latest mode 1109 and the latest mode change time 1110 are updated. The emotion count 705 is obtained by counting a certain number of emotions within a certain period of time. The measuring method is to measure how many times each emotion is spoken within a certain time. The time check 706 analyzes how many seconds ago it was detected by analyzing when the previously detected emotion was detected. The detection method is obtained from the difference between the latest emotion detection time 1108 in the dialog state analysis database 1100 and the current time.

  A predetermined threshold value is set in advance, and the branch is changed between the threshold value and the threshold value. Based on the emotion count 705 and the time check 706, the input switching confirmation 707 or the dialog state explanation / dialog connection confirmation 708 is branched. The input switching confirmation 707 asks the speaker to determine the input method switching question. At this time, if there is an utterance of dialogue content and a question item, dialogue is performed using speech synthesis and speech recognition. The file describing the content to be uttered is selected from the switching / continuation dialogue text database 1500 using the number described in the question use number 1001 of the input switching confirmation / dialog continuation confirmation database 1000. For speech recognition and speech synthesis, the same speech utterance 602 of the flowchart 600 in the word Iloha may be used. The speaker selects “Yes” if he / she wants to switch inputs, and “No” if he / she does not want to branch. When “Yes” is selected, the connection is made as the input switching 710, and when “No” is selected, the connection is forcibly terminated 711. Further, it is necessary to change the latest input method 1111 and the latest input method change time 1112 in the dialog state analysis database 1100. When “Yes” is selected, the latest input method 1111 is “simple”, and the latest input method change 1112 is updated.

  The dialog state explanation / dialog connection confirmation 708 describes what kind of dialog is currently being performed and determines whether or not to continue this dialog. At this time, if there is an utterance of dialogue content and a question item, dialogue is performed using speech synthesis and speech recognition. A file describing the content to be uttered is selected from the current state sentence database 1400 and the switching / continuation dialogue sentence database 1500 using the number described in the question use number 1001 of the input switching confirmation / dialog continuation confirmation database 1000. For speech recognition and speech synthesis, the same speech utterance 602 of the flowchart 600 in the word Iloha may be used. The speaker utters “Yes” to continue the dialogue, or “No” to redo this dialogue from the start. If “Yes” is selected, the connection is repeatedly made to 712, and if “No” is selected, the connection is made to the restart 713. Finally, when analyzing the dialogue correction 700 from the detection of emotion, it is preferable to use the result of measuring the state so far, for example, using the dialogue state analysis database 1100 which is a database of the results. Also good. Moreover, it is preferable to describe the newly determined result in the dialog state analysis database 1100 as needed for each item.

  FIG. 10 is used to manage the contents of the input switching confirmation 707 and the dialog continuation confirmation 708 in FIG. 7, and the input switching confirmation / dialog continuation confirmation database 1000 that is compiled as a database may be used. FIG. 10 is an example of the input switching confirmation / dialog continuation confirmation database 1000. This is preferably managed in units of words. For example, the contents of questions for dialogue, detailed parameters used for speech recognition, and detailed parameters necessary for the machine to act are stored in units of words. Can do. A word is what the machine speaks at once. The question usage number 1001 indicates the actual question content. Use question recognition 1002 indicates whether or not to use recognition in the case of a question. When using question recognition, 1003 indicates whether speech recognition is activated before or after the utterance from the machine. The question recognition DB name 1004 indicates a database name used for recognition. An analysis result substitution destination 1005 indicates a destination to which the recognized result is substituted. The dialogue contents are described in the respective items of the input switching confirmation 1006 and the dialogue continuation confirmation 1007. The data items stored in the input switching confirmation / dialog continuation confirmation database 1000 are not limited to these, but are preferably deleted and added as necessary.

  When using the result 702 of emotion analysis in FIG. 7, it is preferable to measure emotion analysis. For example, an emotion analysis flowchart which is a method of performing analysis from a person's utterance may be used, and FIG. 12 is an example. This is an emotion analysis flowchart 1200 using voice. Hereinafter, the emotion analysis flowchart 1200 will be described. Emotion determination 1202 identifies an emotion from the voice. It is possible to use a technique used for speech emotion recognition and various known techniques, and a program for executing the technique is stored in the recording unit 12 in advance. The degree of emotion can be obtained by examining the number of times of emotion detection per unit time, for example, how many times an angry utterance is detected within 30 seconds. The greater the number of emotions, the higher the emotion level. This is an example of how to determine the emotion level, and the emotion determination flowchart 1600 can also determine the emotion level, and this may be used. Also, instead of voice emotion recognition, using voice recognition, if a word that may contain a certain emotion such as “No” is recognized, it is determined that a certain emotion has been detected. You can also

  Furthermore, it is possible to perform operations by pressing a key on the keyboard of the input unit 13, emotion determination in image recognition by a camera, pressing a button on a mouse, presence or absence of contact on a touch panel, pressing a button on a remote control, or the like. In the determination result update 1203, the result of the emotion determination 1202 is assigned to the latest emotion determination 1106 of the dialog state analysis database 1100. Further, it is preferable to change the dialogue based on the emotion determination result. As a changing method, it is preferable to use the result of determining the emotional state so far, and for example, a dialog state analysis database 1100 that summarizes the dialog state as a database may be used. Then, the changed result may be described in the item of the dialog state analysis database 1100. For example, when a certain emotion is detected, the latest mode 1109 of the dialog state analysis database 1100 is changed from “normal” to “confirmation” for a certain period of time, and the latest input method of the dialog state analysis database 1100 is determined depending on the degree of a certain emotion. It is preferable to change 1111 from “normal” to “easy” for a certain period of time and to describe the changed time.

  The question use number 901 of the word composition database 900 is numbered 1-1-1, which is the number of the word used in the current dialogue, and the database in which the word is actually stored. Is preferably used, and for example, an interactive text database 1300 that is a database of these may be used. FIG. 13 shows an example of the dialog text database 1300.

  The dialogue text database 1300 describes the content that the machine utters during the dialogue. NO. (1007) is a number, and the contents of the number are described in the word (1307). Numbers 1301 to 1306 are paired with numbers, and in the case of 1-1-1, the words are “Please say the station name?”. In addition, it is preferable that the part indicated by $ INPUT in the word can include any word in the part of $ INPUT. For example, by using the question substitution ($ INPUT) 904 in the word composition database 900, $ INPUT can be changed to any word. In addition, it is preferable to use a database in which words necessary for confirming the current state as well as the content of the machine spoken at the time of dialogue are used. For example, the current state is a summary of this as a database A text database 1400 may be used. FIG. 14 shows an example of the current state sentence database 1400. In addition, it is preferable to use a database in which words are used for dialogues at the time of the input switching confirmation 707 and the dialogue continuation confirmation 708 in FIG. 7, and the words are actually stored. A switching / continuation dialogue text database 1500 may be used. FIG. 15 shows an example of the switching / continuation dialogue text database 1500. The database structure of the current state text database 1400 and the switching / continuation dialog text database 1500 is preferably the same as that of the dialog text database 1300.

  In the emotion determination flowchart 1200, it is preferable to specify an emotion from the voice, for example, FIG. 16 is an example, and is an emotion determination flowchart 1600 that is a process of recognizing a person's psychological state. This is because the user's crisis is observed, and if it is judged that a voice that does not occur in daily life, such as the yelling, screaming, or screaming of the user and the perpetrator, etc., is generated. It is a process to determine that it is a crisis.

  FIG. 16 is a flowchart showing a typical configuration example of the emotion determination flowchart 1600. The emotion determination flowchart 1600 includes a voice feature extraction process 1601, an emotion identification process 1602, an emotion database 1604, and a determination process 1605. The voice feature extraction process 1601 extracts the feature amount from the input voice. As the feature amount, in the case of a pitch representing the pitch of the voice, the average value of the pitch, the maximum value of the pitch, the position where the maximum value of the pitch is detected, and the minimum value of the pitch in one utterance containing emotion The detection of the position where the minimum value of the pitch is detected is taken as the feature amount. In the case of energy representing the loudness of the voice, the average value of energy, the maximum value of energy, the position where the maximum value of energy is detected, the minimum value of energy, the energy Detection of the position where the minimum value is detected is used as a feature amount. This is an example, and the pitch, power, tempo, and the like can be obtained from one utterance, and the feature can be obtained from the pitch, power, and tempo. In addition, a feature value may be extracted from a change amount in time such as pitch, power, and tempo from one utterance. Furthermore, as the feature amount, a publicly known emotion feature may be used as the feature amount. Also, instead of voice emotion recognition, using voice recognition, if a word that may contain a certain emotion such as “No” is recognized, it may be determined that the emotion has been detected. it can.

  The voice feature extracted by the voice feature extraction process 1601 is input to the emotion identification process 1602 to recognize the emotion. This emotion identification processing 1602 can be performed using a publicly known method. For example, by using discriminant analysis, learning is performed first, and the coefficient of the discriminant function is obtained from the feature amount of the voice data that is specified in advance. For example, it is preferable to obtain a coefficient of the discriminant function such that the result of the discriminant function is 1 for anger and -1 for calm. Then, using this discriminant function, voice data that is not specified is discriminated. The result is angry for positive numbers and calm for negative numbers.

  In addition, when the degree of emotion is extracted, it can be determined that the degree of anger is high when the positive number is large, and the degree of calm is high when the negative number is large. This method is an example when emotion is specified, and for example, a neural network or multivariate analysis may be used. In addition, when the emotion identification process 1602 is performed, the emotion database 1603 is used. The emotion database 1603 is data obtained as a result of performing the learning process described above in advance. By using this database, learning can be omitted. It is a database in which a plurality of variables for identifying various emotions are stored in emotion identification, and further a database in which a plurality of data is recorded in association with the variables of the data. By comparing the input voice with the emotion database 1603, it is possible to specify the emotion of the currently input voice.

  FIG. 18 shows an example of the emotion database 1603, in which 1801 is a shout, 1802 is a scream, 1803 is an item indicating the contents of 1801 to 1802, and 1804 to 1808 are variables for each item. The items stored in the emotion database 603 are not limited to these items, and can be deleted and added as necessary.

  Further, even if it is not a human voice, even if a sound that causes a crisis such as a sound of firing a handgun or a sound of breaking a glass window is generated, it is determined that the user is in danger and is notified. This process is performed using the same process as the emotion recognition process 1600 of FIG. However, in the emotion recognition process 1600 of FIG. 16, the emotion database 1603 is used to analyze emotions from human speech, but this is replaced with a crisis sound database 1700. The crisis sound database 1700 is a database of dangerous sounds. The crisis sound database 1700 is a database in which a plurality of data is recorded in association with variables of the data. FIG. 17 shows an example of the crisis sound table 1700, in which 1701 is a shooting sound of a handgun, 1702 is a sound of glass breaking, 1703 is an item, and 1704 to 1708 are variables for each item. The items stored in the crisis sound database 1700 are not limited to these items, and can be deleted and added as necessary.

Emotions can be detected not only from anger but also from other emotions. In addition, not only emotion but also, for example, if it can express a person's psychological state such as a degree of tension, a favor level, and a lie determination, this may be used. In the first embodiment, a dialogue having a certain purpose such as a station name search is shown. However, the usage is not limited to this, and the dialogue can be used for a dialogue that enriches communication such as a healing conversation.
As described above, according to the first embodiment, it is possible to perform dialogue control suitable for an emotion by considering the emotion in the dialogue between the human and the machine. As a result, an expressive and smooth dialogue can be realized.

It is a figure which shows the outline | summary of the system for enforcing the dialogue control method of this invention. It is a flowchart which shows the procedure in the dialogue correction method concerning the 1st Embodiment of this invention. It is a flowchart which shows the procedure of the input method of a sub dialog. It is a flowchart which shows the procedure of normal input. It is a flowchart which shows the procedure of simple input. It is a flowchart which shows the procedure in the word Iroha. It is a flowchart which shows the procedure of dialogue correction from the detection of an emotion. It is a figure which shows an example of a dialog correlation database. It is a figure which shows an example of a word structure database. It is a figure which shows an example of an input switching confirmation / dialog continuation confirmation database. It is a figure which shows an example of a dialog state analysis database. It is a flowchart which shows the procedure of emotion analysis. It is a figure which shows an example of a dialog text database. It is a figure which shows an example of a present condition text database. It is a figure which shows an example of a switching / continuation dialogue text database. It is a flowchart which shows the procedure of emotion determination. It is a figure which shows an example of a crisis sound database. It is a figure which shows an example of an emotion database.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Dialogue control apparatus, 11 Control part, 12 Recording part, 13 Input part, 14 Output part.

Claims (5)

An input unit that receives input, a recording unit that records conversation content and emotion determination information, an output unit that outputs the current state, and a control unit;
The control unit
From the input information sent from the input unit, analysis using the emotion determination information recorded in the recording unit to determine the emotion of the input information,
Select the next response sentence using the dialogue content and emotion determination result recorded in the recording unit,
An interactive control device characterized in that output is performed via the output unit.   2. The dialogue control apparatus according to claim 1, wherein the information used for the input unit is a sound.   3. The dialogue control apparatus according to claim 2, wherein the content analyzed by the control unit is a sound feature amount.   2. The dialogue control apparatus according to claim 1, wherein the content analyzed by the control unit detects emotion and the degree of emotion using emotion determination information stored in the recording unit. The response generated by the control unit selects the next response sentence for proceeding with the dialogue from the dialogue content recorded in the recording unit based on the determination of the emotion and the degree of the emotion. The dialogue control apparatus according to claim 1.

Images