JPH0981632A - Information publication device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an information releasing device capable of reducing the mental burdens of a user by realizing natural and smooth interaction for which the feeling of the user is considered. SOLUTION: In this information publication device for inputting the data of plural forms including a text, sound, a picture and a pointing position, extracting the intention and feeling information of the user from the inputted data, preparing a response plan and generating a response to the user, a user feeling recognition part 106 for recognizing the feeling state of the user from the internal state of a response plan preparation part 105, the intention and feeling information of the user and the transition on a time base of interaction condition information including the kind of the prepared response plan is provided and the response plan preparation part 105 selects or changes a response strategy corresponding to the recognized result of the user feeling recognition part 106 and prepares the response plan matched with the response strategy.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an information disclosure apparatus for disclosing data owned by an individual to another person, and more particularly to an information disclosure apparatus for recognizing the intention and emotion of another person who is a user and making an appropriate response. .

[0002]

2. Description of the Related Art In recent years, a human interface, which adopts a plurality of information transmission means possessed by humans, such as natural language interface / multimodal interface, has been actively constructed.

In addition to these types of media, emotional communication between speakers plays a major role as a factor for efficiently promoting dialogue. In fact, in the current situation that there are not a few cases in which dialogues follow parallel lines due to the difference in context recognition and utterance intention interpretation between speakers in the exchange of e-mails, it supports the extraction / recognition / transmission of the psychological situation or emotions of the speaker. It represents the need for a device to do this.

Kanazawa et al. (Proceedings of the Institute of Electronics, Information and Communication Engineers D-II, Vol.
J77-D-II, No.8, pp.1512-1521) and Cahn et al.
rating Expression in Synthesized Speech ", Technic
al Report, Massachusett Institute of Technology, 19
The research of 90) has already been reported.

These focus on the "voice temperament information" such as the pitch and accent of the voice signal, and anger, joy, sadness,
It recognizes, understands, and generates intentional and emotional information such as consent, impression, and connection.

There is also an attempt to recognize an emotion from a character string expressing the emotion in the text. A method of extracting emotion information included in a word string by using an emotion dictionary in which emotion information in units of words is quantified and registered by Fujimoto et al.
82376) or a method having a rule for converting the emotion information based on the result of syntax analysis (Japanese Patent Publication No. 6-8237).
7) etc. are proposed.

However, these are for extracting the emotion contained in one sentence or one utterance, and not for recognizing the emotion of the user through dialogue. In reality, even if the same utterance or linguistic expression is used, the user's emotions often differ depending on the situation. Also, the utterances and linguistic expressions expressing emotions differ depending on the user.

On the other hand, there is also an attempt to recognize the emotions of the characters from the description of the story. WGLehnert et al. ("The Role
of Affect in Narrative Structure ", Cognition and
Emotion, 1987, pp. 299-322) and MGDyer ("Em
otions and their Computations: Three Computer Mod
els ", 1987, Lawrence Erlbaum Associates Limited), which recognizes situations from sentences written in natural language and infers the emotional state of characters. Linguistic expressions and situations are extracted from narrative descriptions, and emotions and situations are not recognized using only the utterances and dialogue contents of characters.

Research has also been conducted on a dialogue system that has a discourse structure model and generates an appropriate response to a user's speech intention. In the text dialogue, Sumita et al. (“Consideration on naturalness of response in question answering system”, IEICE Tech.
NLC86-16, pp.25-32, 1986), Ukita, et al. ("Device operation guidance system by natural language input", IEICE Technical Report OS88-18, pp.
13-18, 1988), in the speech, Araki et al. (“Understanding Speech Using the Structure of Dialogue and the Concept of Words”, IPSJ
42nd National Convention, 3, pp. 61-62, 1991). These are intended to recognize the user's intention according to the situation from the user's utterance or input text and generate an appropriate response. However, it does not recognize the emotion of the user.

Further, in the information disclosure device and the multimodal information input / output system described in Japanese Patent Application No. 7-86266, emotion recognition is determined by using emotion information included in one sentence or one utterance. No information on the status of the dialogue is used.

[0011]

As described above, the conventional information disclosure device has no means for analyzing the emotion of the user based on the situation of the dialogue. For this reason, it becomes difficult to generate a response in consideration of the user's emotions, and the dialogue becomes distorted so that the user's intention cannot be correctly understood, which imposes an unnecessary mental burden on the user.

[0012] The present invention has been made in view of such a situation, and recognizes the emotion of the speaker according to the situation of the dialogue and realizes a natural and smooth dialogue in consideration of the emotion of the user. An object of the present invention is to provide an information disclosure device capable of reducing the mental burden on the user.

[0013]

SUMMARY OF THE INVENTION According to the present invention, input means for inputting a plurality of forms of data including text, voice, image and pointing position, and user's intention and emotion information from the data input by this input means. And a response plan creating unit that creates a response plan based on the extraction result of the extracting unit, and a response creating unit that creates a response to the user based on the created response plan. In the information disclosure device, the internal state of the response plan creating means, the extracted intention and emotion information of the user, and the transition of the conversation situation information including the type of the created response plan on the time axis from the emotion of the user An emotion recognition unit for recognizing a state is provided, and the response plan creation unit responds according to a recognition result of the emotion recognition unit. Select or change the substantially, characterized in that to create a response plan that matches the response strategy.

According to the present invention, it is possible to recognize the emotional state of the user from a plurality of forms of input such as voice, text and image and the situation of the dialogue, and make an appropriate response. That is, according to the present invention, when the user is angry, "apologize", "arbitration", "words for quarrel", etc. It is possible to generate a response such as "introduction" that leads the user to a positive or negative emotion. As a result, in a situation where many variations of responses to the user can be generated, it is possible to determine a response that was conventionally standardized or randomly determined in consideration of the user's emotions, and a more natural response generation can be achieved. realizable.

[0015]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below with reference to the drawings.

(First Embodiment) First, a first embodiment of the present invention will be described.

FIG. 1 is a functional block diagram of the information disclosure device of the first embodiment.

As shown in FIG. 1, the information disclosure device 10 of the embodiment has an input unit 101, a data storage unit 102, a data search management unit 103, a request reception unit 104, a response plan creation unit 105, and user emotion recognition. It has a unit 106 and a response generation unit 107.

The input unit 101 inputs user data such as text, images and voice. The data storage unit 102 stores a list of users who can access the information disclosure device 10, a relationship between each user and an information provider, a user response rule (text, voice, etc.), a user history, and the like. The data search management unit 103 extracts user information from the data stored in the data storage unit 102.

The request accepting unit 104 accepts user data such as text, images and sounds input by the input unit 101 via a communication means such as a network and extracts the intention of the user. The user emotion recognition unit 106 extracts the user's emotion from the intention extracted by the request reception unit 104 and the data such as the text, the image, and the sound received by the request reception unit 104.

The response plan creating section 105 is the request receiving section 1
A response plan is created from the intention extracted in 04 and the user emotion extracted by the user emotion recognition unit 106. The response generation unit 107 generates and outputs a response from the response plan created by the response plan creation unit 105.

The operation of the embodiment will be described below. Here, a person who discloses and provides information is called an information provider, and a person who accesses the information disclosure device to obtain information is called a user. The information disclosure device is called an agent, the purpose of the user accessing the agent is a dialogue purpose, and a word expressing an emotion is called an emotion word.

The operation procedure of the information disclosure apparatus of the embodiment will be described with reference to FIG.

The user accesses the agent in textual form, for example by means of a window-based interface. Here, the agent confirms whether or not the user is registered for this access, and if not registered, refuses the dialogue (step 201 in FIG. 2).

At step 202, the agent retrieves user information. That is, the personality information database is searched for the personality of the user, the social relationship with the information provider, the trust relationship, and the like. Then, in step 203, the utterance intention of the user is extracted, and the emotional word included in the text sentence or the voice is extracted by morphological analysis.
At 4, the user's emotion is estimated from the emotion word used for input. Since it is difficult to estimate the emotion of the user only by the emotional words included in the text sentence or the voice sentence, in the same embodiment, for example, emotion extraction from the emotional words and contextual emotion extraction along with the dialogue development are performed. Both sides should recognize the user's emotions.

For example, emotional words are registered in advance in a dictionary together with emotion types, likelihoods and impacts of the words. Then, using the dialogue count emotion model (see FIG. 3) in which the emotion is analogized from the dialogue count held in the table format in the user emotion recognition unit 106, (expectation) / (anxiety) /
(Margin) / (frustration) / (thanks) / (convinced) / (give up) /
Estimate the emotion generated from the dialogue such as (anger) and confirm whether the meaning of the emotion word matches the emotion at the dialogue stage.
In this model, emotions are set according to the number of dialogues, but the number of dialogues and the designation of emotions may depend on the user model. For example, the degree of impatience in the user model is expressed in five stages, and when the impatience degree is 5, it is set by subtracting 1 from the standard number of each emotion.

In this method, for example, when the emotional word is a highly ambiguous word, the emotion corresponding to the dialogue stage is added as shown in FIG. The emotion of the dialogue count model is (anxiety)
If / (expectation) includes [Ane; expectation (5,1), frustration (5,8);], there is a correction such as increasing the likelihood of (expectation). Alternatively, when an emotion that does not suit the dialogue stage comes in, the likelihood of emotion according to the dialogue count emotion model is increased in order to reflect satisfaction / dissatisfaction as a tendency.
For example, if the dialogue count emotion model indicates an emotion of (frustration), and if the text sentence includes “idiot; anger (9,8);”, the likelihood of anxiety is increased to increase the likelihood of anger. Lower. After such correction, the emotion is identified by the value of the likelihood × impact. For example, if W1 = 40, W2 = 60, etc., and the likelihood × impact is between W1 and W2, it corresponds to (the number of dialogues + 1). The number of conversations is the emotion, and if W2 or more, the conversation is equivalent to (the number of conversations + 2).

Alternatively, the distribution of emotions is [margin-frustrated],
It is arranged in a three-dimensional space consisting of [satisfaction-dissatisfaction] and [acceptance-rejection], and emotional words are expressed by the coordinates of this three-dimensional space and registered in a dictionary together with the values of each axis.

For example, after searching for each degree of emotional words [margin-frustration], [expectation-dissatisfaction] and [acceptance-rejection], the degree of each axis is corrected by a combination of utterance intentions of the agent and the user. To do.

For example, if an emotional word is [Woooo (margin:-
4, expectation: -3, acceptance: -2)]
When the agent apologizes and the user requests the utterance intention, the emotion word margin and acceptance are set to -2.
Alternatively, when the utterance intention is affirmative and an emotional word having a low acceptance is introduced, the acceptance is set to +2 or the like.
Then, the corrected value is applied to each condition to specify the emotion. This correction condition is held as a table in the user emotion recognition unit 106.

An example of the correction conditions is shown in FIG. From the correction result, the emotion expressed by the emotion word is specified under the condition as shown in FIG. 6, for example.

For example, in the dialogue shown in FIG. 7, when the emotional word "Usoo (margin: -4, satisfaction: -3, acceptance: -2)" appears, the combination of the utterance intentions causes "Usso (Margin: -6, satisfaction: -4, acceptance: -3)]). From FIG. 6, the emotion expressed by the emotion word is estimated to be anger.

In step 205 of FIG. 2, step 20
The emotional word result in 4 is received, and the emotion of the user in the dialogue is analyzed.

Further, the user's dialogue purpose can be classified into “request”, “confirmation”, and “command” based on the utterance intention. Then, it is considered that the emotional transition can be set according to the progress of the achievement of the purpose. Here, this is called an emotion transition model.

For example, in the case of [request], the user's emotion at the access stage has a high (expectation). Ideally, the user who finally obtained the request ends the conversation with feelings of (satisfaction) and (gratitude). But,
It seems that feelings such as (anxiety) and (distrust) develop into the user while the agent repeats the supplementary question n times due to lack of information. Also, the intent is not recognized,
Alternatively, the user's emotions are controlled by (disbelief), (frustration), and (disappointment) after m times such that the result presented by the agent is misguided.
Furthermore, if the intended result cannot be obtained even after the total number of dialogues is k, (confused) and (anger) become intense. If the emotion of the user at the end of the dialogue reaches (confused) or (anger), the agent has lost credibility. To avoid this situation, the agent adds a response generation plan such as clarification / apology. An example is shown in FIG.

In the case of [confirmation], the emotion (expectation) of the user at the access stage is considered to be smaller than that at the time of [request]. In addition, the emotion of the user who finished the confirmation is (convinced),
It is desirable to end the dialogue with feelings of (satisfaction) and (gratitude). Similar to [Request], if the intention communication is delayed, a feeling of (frustration) occurs, but it is considered that it eventually shifts to a self-converging feeling such as (giving up). An example is shown in FIG.

In the case of [command], the feeling of the user at the access stage is much higher than that at the time of [request]. Further, the user feeling at the end of the dialogue is preferably (satisfaction) or (trust). Also, if it takes time to achieve the purpose (frustration),
It can be expected that emotions will fluctuate significantly if it is not achieved (anger). To avoid this situation, agents try to reduce emotional swings by adding appropriate response generation plans such as clarifications / apologies. An example is shown in FIG.

The emotion transition model according to the conversation purpose as described above is held as a table in the user emotion recognition unit 106. Numerical values in this table are determined by the personality (temperedness) of the user registered in the user model.
For example, the degree of impatience of the user model is expressed in five levels,
When the temperament is 5, the standard value of each emotion is subtracted by 1 and set.

As an example, the intention acquisition stage as shown in FIG.
Using a discourse transition model consisting of three stages of the lack information acquisition stage and the answer presentation stage, the rules of the emotion transition model whose dialogue purpose is [request] are shown. Here, the feeling at the time of access is (expectation).

Confirm the intention to the user at the intention acquisition stage, and
Success is obtained when the response from ES is obtained, and failures other than YES are regarded as failures.
Then, the number of successes + the number of failures = F1. When F1 = -2, move to (anxiety), set -4 (frustrated), and set -6 (confused).

Further, the number of successes at the stage of acquiring the insufficient information + the number of failures = F2. When F2 is -2 (frustrated), -4 is (confused), and -6 is (angry).

Then, the number of successes in the answer presentation stage + the number of failures = F3. F3 = -1 (disappointed), -2 (confused), -3 (angry), etc.

If, in the intention acquisition stage, the process goes to the impatience stage and reaches the insufficient information acquisition stage (F1 = 4), F2-2
The transition starts from (frustrated). Further, if the total number of failures in the intention acquisition stage and the insufficient information acquisition stage is -3 or more, it starts from (disappointment) in the answer presentation stage, and from -5 (confusion) in the answer presentation stage.

When the dialogue sentence contains an emotional word, the emotional word analysis result shown in FIG. 11 is reflected in the emotional transition.

For example, as shown in FIG. 12, (anxiety)
= -1, (expectation) = + 1, (frustration) =-2, (margin) =
+2, (confused) = -3, (convinced) = + 3, (anger) =-
An emotion correspondence table such as 4, (gratitude) = + 4 is provided in the emotion transition plan storage unit.

Now, let s be the absolute value of (transition model numerical value)-(emotional word numerical value), and F be the number of failures + successes at each stage of the discourse transition model. Then, using this table, the emotions of the emotion transition model and the emotions of the emotional words are represented by numerical values. If (transition model numerical value) × (emotional word numerical value) <0, it is determined that the expression of the emotional word is different from the context, and the expression of the emotional word is ignored. Alternatively, the emotion of the user may be confirmed here.

If (transition model value) × (emotion word value)> = 0, the emotion word and the context match, and the absolute value of the value of (transition model value)-(emotion word value). If s == 2 and (transition model value)> 0, then F + 1,
If (transition model value) <0, F-1 is set. Alternatively, if S == 3 and (transition model numerical value)> 0, then F + 2,
If (transition model value) <0, F-2 is set.

As a result, the emotion to which F belongs is specified as the user emotion.

Further, it is possible that the same emotional word has different emotional types and meanings depending on the voice or text sentence. Especially, the voice expresses the emotion of the user in real time. Therefore, the emotional words included in the voice can be processed as follows.

For example, the research by Kanazawa et al. (Proceedings of the Institute of Electronics, Information and Communication Engineers D-II, Vol.J77-D-II, No.8, pp.1512-1521) is based on non-language such as pitch and accent of speech signal. The present invention relates to a method of recognizing and understanding intentional and emotional information such as anger, joy, sadness, consent, impression, and connection by paying attention to "sound-rhythm information" that is information.

Of the emotional words having non-verbal information such as [aa] and [er] issued by the user during the dialogue with the agent, the emotional words that are frequently used are [margin-frustrated] and [satisfied-dissatisfied]. And each degree of [acceptance-rejection] is registered to create a dictionary. Speech recognition of emotional words that have linguistic information such as [what it] and [don't know] is described in Takebayashi et al. (Journal of the Institute of Electronics, Information Processing, D-11, Vol. J77-
D-11, No8, pp1417-1428, etc.). The method of performing emotion recognition using the results of both linguistic information voice recognition and non-verbal information voice recognition for one utterance is detailed in the above-mentioned Japanese Patent Application No. 7-86266, and will be omitted.

Information such as vocabulary of emotional words and types of emotions is
You may use together with the content of the emotional word dictionary of a text sentence. Further, an emotion word definition dictionary that defines the emotion of the user by combining the language information voice recognition and the non-language information voice recognition may be used.

Further, when the emotions of the user in one dialogue pair are analyzed, the emotions that occur reflexively to the display contents of the agent occur during the response sentence display, and the emotions that occur with respect to the user's own operating procedure during inputting. In many cases, they give their impressions of the whole dialogue. Therefore, the progress of one dialogue pair is classified into the inputting of a text sentence by the user waiting for the agent's response and the displaying of the agent's response sentence, and at which timing the user uttered which emotional word. The user's emotion in one dialogue pair is estimated by the analysis.

As an analysis method, for example, the degrees of [margin-frustration], [satisfaction-dissatisfaction], and [acceptance-rejection] of emotional words issued during input and display are summed up to calculate an average value. To do. The emotions as shown in FIG. 13 are set from combinations of the values of the degrees obtained by subtracting the average value of the emotional words being displayed from the average value of the emotional words being displayed, and furthermore, [expectation: +1, anxiety: -1, Margin: +2, Consent: +
2, Frustration: -2, Puzzle: -2, Thanks: +3, Anger:-
3] etc. are given. This makes it possible to estimate and quantify the emotion of the user in the dialogue pair.

In addition, paying attention to the above-mentioned utterance timing, among [margin-frustration], [satisfaction-dissatisfaction] and [acceptance-rejection], [satisfaction-dissatisfaction] and [acceptance-rejection] are displayed.
There is a method of totaling the values of ",""frustration" and "satisfaction" while inputting, and dividing the total value of each degree by the number of total words to obtain an average value. To estimate the user's feelings in a dialogue pair.

Further, the voice emotion and the emotion of the text sentence are matched.

For example, the emotion F estimated from the text sentence
Also [Expectation: +1, Anxiety: -1, Consent: +2, Frustration:-
2, confused: -2, thankfulness: +3, anger: -3], and digitized in the same manner as voice emotion, and E = (text sentence emotion (n) -text sentence emotion (n-
1)) + (Voice Emotion (n) -Voice Emotion (n-1)) n = Current Number of Dialogues The flow of emotions in the progress of the dialogue is grasped from the calculation formula. For example, emotions are set as shown in FIG. 14, and E is set as the degree of satisfaction. When the number of conversations is the fifth and the user's emotions are uneasy up to that point, it is estimated that E <0 indicates impatience, E == 0 causes confusion, and E> 0 indicates optimism.

Further, it is considered that voice is considered to represent more latent emotion of the user than text sentences, and a method of increasing the priority of voice emotion can be considered. For example, when (the numerical value of the text sentence emotion × the numerical value of the voice emotion) is 0 or less, the text sentence emotion (n) = the text sentence emotion (n) + the voice emotion (n) is calculated, and E is calculated.

With the method described above, it is possible to estimate the emotion of the user according to the situation of the conversation by using the emotion word and the emotion transition model.

(Second Embodiment) Next, a second embodiment of the present invention will be described.

The same embodiment shows a method of selecting and generating an appropriate response using an emotion recognition result in a dialogue device capable of generating a plurality of responses to a user's intention input.

FIG. 15 is a functional block diagram of the information disclosure device of the embodiment.

The input unit 201 receives input of texts, still images, moving images, voices, and the like via a communication means such as a network. Further, it may be directly accepted from an input device such as a keyboard, a mouse, a microphone, a camera, etc., not via a network.

The intention / feeling information extraction unit 203 extracts expressions representing intentions and feelings contained in the input, and converts the result into, for example, a semantic expression to the emotion recognition unit 203 and the response plan creation unit 204. send.

The response plan creating unit 204 makes a plan using knowledge or rules stored in advance in order to create an appropriate response to the user's intention or feeling extracted by the intention / feeling information extracting unit 202. For example, a dialogue transition model representing the state of dialogue is prepared, and a response plan is created according to the extracted intention of the user. In addition, the emotion recognition unit 20
The response plan is changed according to the result of 3 and the contents of the history storage unit.

In the emotion recognition unit 203, the emotion information extracted by the intention emotion information extraction unit 202 and the response plan creation unit 2
04 or the dialogue transition state in the history storage unit, the emotion of the user is recognized.

The response generation unit 205 generates a response according to the response plan determined by the response plan generation unit 204, for example, as data such as text, voice, still image and moving image, or a combination thereof. To do.

FIG. 16 shows a configuration in which a user information storage unit 206 and a history storage unit 207 are added to the configuration shown in FIG.

The user information storage unit 206 is registered with user information about the personality and social role of the user, familiarity with the device, and social or personal interpersonal relationship information between a plurality of users. I am doing it.
The emotion recognition unit 203 and the response plan creation unit 204 change the emotion recognition method and the created response plan for each user according to the content of the user information storage unit 206.

The history storage unit 203 stores the user's intention and emotion recognition result, the response generated by the system, and their semantic expressions in association with the dialogue transition model of the response plan generation unit 204.

FIG. 17 shows an input unit 201, a response generation unit 205, an emotion recognition unit 203, and a response plan creation unit 204.
Are set as different processes (209a, 209b), and data is transferred via the data communication units 208a and 208b.

FIG. 18 shows the input unit 201 and the intention / feeling information extraction unit 202 separated by voice and text (201a to 201b, 202a to 202b).

FIG. 19 shows the operation procedure of the embodiment. Further, FIG. 20 shows an example of the response of the same embodiment.

First, a user who desires a dialog performs an access start operation. For example, in the window-based interface as shown in FIG. 20, the command string “pip ya” is displayed in the window in which the user can enter text.
for example, "mamoto".

In this embodiment, a command pip has a function of interactively disclosing information on behalf of a human being.

Here, it is assumed that access to the information disclosure device is started. For example, as shown in FIG. 20, the image of the information disclosure agent is displayed in the upper left and the image of the user himself is displayed in the lower left, and the dialog between the user and the agent is performed on the text basis in the right window. Details of the operation of the information disclosure device are omitted because they are detailed in Japanese Patent Application No. 7-86266. The present invention is not limited to the information disclosure device as the main function of the dialogue device, but can be applied to an information providing service to an unspecified information requester such as a database service.

After starting access, the system creates the first response plan. In the response plan creating unit 204, for example, the request response is processed according to the dialogue transition model as shown in FIG. A response plan is created in the state indicated by the thick frame ellipse in FIG. 21, and the intention is analyzed in the state indicated by the broken line ellipse.

In the dialogue transition model example of FIG. 21, the dialogue with the user is represented by four stages classified by the depth of information shared by the system and the user. Level 0 represents a level for acquiring the type of user request. Level 1 represents a step of acquiring information held by the user if necessary to fulfill the user's request. Level 2 asks the user for confirmation when it is necessary to confirm with the user the operation performed by the system before execution. At level 3, the operation is actually performed and the result is reported to the user.

In the START state, a response generation plan for greeting is created, and then in the request acquisition state, a response plan for request acquisition is created. In the request acquisition state, a response plan that prompts the user to input a request is created. If no request is input, the process returns to the request acquisition state. When the likelihood of the input request is low, a response plan for confirming the request type in the request confirmation state is created.

If a request having a likelihood higher than a certain level can be obtained, the execution condition of the request is checked without confirmation. When the execution conditions are different for each request, a list (execution condition list) for designating the likelihood and the execution condition for each request type as shown in FIG. 22 is prepared and referred to.

Also, when the priority order for checking the execution condition is changed for each request, or when the transition destination is changed when the execution condition is not satisfied, the same method can be realized as shown in FIG.

For example, if the request is a schedule reference, it is checked whether the schedule search conditions have been acquired. For example, in FIG. 22, the date (dat
If e) is instructed by the user or two or more conditions of item type (act), title, place, week and month are specified, the search is executed. If the search conditions are insufficient, a response plan is created that transitions to the information acquisition state and prompts for input of the missing information before executing the search. The information acquisition state is classified into level 1 in which the content of the conversation is deeper than level 0. When the request is a message to another user, if the message content has not been acquired yet, a response plan for acquiring the message content in the information acquisition state is created.

When the user's request is a direct dialogue (the users directly interact with each other), it is checked whether the other user can be contacted. If not, the execution condition is not satisfied.
Create a response plan such as an apology and return to the request acquisition state.

If the required information has been acquired, or if the request requires pre-approval, the pre-approval state is entered. For example, in the case of a message, a response plan for confirming the message content with the user before recording the message is created.

When all the conditions are satisfied, the state transits to the request execution state. For example, a search for a schedule is performed, and a response plan that presents the result to the user is created. Alternatively, if the request cannot be executed due to a search failure or the like, a response plan is created to inform the user and apologize. This request execution is set to level 3, and after execution, transition to the request acquisition state for acquiring the next request except execution of the dialogue end request, and the level returns to 0.

As described above, in FIG. 21, the transition of the dialogue with the user is classified at the information sharing stage between the user and the system. However, in addition to this, the number of stages can be increased or decreased, or the type of intention of the user or the type of request. There may be a method of classifying each item or not classifying the states. If classification is not performed, the emotion recognition unit 203 and the response generation unit 205 use the state name as it is to describe the process, but the process can be changed minutely. On the contrary, by classifying, there is an advantage that processes such as dialogue history memory, emotion recognition, and response generation can be described for each classification item.
There is also a dislike that the process is too rough by itself. The classification method of FIG. 21 is particularly effective when the failure or success of the system has different psychological effects on the user depending on the information sharing stage.

The response plan creating unit 204 records a history in the history storage unit 207 according to the level information of the dialogue transition model of FIG. 21, for example, as shown in FIG.
Each rectangle in FIG. 23 represents one unit of the dialogue history.
In the example of FIG. 23, one unit of the dialogue history is created each time the response plan creation state of FIG. 21 is passed, so that one or more response plans are always included. If the intention analysis state has passed immediately before reaching the response plan creation state, the intention information input by the user is also included. For example, intention analysis,
At each stage of condition judgment, request execution, etc., a score is given according to the request type, success, failure, transition destination, etc., and the history storage unit 2
07 is stored.

Extraction of the utterance intention of the user is realized by the following procedure, for example.

The user inputs a sentence into the text window shown in FIG. 20, for example, using an input device such as a keyboard. The intention / feeling information extraction unit 202 first extracts the utterance intention of the user as shown in FIG. 24 from the input text sentence. Regarding the request of the utterance intention, for example, a request of the type shown in FIG. 25 is accepted.

First, the user's input is subjected to morphological analysis, divided into words and added with part-of-speech information. When the request is a schedule or a document reference, it is essential to extract words such as a person's name, a place name, a number, and a proper noun.

Next, matching is performed using a keyword dictionary as shown in FIG. 1 in the keyword dictionary
One or more category candidates are described for each keyword. For example, likelihood and strength are specified as the category candidates. In addition to the category name, the attribute name (tai
(l, end, etc.) and items are added.
For example, suppose the user inputs "Do you know the schedule for the next meeting?" According to the dictionary of FIG. 26, “this time” is tense
It can be seen that (meeting) and “meeting” are included in the category of act (item type), and “plan” is included in the schedule. The keyword “meeting” also includes candidates for the category “actail”. When "acttail" follows a noun, it indicates an attribute that represents a conference name such as "XX conference", but in this example sentence, as a result of morphological analysis, since there is no noun immediately before, act is adopted.

The likelihood of these keywords determines the intention of the sentence. In this example, there is no keyword indicating the intention, but since the schedule is included as the request target in FIG. 25, the intention is “request”. The semantic expression of the utterance is expressed in a format as shown in FIG. 27 and sent to the response plan creating unit 204.

The emotion information included in the text is extracted from the user's input sentence by the following method using the emotion word dictionary as shown in the first embodiment, and the emotion recognition unit 203 and the response plan creation unit 204 are extracted. Sent to.

Since the dialogue apparatus of the embodiment is mainly intended to execute the request of the user, the emotion recognition method regarding the request will be described. There are many other attempts to classify general emotions (Yasuyuki Fukui: Psychology of emotions, Kawashima Shoten). For example, there is a study in which three axes such as [comfort-discomfort], [strength], and [direction (directivity)] are associated.

For example, it is assumed that the feelings involved in requesting (ordering) the other party are limited, and [margin-imminent] and [accept-reject] are adopted in addition to [pleasant-uncomfortable]. [Pleasure-discomfort]
Is mainly an emotional axis that represents the user's evaluation of the response content of the system, the user's prediction, and the actual result.
[Margin-urgency] is an axis of emotions that accompanies the difference between the desire achievement achievement and the actual achievement situation mainly determined by the user's own situation or personality such as time constraint. [Acceptance-Rejection]
Is the emotional axis that indicates whether the user accepts the system itself or its response. FIG. 28 shows an outline of the above three axes.

It is assumed that the emotion of the user included in the input sentence has coordinate values in the space composed of the above-mentioned three axes. For example, assuming that the maximum value on each axis is 5 and the minimum value is -5, "a little" in the input sentence 1 shown in FIG. 27 is [margin-imminent], [pleasant-unpleasant] and [accept-reject]. Is defined as having a coordinate value of (-2, -1, -1) on the axis of.
In addition, the “please” is (−2,1,1), the ending “Arudan” is (−1, −1, −1), and the “kana” of the input sentence 3 is (3, −1,1). ), "Bad" of the input sentence 4 is (-2, -2, 0). These values are averaged for each sentence. Then, the emotion expression information of the input sentence 1 is (-2,
-1, -1).

Similarly, the emotion names actually found in the requested task are displayed in the above-mentioned three-axis regions, for example, as shown in FIG.
Assign like. The emotional areas may overlap with each other. For example, as shown in the area of FIG. 29, "give up"
Is uncomfortable, but accepts the results and represents an area of emotion that is generous. "Consent" is a little different from "give up" in that it accepts the result rather than being offensive. The situation is urgent for "frustration." “Expectation” and “anxiety” are
Both of them are a little urgent and have not yet accepted or rejected the situation, but the expected behavior of the system is "expected" if comfortable, and "anxious" if uncomfortable.

Each area can be redefined by a user such as a system administrator. For example, a graphics interface as shown in FIG. 29 may be prepared and a pointing device such as a mouse may be used for the instruction. Further, as shown in FIG. 30, each area may be designated by a table or the like.

The emotional state of the user is defined from the values of the three axes described above, for example, as "acceptance = satisfaction" with respect to the user's system. It has a value of plus (satisfaction) or minus (dissatisfaction). For example, if the situation is pleasant but very urgent, the system is less satisfied. Further, if the system response is uncomfortable even if there is a margin, the satisfaction level becomes low. On the contrary, even when the result is not obtained, the satisfaction is not so low when there is a margin. That is, for example, it is determined by taking the average value of the values on the three axes, or by comparing the lowest value and the highest value, adopting the one with the larger absolute value, and adding the average values of the other two axes.

In the emotion recognition unit 203, the history storage unit 207
The emotion of the user is recognized from the dialogue history and the user's intention and satisfaction stored in FIG.

When the degree of satisfaction is used, the emotion of the user is described in several patterns as the dialogue changes, as shown in FIG. It is assumed that each emotion such as “expectation”, “anxiety”, and “optimism” is defined with a certain range around the characters in FIG. In reality, a state of "half expectation and half anxiety" is also conceivable, and the emotion of the user is not fixed at one point in FIG.

For example, in the case of a general user, if the number of conversations is small and the stages of the conversation progress to levels 0, 1, and 2, the user's satisfaction becomes positive and the stage does not progress forever. It becomes negative if you repeat only failures. Satisfaction varies depending on the score determined by the success or failure of the response stored in the history storage unit 207, but if it is early, thank you if you succeed at level 3, you can be satisfied even if you fail, but as the time becomes late I feel resigned and angry.

Therefore, the emotion transition model of FIG. 14 is used to adjust the state transition stage of the dialogue and the degree of satisfaction with the extracted emotion word. For example, the number of times of dialogue as a guide may be determined and compared with the flow of dialogue. That is, the vertical line in FIG. 14 is determined by the number of dialogues, and the degree of satisfaction is determined in a certain range on the line by the state transition stage, the above-mentioned score, and the like. Furthermore, if the satisfaction value obtained from the emotional words is collated and if they do not match, for example, the center of the satisfaction degree obtained in FIG. 14 and the satisfaction value obtained from the emotional words are averaged. It can be determined by the method.

At the initial stage of request acquisition in the dialogue, the range of the degree of satisfaction cannot be limited unless emotional words are entered. However, as shown in FIG. 14, it is rare that the degree of satisfaction is extremely touched at the beginning of the dialogue. I don't think so. For example, if the probability of the emotion is expressed in inverse proportion to the distance from the center of the character position representing the emotion in FIG. 14, “expectation” has a higher probability than “anxiety” at the beginning of the dialogue, so it is somewhat Satisfaction is higher than “expectation”. In a situation where the dialogue is a little advanced, the probability of "anxiety" is high if the request is still in the demanding state, so the degree of satisfaction is set to be slightly low.

It is also possible to allow a user such as a system administrator to redefine these emotional transitions and probability distributions. For example, a graphics interface as shown in FIG. 29 may be prepared and a pointing device such as a mouse may be used for the instruction. It is also possible to define different probability distributions for each emotion type.

As shown in FIG. 16, the user information storage unit 2
Consider the case where the user's operation history and personality information are stored in 06. If you are not the first user, set the emotional transition pattern of the user in the operation history,
It is possible to record it in the form of a locus on FIG. 14 and estimate it by using the most frequent pattern or average pattern. Also, by registering personality information in advance, such as the user being impatient, irritable, or humorous but warm and accustomed to a computer, even for the first time user to estimate using the registered pattern. be able to. "Angry boss"
By registering a plurality of typical patterns such as "Is a colleague who is impatient", information such as the status of the user can be obtained from a shared database and the emotion can be estimated.

Further, as shown in the first embodiment, emotion recognition can be performed from a plurality of input information by using a method such as facial expression recognition and voice recognition. For example, in FIG.
As shown in Figure 2, the frequency of soliloquy with non-verbal information such as intonation is frequently divided into two axes: [pleasant-unpleasant] and [loudness / loudness] such as loudness of voice and intensity of intonation. If you assign them, you will be given "joy,""surprise,""satisfaction,"
Can be categorized into emotions such as "anger" and "disappointment". The emotion may be defined by assigning it to the values on the above three axes. By combining this with the emotion recognition of text, more accurate emotion recognition result can be obtained. The emotion value determined by the emotion recognition unit 203 is the response plan creation unit 204.
Passed to.

The response plan is created by a combination of the intention type and the intention content of the system as shown in FIG. 32, for example, according to each response plan creation state of FIG. For example, when generating a text sentence, a sentence pattern such as a plain text or an interrogative sentence and information to be disclosed / requested are added. Further, facial expression information such as an attitude and familiarity for giving a facial expression to the response may be added.

The facial expression information is not limited to facial expressions,
This is information for giving a facial expression to a text sentence or a voice response. When the response generation unit receives the response plan, FIG.
It is converted into a response sentence as shown in the second sentence example and output.

First, a simple response plan that does not depend on the emotion recognition result and the processing of the response generation unit will be described with an example. The response generation unit 205 actually generates a response shown to the user from the response plan of one sentence unit according to the format shown in FIG. For example, when a text sentence as shown in FIG. 34 is generated, for example, a slot method (Makoto Nagao: “Artificial Intelligence Series 2 Linguistic Engineering” Shokoido, 1983) is used in advance in a format as shown in FIG. From the registered response sentence dictionary, a sentence applicable to the passed response plan is selected, and necessary information is embedded to generate a sentence.

In the response plans of FIGS. 32 and 33, as the intention type of the response, for example, accept (accepts the user's request), answer (answers the user's question), chime (conflict), and confirm.
m (confirm the user's request), confuse (expresses that the user's intention is not understood), goodby
e (declaration of dialogue end), greeting (first greeting), reject (reject user request), re
quest (request information from the user), s
orry (apologies), suggest (suggests),
thank you. The intent contents include a purpose, a schedule, a direct dialogue between users, a user situation, and a message.

The number of times is the number of times the same response plan is created in the same situation. FIG. 34 shows a response plan and a generated sentence example generated by the system for the dialogue example of FIG. Again, I ask "what kind of requirement?" Until the user's request is received, but once the request is obtained, "Is there any other request?"
And uses the frequency information to change the response. The frequency information can be counted from the dialogue history.

In the item, information disclosed in the response plan or information requested to the user is described. For example, in the response plan 3 of FIG. 34, the item is “act = meeting & date =?”, And the generated sentence is “when is the meeting?”.

On the other hand, the response generation unit 205 is shown in FIG.
The response sentence example of the format like this is registered, and the sentence example in which the kind of intention, the content of intention, the sentence pattern, the item, and the like match is found. The item act is used as a variable $ act in the example sentence, and is replaced with the value “meeting” specified in the response plan to complete the example sentence. At the end of the sentence, "?" Is added in the case of an interrogative sentence, and "." Is added in the ordinary sentence.

FIG. 36 is an example in which facial expression information of the response is added to FIG. For example, the response generation unit specifies values such as intimacy and attitude. The degree of intimacy is the degree of familiarity between the system and the user, or the degree of familiarity between the user who discloses information and the user who requests the information, and the user information storage unit 206 is set in advance.
Use the value registered in. The attitude has four levels of coarseness, such as polite, polite, normal, and rude. For example, an appropriate value is set according to the social status of the users, the social relationship between the users, or the user's emotion recognition result. decide.

The example of FIG. 37 is obtained by adding the attitude and familiarity to FIG. 34. The user “Sato” is a friend of the user “Yamamoto” (the owner of the agent) who discloses information and is the same age. Therefore, for example, intimacy = 4, attitude = 0
Value is added to the response plan.

For example, if it is found as a result of emotion recognition that "Sato" is angry, processing such as lowering the degree of intimacy and increasing the value of attitude is performed. On the other hand, if "Sato" is in a good mood, increase the degree of intimacy. On the contrary, if the acquaintance has a low intimacy, the intimacy and attitude will not change even if the other person is angry. This can be realized by registering the response strategy for determining these values in advance as a rule as shown in FIG. It is also possible to change the rule by a user such as a system administrator.

FIG. 39 shows an example of a response sentence example dictionary including familiarity and attitude. The upper part of FIG. 39 is an example of an answer sentence to the user's inquiry about the schedule, and the place (pl
ace) and the item type (act) of the schedule. Intimacy or attitude is a value such as 0 or 1,
A value having a range such as "1-3" is designated.

At the bottom of FIG. 39, greeting variations are described not only by intimacy and attitude, but also by time zone and classification such as whether or not the user has accessed before (known / unknown). . $ Place, $ a
ct and $ user are variables, and when the sentence is actually generated, the actual character string specified in the item of the response plan and the user name such as the login name are applied. Even if the time period is not included in the response plan, the time is checked by the response generation unit 205. For example, “morning” is used from 3:00 am to 11:00, “daytime” is used from 11:00 to 5 pm, and thereafter. You can ask for "night".

The response generation unit 205 searches the response sentence dictionary as shown in FIG. 39 for a sentence example in which the above-mentioned intention type, intention content, sentence pattern, number of times, intimacy, attitude, item content, and the like match. Further, the sentence example used once may be marked by using a used flag or the like, and different sentence examples may be used as much as possible in the same dialogue. If there is no match, select a sentence example with less items specified than the items specified in the response plan, or a sentence example with careful attitude.

In addition, when performing rule composition based on the case structure of a sentence, if the attitude is polite, only the end of the sentence is "damasu sentence", and if it is rude, a response sentence is generated. You may. Prepare multiple vocabulary and sentence patterns to use, and change according to values such as attitude and intimacy.

The response sentence generated by the response generation unit 205 is displayed in, for example, the text window shown in FIG. Also, a plurality of facial expressions of the image of the agent on the upper left of FIG. 20 are prepared as shown in FIG.
It may be changed according to the response plan. In this example, an image to be displayed is designated according to the type of utterance intention designated in the response plan and familiarity. Similarly for still images as well as moving images, the response is changed by preparing and switching multiple patterns in the same manner. You may prepare multiple patterns of facial expressions of actual people or dolls in photographs and videos. Also, P.
Ekman and WV Frisen ("Facial Action Coding System",
Consulting Psycologist Presss, 1977) can be used to synthesize facial expressions of different degrees using three-dimensional CG and the like.

Further, it is also possible to generate a response based on the response strategy by the following method.

For example, in FIG. 14, the feelings at the request acquisition stage can be "expectation", "anxiety", "confusion", "frustration" and "anger". This emotion is added to the items of the response plan shown in FIGS.
It is possible to change the response sentence, facial expression, etc. by giving the kind of emotion in the form like. It is also possible to add a satisfaction value. This makes it possible to express differences in strength even with the same emotion.

There is also a method in which after determining the emotion type of the user in FIG. 14, the values on the axes of FIG. 29 are recalculated and the response strategy is determined from the values of these three axes. That is, in a "denied" state, the system's response is untrusted, so it is necessary to carefully understand and execute the user's input.
It does not appeal to the user's emotions and generates a clerical response that does not look cold. If you apologize, take a serious attitude. Conversely, a strong "acceptance" may result in a relaxed response. Intimate attitude, light-heartedness and failure are acceptable.
Also, in the "urgent" state, the system does not generate too much extra response. However, if we cannot answer the user's request, we will promptly provide relevant information and show our sincerity. Clarify your system's capabilities and advise other users to seek help early. If the user's intention is difficult to understand, some options may be displayed and selected. On the other hand, when there is a "margin", the system itself tries to handle it as much as possible.

In the "unpleasant" emotional state, the user wants to slam his system straight or disgust to end the access. It is necessary to change the response policy before the "discomfort" becomes strong. If the system does not understand the user's intentions, then it is likely that the user is requesting that the system cannot. Present the service content of the system and let the user select it. If the user repeatedly requests that the system cannot do it, introduce the other user or system and terminate the access. If there is a tendency for "discomfort" even though the dialogue stage is proceeding smoothly, try changing the response attitude. In the case of "pleasant", the response policy may be continued.

In order to realize the above response, for example,
The response policy is set by "redundancy", "synchronization", "accuracy", "dominance", "information disclosure", and the like.

Redundancy sets the percentage of extra responses.
For example, it indicates whether or not to make an excuse when it fails.

The synchronism sets the degree of evaluation for the user and the user's story. A high degree of synchronicity makes it frustrating and a low degree of clerical response.

The accuracy sets whether to always respond honestly or to cheat. In other words, if you can't answer the user's request, to be honest, "I can't tell you".
Even if it becomes, it is possible to cheat such as "I do not understand a little".

The superiority sets the power relationship between the system and the user. When the superiority is high, the attitude is high for the user, and when the superiority is low, the attitude is poor.

The information openness is set by comparing the information held by the system with the appropriate access right of the user, and whether it is open to the public a little or a little. If the information disclosure is high, we will give away information that we normally do not teach, but if it is low, we will spare more than usual.

These values are added to the response plan to generate a response sentence in the format shown in FIG. 42, for example. here,
Each response policy is specified between 0 and 5. When "rejection" is strong, "redundancy" and "synchronism" are lowered.
Even if the "urgency" is strong, the "redundancy" is lowered, and if possible, the "information disclosure" is raised. When "discomfort" increases, if the other party is the boss, the "superiority" of the system is lowered. Prepare multiple rules for changing the response policy as described above, and
It is stored together with the application condition that the axis value is applied when it exceeds a certain threshold value.

Alternatively, the application condition may be described by a combination of values on the three axes. For example, even with the same degree of urgency, a rule can be described in which “redundancy” may be set higher when “accepting”, but lower when “rejecting”.

Further, according to the response plan as shown in FIG.
You can also change the facial expression of the image when responding. If the synchronization is high, the expression will be respectful, and if it is low, the expression will be clerical. If the redundancy is high, the expression is rich, and if the redundancy is low, the expression is limited to a few simple and easy-to-understand expressions. If the superiority is high, the expression is arrogant, and if it is low, the expression is frightening. A plurality of facial expression patterns as described above may be prepared, and the display may be changed when the response sentence is generated.

The response strategy can be realized not only by changing the facial expression of the response sentence, but also by changing the dialogue transition model. For example, a new state is added to the dialogue transition model example of FIG. 21, and the emotional state of the user, the intimacy with the user, and the attitude are added to the transition condition, so that a complicated response strategy as shown in FIG. 43 is obtained. Based response can be realized.

In this method, it is disliked that the dialogue transition model becomes complicated, but in that case, it is possible to maintain simplicity by preparing a plurality of dialogue transition models and switching to another model according to the emotion of the user. Further, if it is realized by using the emotional state of the user as a condition and having a rule for dynamically adding / changing a transition condition or a state, the problem of combination explosion can be avoided. For example, when the user is impatient, a strategy such as lowering the likelihood of the execution condition list shown in FIG. 22 or accelerating the conversation speed by reducing the execution conditions can be realized.

[0138]

As described above in detail, according to the present invention, by recognizing the emotion of the user according to the situation of the dialogue,
It is possible to generate a response in accordance with a response strategy considering the user's feeling. In other words, it is possible to generate an appropriate response that considers even non-explicit requests, rather than answering only requests that the user expresses explicitly, reducing the mental burden on the user, and smoothing the dialogue. Efficiency and efficiency. At this time, the user's intention and emotion can be more accurately understood by using a plurality of input information such as text, voice, and image, instead of a single input form. In addition, by using a plurality of response forms in combination and knowing the change in the emotional state of the user with respect to them, it becomes possible to select the response form according to the user's preference and situation.

The present invention can be applied not only to the disclosure of information but also to a dialogue system for other uses that handle similar emotions. Also, by applying it to communication between users and adding a function to convey each user's emotional state instead, misinterpretation of intention between users is reduced, and dialogue on the network and achievement of the dialogue purpose are achieved. Can be efficiently performed.

[Brief description of drawings]

FIG. 1 is a functional block diagram of an information disclosure device according to a first embodiment of the present invention.

FIG. 2 is a flowchart for explaining an operation procedure of the information disclosure device of the first embodiment.

FIG. 3 is a diagram showing a dialogue count emotion model of the first embodiment.

FIG. 4 is a diagram showing a dialogue count emotion model in which the emotion of the first embodiment is added.

FIG. 5 is a diagram showing correction conditions according to the first embodiment.

FIG. 6 is a diagram showing an algorithm for identifying an emotion expressed by an emotion word according to the first embodiment.

FIG. 7 is a diagram showing an example of a dialogue according to the first embodiment.

FIG. 8 is a diagram showing an emotion transition model of the first embodiment.

FIG. 9 is a diagram showing an emotion transition model of the first embodiment.

FIG. 10 is a diagram showing an emotion transition model of the first embodiment.

FIG. 11 is a diagram showing a discourse transition model of the first embodiment.

FIG. 12 is a diagram showing an emotion correspondence table according to the first embodiment.

FIG. 13 is a diagram showing an algorithm for digitizing user emotions according to the first embodiment.

FIG. 14 is a diagram showing an emotion transition model of the first embodiment.

FIG. 15 is a functional block diagram of an information disclosure device according to a second embodiment of the present invention.

FIG. 16 is a functional block diagram of an information disclosure device according to a second embodiment.

FIG. 17 is a functional block diagram of the information disclosure device according to the second embodiment.

FIG. 18 is a functional block diagram of an information disclosure device according to a second embodiment.

FIG. 19 is a flowchart for explaining an operation procedure of the information disclosure device of the second embodiment.

FIG. 20 is a diagram showing an example of a response of the second embodiment.

FIG. 21 is a diagram showing a dialogue transition model of the second embodiment.

FIG. 22 is a diagram showing an execution condition list according to the second embodiment.

FIG. 23 is a diagram showing a dialogue history storage structure of the second embodiment.

FIG. 24 is a diagram showing an example of utterance intention according to the second embodiment.

FIG. 25 is a diagram showing an example of a user request according to the second embodiment.

FIG. 26 is a diagram showing a keyword dictionary of the second embodiment.

FIG. 27 is a diagram showing a semantic representation of speech according to the second embodiment.

FIG. 28 is a diagram showing three axes forming the emotional space according to the second embodiment.

FIG. 29 is a diagram showing a state in which an emotion name is assigned to the emotion space according to the second embodiment.

FIG. 30 is a diagram showing an emotion area table according to the second embodiment.

FIG. 31 is a diagram showing a state in which non-verbal information is assigned to the emotion space according to the second embodiment.

FIG. 32 is a diagram showing a response plan creation table according to the second embodiment.

FIG. 33 is a diagram showing a format of a response according to the second embodiment.

FIG. 34 is a diagram showing an example of response generation according to the second embodiment.

FIG. 35 is a diagram showing a format of a response according to the second embodiment.

FIG. 36 is a diagram showing a format of a response to which facial expression information of the second embodiment is added.

FIG. 37 is a diagram showing an example of response generation in which the attitude and intimacy of the second embodiment are added.

FIG. 38 is a view showing a response strategy of the second embodiment.

FIG. 39 is a diagram showing an example of a response sentence example dictionary according to the second embodiment.

FIG. 40 is a diagram showing a facial expression pattern of an image of an agent according to the second embodiment.

FIG. 41 is a diagram showing an example of response generation after emotion recognition according to the second embodiment.

FIG. 42 is a diagram showing an example of an application plan of the second embodiment.

FIG. 43 is a diagram showing a response strategy of the second embodiment.

[Explanation of symbols]

10 ... Information disclosure device, 101 ... Input unit, 102 ... Data storage unit, 103 ... Data search management unit, 104 ... Request reception unit, 105 ... Response plan creation unit, 106 ... User emotion recognition unit, 107 ... Response generation unit, 20 ... Information disclosure device, 20
1 ... Input unit, 202 ... Intentional emotion information extraction unit, 203 ... Emotion recognition unit, 204 ... Response plan generation unit, 205 ... Response generation unit, 206 ... User information storage unit, 207 ... History storage unit, 208a-c ... Data Communication unit, 209a-c ... Process, 210 ... Dialog management unit, 211 ... Search unit, 212 ...
Data storage.

Claims (1)

[Claims] 1. Input means for inputting a plurality of forms of data including text, voice, image, and pointing position; extraction means for extracting user's intention and emotion information from the data input by the input means; In the information disclosure device having a response plan creating means for creating a response plan based on the extraction result of the extracting means, and a response creating means for creating a response to the user based on the created response plan, the response plan An emotion recognition means for recognizing the emotional state of the user from the internal state of the generating means, the extracted intention and emotional information of the user, and the transition on the time axis of the conversational situation information including the type of the response plan created. Comprising, the response plan creation means selects or changes a response strategy according to the recognition result of the emotion recognition means, Information disclosure and wherein the creating a response plan that matches the response strategy.