JP7221902B2 - Dialogue device, program and method for switching dialogue content according to user's interest level - Google Patents

Description

TECHNICAL FIELD The present invention relates to technology of a dialogue device that realizes natural dialogue with a user.

A dialogue device generally picks up a user's uttered voice with a microphone, converts the uttered voice into text by voice recognition, generates a dialogue sentence according to the text, and generates a dialogue sentence from the dialogue sentence by speech synthesis. The dialogue voice is uttered from the speaker. Examples of interactive AI (Artificial Intelligence) include dialogue systems such as “Siri (registered trademark)” and “Talking Concier (registered trademark)”, “Google Home (registered trademark)” and “Amazon Echo (registered trademark)”. trademark)”. Such technology is suitable for performing specific tasks, such as playing music or providing weather forecasts and news.

There are also robot technologies such as "SOTA (registered trademark)" and "Unibo (registered trademark)" that personify the user's conversation partner. According to this technology, a robot extracts a keyword from the user's surroundings, generates a dialogue sentence with the keyword embedded in a template, and utters the dialogue sentence to the user. This makes it possible to obtain a trigger for dialogue from the surrounding situation of the user.

Conventionally, there are techniques for continuing dialogue in addition to responding to requests from users. For example, there is a technique in which a plurality of dialogue learning engines are provided, and a strongly related dialogue learning engine is selected using the utterance sentence input by the user and the past dialogue history, and the dialogue learning engine replies ( For example, see Patent Document 1).
In addition, after associating a keyword list with each topic, multiple keywords are extracted from the user's utterances by morphological analysis, and the conversation continues on topics that have a predetermined relationship (similarity and hierarchical relationship) with the keyword list. There is also a technique to allow the operation (see, for example, Patent Literature 2). According to this technique, for dialogues unrelated to keywords, the dialogue proceeds according to a prepared dialogue scenario.

JP-A-2007-47488 JP 2017-49471 A

NTT Communication Science Laboratories, "Challenges to Versatile Semantic Analysis Technology", [online], [March 5, 2020], Internet <URL: https://www.ntt.co.jp/ journal/0806/files/jn200806024.pdf> MathWorks, "Face Recognition", [online], [March 5, 2020], Internet <https://jp.mathworks.com/discovery/face-recognition.html>

The techniques described in Patent Literatures 1 and 2 described above have a problem in that there are few topics to talk about because the dialogue progresses according to dialogue scenarios prepared in advance. In particular, the technology described in Patent Literature 2 refers only to a predetermined relationship between a user's utterance and a topic, so development of the topic depends on the keyword list.

In response to this, the inventors wondered whether it would be possible to develop a "chat dialogue type AI" that expands the content of dialogue on topics of high interest to the user in order to continue the dialogue.

SUMMARY OF THE INVENTION Accordingly, it is an object of the present invention to provide an interactive device, a program, and a method for switching dialogue contents according to the user's degree of interest in order to continue chat-like dialogue without boring the user.

According to the present invention, in a dialogue device that dialogues with a user using a dialogue learning engine selected from a plurality of dialogue learning engines,
Vocabulary extraction means for acquiring dialogue scenarios from all dialogue learning engines and extracting a plurality of vocabularies based on predetermined conditions from all dialogue scenarios;
a thesaurus dictionary creating means for creating a thesaurus dictionary in which the plurality of extracted vocabularies are classified into clusters having similar semantic attributes, and marking the vocabularies corresponding to the content of the current dialogue;
user data acquisition means for acquiring multimedia data based on an interacting user;
an interest level estimation engine for estimating a user's level of interest in current dialogue content from multimedia data;
dialogue learning engine selection means for selecting a dialogue learning engine having a dialogue scenario with the highest degree of similarity to the marker vocabulary of the thesaurus dictionary;
It is characterized by having a marker vocabulary movement control means for moving the marker to another vocabulary belonging to the same cluster as the marker vocabulary when the interest is high, and moving the marker to a vocabulary belonging to a cluster different from the marker vocabulary when the interest is low.

According to another embodiment of the dialogue device of the present invention,
Vocabularies and dialogue scenarios are expressed in vectors so that the closer they are in terms of semantic attributes, the closer the distance is.
It is also preferable that the thesaurus dictionary creation means classify the terms into the same cluster as the distance between the vectors of the vocabularies becomes closer.

According to another embodiment of the dialogue device of the present invention,
It is also preferable that the marker vocabulary movement control means moves the marker to a vocabulary belonging to a cluster that is farther from the marker vocabulary as the interest level is lower.

According to another embodiment of the dialogue device of the present invention,
The plurality of dialogue learning engines also preferably includes a current affairs dialogue learning engine, a television dialogue learning engine, and/or a specialized dialogue learning engine in addition to the general purpose dialogue learning engine.

According to another embodiment of the dialogue device of the present invention,
The interactive device is connected to the camera and
Multimedia data is a feature quantity of a user's face image captured by a camera,
The interest degree estimation engine learns by associating the feature amount of the face image with the user's degree of interest in the learning stage. It is also preferable to output the degree of interest of

According to another embodiment of the dialogue device of the present invention,
It is also preferable that the feature amount of the facial image in the interest level estimation engine is based on facial expression, line of sight and/or gesture.

According to another embodiment of the dialogue device of the present invention,
The dialogue device is connected to a microphone and
The multimedia data is an utterance sentence obtained by speech recognition from the user's utterance voice picked up by a microphone,
The interest level estimation engine learns by associating the feature quantity of the user's uttered sentence with the user's interest level in the learning stage, and inputs the feature quantity of the uttered sentence as multimedia data in the estimation stage. , it is also preferable to output the degree of interest of the user.

According to the present invention, in a program that causes a computer installed in a device that interacts with a user to function using a dialogue learning engine selected from a plurality of dialogue learning engines,
Vocabulary extraction means for acquiring dialogue scenarios from all dialogue learning engines and extracting a plurality of vocabularies based on predetermined conditions from all dialogue scenarios;
a thesaurus dictionary creating means for creating a thesaurus dictionary in which the plurality of extracted vocabularies are classified into clusters having similar semantic attributes, and marking the vocabularies corresponding to the content of the current dialogue;
user data acquisition means for acquiring multimedia data based on an interacting user;
an interest level estimation engine for estimating a user's level of interest in current dialogue content from multimedia data;
dialogue learning engine selection means for selecting a dialogue learning engine having a dialogue scenario with the highest degree of similarity to the marker vocabulary of the thesaurus dictionary;
The computer functions as marker vocabulary movement control means for moving the marker to another vocabulary belonging to the same cluster as the marker vocabulary when the interest is high, and to the vocabulary belonging to a cluster different from the marker vocabulary when the interest is low. do.

According to the present invention, in a dialogue method for a device that dialogues with a user using a dialogue learning engine selected from a plurality of dialogue learning engines,
The device
Acquire dialogue scenarios from all dialogue learning engines, extract multiple vocabularies based on predetermined conditions from all dialogue scenarios, and create a thesaurus dictionary that classifies the multiple extracted vocabularies into clusters with similar semantic attributes. and a thesaurus dictionary that marks the vocabulary corresponding to the content of the current dialogue;
an interest level estimation engine for estimating the user's level of interest in current dialogue content from multimedia data based on the user during the dialogue;
a first step of obtaining multimedia data based on an interactive user;
a second step of estimating the user's interest in the current dialogue content from the acquired multimedia data using an interest estimation engine;
a third step of moving the marker to other vocabularies belonging to the same cluster as the marker vocabulary with higher interest and to vocabularies belonging to different clusters from the marker vocabulary having lower interest;
a fourth step of selecting the dialogue learning engine having the dialogue scenario that has the highest degree of similarity with the marker vocabulary of the thesaurus dictionary;
is repeatedly executed.

ADVANTAGE OF THE INVENTION According to the dialogue apparatus, program, and method of the present invention, it is possible to progress the dialogue according to the user's degree of interest in order to continue the dialogue, such as chatting, without getting tired of the user.

1 is a system configuration diagram showing a surrounding environment of a dialogue device according to the present invention; FIG. 1 is a functional configuration diagram of an interactive device according to the present invention; FIG. FIG. 4 is an explanatory diagram of a vocabulary extraction unit and a thesaurus dictionary creation unit in the present invention; FIG. 4 is an explanatory diagram of an interest degree estimation engine in the present invention; FIG. 4 is an explanatory diagram of a marker vocabulary movement control section and a dialogue learning engine selection section in the present invention; FIG. 3 is a sequence diagram representing interaction between the interaction device and the user;

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

FIG. 1 is a system configuration diagram showing the surrounding environment of a dialogue device according to the present invention.

According to FIG. 1, the dialogue device 1 is a robot as a character, and can advance the dialogue according to the user's interest level in order to continue the dialogue with the user, such as chatting.
The dialogue device 1 is equipped with a microphone and a speaker as input/output devices of a user interface for dialogue. A microphone picks up a user's uttered voice, and a speaker utters a dialogue voice to the user.
Further, it may be one equipped with keys and a display as an input/output device of a user interface for interaction. It is also possible to obtain a spoken sentence by user's key input and clearly show the spoken sentence to the user through a display.

The dialogue device 1 acquires user-based multimedia data in order to estimate the user's degree of interest during dialogue. Here, there are the following two embodiments as a user interface for acquiring multimedia data.
<First Embodiment: Camera Capturing User's Face Image>
<Second Embodiment: Microphone for Picking up User's Voice>
(The microphone is similar to the user interface for interaction)

FIG. 2 is a functional configuration diagram of the interactive device in the present invention.

According to FIG. 2, the dialogue device 1 includes a plurality of different dialogue learning engines 101 to 10n, a vocabulary extraction unit 11, a thesaurus dictionary creation unit 12, a user data acquisition unit 13 (face image recognition unit 131, voice recognition unit 131). 132), an interest degree estimation engine 14, a marker vocabulary movement control unit 15, a dialog learning engine selection unit 16, a dialog execution unit 171, and a voice conversion unit 172. These functional components can be realized by executing a program that causes a computer installed in the interactive device to function. In addition, the processing flow of these functional components can also be understood as an interaction method of the device.

[Dialogue learning engines 101-10n]
The dialogue device 1 comprises a plurality of different dialogue learning engines 101-10n. Each interactive learning engine 10 stores a unique "interactive scenario." The dialogue learning engine 10 progresses the dialogue while tracing the dialogue scenario according to the utterance sentence from the user.
A dialogue scenario is a structure in which dialogue nodes including dialogue sentences are traced in a tree-like manner according to the utterance sentences of the user. For example, it may be written in a FST (Finite State Transducer) script file that describes a response pattern to a user's input.

The dialogue learning engine 10 may include, for example, a current affairs dialogue learning engine, a television dialogue learning engine, and/or a specialized dialogue learning engine in addition to a general purpose dialogue learning engine. It is equipped with dialogue learning engines in multiple different fields in order to develop topics in different fields while maintaining commonality with ongoing topics, just like chatting between people. As a result, it is possible to realize a chat-like dialogue that is rich in topics and is difficult to get tired of.

(General-purpose interactive learning engine)
A general-purpose dialogue learning engine stores, for example, a dialogue scenario that progresses a daily dialogue. For example, it assumes general human-to-human dialogue constructed by crowdsourcing.
(Dialogue learning engine for current affairs)
The current affairs dialogue learning engine stores dialogue scenarios that progress dialogues based on, for example, news topics. For example, conversations are conducted based on news that has become a hot topic on SNS (Social Networking Service) and comments (tweets, etc.) on the news.
(Dialogue learning engine for TV)
The television dialogue learning engine stores, for example, dialogue scenarios that progress dialogues based on television program content. For example, topics based on the metadata of the program content currently being broadcast (for example, an electronic program guide, subtitles of narration, etc.) are advanced.
(Professional interactive learning engine)
A specialized dialogue learning engine stores dialogue scenarios that develop dialogues based on a particular field of expertise, such as science and technology.
In addition, it is preferable to have a dialogue learning engine with various properties.

FIG. 3 is an explanatory diagram of the vocabulary extraction unit and the thesaurus dictionary creation unit in the present invention.

[Vocabulary extraction unit 11]
The vocabulary extraction unit 11 acquires dialogue scenarios from all the dialogue learning engines 101 to 10n, and extracts a plurality of vocabularies (a group of effective words) based on predetermined conditions from all the dialogue scenarios. A dialogue scenario is a text group that describes a response sentence to a user's utterance sentence.
The vocabulary extraction unit 11 extracts common nouns by morphological analysis from a large amount of texts of these dialogue scenarios (see FIG. 3). A large number of extracted common nouns are output to the thesaurus dictionary creation unit 12 .

[Thesaurus dictionary creation unit 12]
The thesaurus dictionary creating unit 12 creates a thesaurus dictionary in which the plurality of extracted vocabularies are classified into clusters having similar semantic attributes.

A "vocabulary" is represented by a vector (variance vector) based on Word2vec, for example, so that the distance (Euclidean distance) becomes closer as semantic attributes are closer.
"Word2vec" is a technology that converts words into vectors and compresses the dimensions in order to capture the meaning and grammar of words. For two vocabularies, the higher the similarity, the shorter the vector-to-vector distance, and the lower the similarity, the longer the vector-to-vector distance. Naturally, vocabularies belonging to the same category have a high degree of similarity (a short distance between vectors).
The lexical similarity is represented by cosine similarity as follows.
S(a,b)=cos θ=(Va·Vb)/(|Va||Vb|)
Va: feature vector of the first vocabulary Vb: feature vector of the second vocabulary S(a,b): 0 to 1 (approaching 1 as the degree of similarity increases)

The "thesaurus dictionary" means the distance from the first vocabulary to the second vocabulary, and the degree of topic development when the first topic transitions to the second topic.
The thesaurus dictionary of the present invention classifies the vectors of each vocabulary into the same cluster as the Euclidean distance (synonyms or related words) is closer. For example, a clustering method such as k-means is used to classify a large number of vocabularies into multiple clusters. According to FIG. 3, one category includes multiple vocabularies. Vocabularies belonging to the same cluster have close Euclidean vector distances.
Note that the thesaurus dictionary may be classified using Wordnet into a sub-tree structure in which a plurality of vocabularies are placed under each category hierarchy.

According to FIG. 3, one category and a plurality of vocabularies have the following relationship.
Category "Domestic"=> Vocabulary "Politics""Society""People"
Category "International"=> Vocabulary "China, Korea, Russia""U.S.EU""..."
・・・・・・・・
For example, the distances between vectors of the vocabularies "politics", "society" and "people" are relatively close. In addition, the distance between the vectors of the vocabularies “China, Korea, Russia” and “US, EU” becomes relatively close. On the other hand, the vector-to-vector distance between the vocabulary "society" and the vocabulary "China, Korea, Russia" is relatively long.
A category is also represented by a vector. For example, the vector for the category "domestic" may be the average of the vectors for the lexical terms "politics", "society" and "people".

The thesaurus dictionary creating unit 12 also expresses each dialogue scenario as a vector. For example, for "all vocabularies included in the dialogue scenario", vectors of all vocabularies are averaged and represented by one vector. That is, one vocabulary or category and one dialogue scenario can be judged by the similarity of the vector distance.
Vocabularies, categories, and dialogue scenarios are expressed in vectors such that the closer the semantic attributes, the closer the distance.

In addition, the thesaurus dictionary has a marker attached to the vocabulary corresponding to the content of the current dialogue. According to the present invention, by moving a marker attached to a vocabulary in the thesaurus dictionary, it is developed so as to switch to a topic based on that marker vocabulary or category. Movement of the marker vocabulary will be described later by the marker vocabulary movement control unit 15 .

Note that even if an arbitrary vocabulary belongs to the category with the highest degree of similarity, if the distance between the vectors of the vocabulary and the category is equal to or greater than a predetermined threshold, it is determined to be an "unknown word." good too. In that case, it is also possible to perform clustering on all unknown words and construct a plurality of new categories.

<First Embodiment: Camera Capturing User's Face Image>
[User data acquisition unit 13]
The user data acquisition unit 13 acquires multimedia data based on the user in conversation. Multimedia data in the first embodiment is a face image captured by a camera.

(Face image recognition unit 131)
The face image recognition unit 131 receives a user's face image (video) captured by an in-camera, and detects a face area of each user from each image. Facial regions are searched for portions of the image that match a template constructed from facial features. For example, the relative positions and sizes of facial parts, and the shapes of eyes, nose, cheekbones, and chin are used.
Then, the facial image recognition unit 131 extracts time-series changes in facial parameters from each time-series image as a feature amount. The time-series feature amount of the face image is based on facial expression, line of sight and/or gesture. As a face recognition algorithm, there are various existing methods (for example, see Non-Patent Document 2).
Then, the time-series feature amount of the face image is output to the degree-of-interest estimation engine 14 .

[Interest degree estimation engine 14]
The interest level estimation engine 14 estimates the user's level of interest in the content of the current dialogue from multimedia data (feature amounts of face images).

FIG. 4 is an explanatory diagram of the degree-of-interest estimation engine in the present invention.

According to FIG. 4, the interest level estimation engine 14 learns by associating the feature amount of the face image with the user's interest level in the learning stage.
For example, a data set (for example, 45,723 images) of IMDb (Internet Movie Database) can be used as the face image of the training data in the learning stage. A time-series feature amount is extracted from each face image, and the user's degree of interest is assigned to the feature amount. For example, a relatively high degree of interest is given to the feature amount of a face image with eyes wide open, and a relatively low degree of interest is given to the feature amount of a face image with downcast eyes. The teacher data in which the feature amount of the face image and the degree of interest of the user are associated with each other are learned based on, for example, a convolutional neural network.
After that, as an estimation step, the interest level estimation engine 14 outputs an estimated interest level by inputting the feature amount of the user's face image. The estimated degree of interest is output to the marker vocabulary movement control section 15 .

FIG. 5 is an explanatory diagram of the marker vocabulary movement control section and the dialogue learning engine selection section in the present invention.

[Dialogue learning engine selection unit 16]
The dialogue learning engine selection unit 16 selects any one dialogue engine from among a plurality of dialogue learning engines. At this time, the dialogue learning engine having the dialogue scenario with the highest degree of similarity to the "marker vocabulary" of the thesaurus dictionary is selected.

According to FIG. 5, the thesaurus dictionary represents a cluster including a plurality of vocabularies for each category. In addition, each category is linked to one of the dialogue learning engines as follows.
Dialogue learning engine for current affairs <-> Category "domestic""international"
Economic Dialogue Learning Engine <-> Category "Economy"
Dialogue learning engine for TV <-> Category "Entertainment""Sports"
Interactive learning engine for technology <-> Category "IT""Science"
General-purpose interactive learning engine <-> Category "Life"
Dialogue learning engine for region <-> Category "Region"
According to the thesaurus dictionary, each category to which multiple "vocabularies" belong is represented by a vector. The entire dialogue scenario of each dialogue learning engine 10 is also represented by a vector. In this case, each category can be associated with the dialogue learning engine 10 that has the most similar dialogue scenario as a vector representation.
As a result, it is possible to select the dialogue learning engine 10 having the dialogue scenario with the highest degree of similarity to the "marker vocabulary".

[Marker vocabulary movement control unit 15]
The marker vocabulary movement control unit 15 moves vocabulary (synonyms or related words) belonging to the same cluster as the marker vocabulary as the interest level is higher, and vocabularies (antonyms or irrelevant words) belonging to a cluster different from the marker vocabulary as the interest level is lower. word).

FIG. 6 is a sequence diagram representing the interaction between the interactive device and the user.

According to the sequence of FIG. 6(a), the marker vocabulary movement control unit 15 of FIG. 5 represents a case where "user's degree of interest is high".
(S11) It is assumed that the interactive device 1 puts a marker on the vocabulary "game" in the sequence dictionary at the elapsed time t0. According to FIG. 5, it is assumed that the vocabulary "game" belongs to the category "entertainment" and the dialog learning engine for television is selected.
(S12) A dialog learning engine for television is selected for the category "entertainment" of the marker vocabulary "game" (see FIG. 5). The dialogue learning engine for television outputs a dialogue sentence suitable for "game" from among the dialogue scenarios, "In games, AAA is popular now!".
(S13) In response, the interactive device 1 captures the user's facial image (multimedia data). Assume that the user's interest level of 0.9 is estimated from the face image. For example, a threshold value of 0.7 or more can be determined as having a high degree of interest.
(S14) At this time, the interactive device 1 moves the marker in the sequence dictionary to the vocabulary "television" belonging to the same category (cluster) at the time point t1, at the vector distance corresponding to the degree of association (see FIG. 5). .
(S15) A dialogue learning engine for television is selected for the category "entertainment" of the marker vocabulary "television". The dialogue learning engine for television acquires the user's utterance sentence "I like BBB", and selects the dialogue sentence "BBB is good at entertainer X!" is outputting

According to the sequence of FIG. 6(b), the marker vocabulary movement control unit 15 of FIG.
(S21) Same as S11 in FIG. 6(a).
(S22) Same as S12 in FIG. 6(a).
(S23) In response, the interactive device 1 captures the user's facial image (multimedia data). Assume that the user's interest level of 0.2 is estimated from the face image. For example, a threshold value of 0.3 or less can be determined as having a low degree of interest.
(S24) At this time, the interactive device 1 moves the marker in the sequence dictionary at the elapsed time t1 to the word "health" belonging to another category (cluster) at a vector distance corresponding to the degree of association (see FIG. 5). ).
(S25) A general-purpose interactive learning engine is selected for the category "life" of the marker vocabulary "health." The general-purpose dialogue learning engine acquires the user's utterance sentence "..." and outputs the dialogue sentence "Do you exercise every day?" suitable for "health" from the dialogue scenario.

In this way, the marker vocabulary movement control unit 15 moves the marker to a vocabulary belonging to a cluster having a longer Euclidean distance from the marker vocabulary as the interest level is lower. That is, the distance by which the marker is moved next is controlled according to the degree of interest. As the distance of the current instruction increases, dialogue sentences are generated on different topics.

[Dialogue Execution Unit 171/Voice Conversion Unit 172]
Dialogue execution unit 171 uses the dialogue learning engine selected by dialogue learning engine selection unit 16 to output a dialogue sentence for the user.
The voice conversion unit 172 converts the dialog sentence output from the dialog execution unit 171 into a voice signal by voice synthesis, and outputs the voice signal to a speaker.
In this manner, the dialogue device 1 progresses the dialogue with the user by voice.

<Second Embodiment: Microphone for Picking up User's Voice>
According to the first embodiment, it has been described that the user-based multimedia data is the user's face image captured by the camera.
On the other hand, according to the second embodiment, it is assumed that user-based multimedia data is the user's voice picked up by a microphone. In this case, according to FIG. 2, the user data acquisition unit 13 functions as the voice recognition unit 132.

(Voice recognition unit 132)
The speech recognition unit 132 outputs an utterance sentence by speech recognition from the user's voice picked up by the microphone.
In this case, the interest level estimation engine 14 learns by associating the feature quantity of the uttered sentence with the user's interest level in the learning stage. In addition, the interest level estimation engine 14, as an estimation stage, inputs an utterance sentence as multimedia data and outputs the user's level of interest.

As described above in detail, according to the dialogue apparatus, program and method of the present invention, the dialogue proceeds according to the user's interest level in order to continue the dialogue like chatting without getting tired of the user. be able to.
As a result of experiments on subjects, it was found that the dialog system of the present invention can perform conversations such as casual conversations two to three times longer than the dialog systems of the prior art for a wide range of topics such as politics and sports. was able to continue.

For the various embodiments of the present invention described above, various changes, modifications and omissions within the spirit and scope of the present invention can be easily made by those skilled in the art. The foregoing description is exemplary only and is not intended to be limiting. The invention is to be limited only as limited by the claims and the equivalents thereof.

1 dialogue device 10 dialogue learning engine 11 vocabulary extraction unit 12 thesaurus dictionary creation unit 13 user data acquisition unit 131 face image recognition unit 132 speech recognition unit 14 interest level estimation engine 15 marker vocabulary movement control unit 16 dialogue learning engine selection unit 171 dialogue execution Unit 172 Voice conversion unit

Claims (9)

In a dialogue device that dialogues with a user using a dialogue learning engine selected from a plurality of dialogue learning engines,
Vocabulary extraction means for acquiring dialogue scenarios from all dialogue learning engines and extracting a plurality of vocabularies based on predetermined conditions from all dialogue scenarios;
a thesaurus dictionary creating means for creating a thesaurus dictionary in which the plurality of extracted vocabularies are classified into clusters having similar semantic attributes, and marking the vocabularies corresponding to the content of the current dialogue;
user data acquisition means for acquiring multimedia data based on an interacting user;
an interest level estimation engine for estimating a user's level of interest in current dialogue content from multimedia data;
dialogue learning engine selection means for selecting a dialogue learning engine having a dialogue scenario with the highest degree of similarity to the marker vocabulary of the thesaurus dictionary;
a marker vocabulary movement control means for moving the marker to another vocabulary belonging to the same cluster as the marker vocabulary for a higher level of interest and to a vocabulary belonging to a cluster different from the marker vocabulary for a lower interest level. Device. Vocabularies and dialogue scenarios are expressed in vectors so that the closer they are in terms of semantic attributes, the closer the distance is.
2. The dialogue apparatus according to claim 1, wherein the thesaurus dictionary creating means classifies the words into the same cluster as the distance between the vectors of the words is closer. 3. The dialogue apparatus according to claim 1, wherein the marker vocabulary movement control means moves the marker to a vocabulary belonging to a cluster having a greater distance from the marker vocabulary as the degree of interest is lower. 4. Any one of claims 1 to 3, wherein the plurality of dialogue learning engines includes a current affairs dialogue learning engine, a television dialogue learning engine, and/or a specialized dialogue learning engine in addition to the general purpose dialogue learning engine. or 1. A dialogue device according to claim 1. The interactive device is connected to the camera and
Multimedia data is a feature quantity of a user's face image captured by a camera,
The interest degree estimation engine learns by associating the feature amount of the face image with the user's degree of interest in the learning stage. 5. The interactive device according to claim 1, wherein the degree of interest of the user is output. 6. The dialogue apparatus according to claim 5, wherein the feature amount of the facial image in the interest level estimation engine is based on facial expression, line of sight and/or gesture. The dialogue device is connected to a microphone and
The multimedia data is an utterance sentence obtained by speech recognition from the user's utterance voice picked up by a microphone,
The interest level estimation engine learns by associating the feature quantity of the user's uttered sentence with the user's interest level in the learning stage, and inputs the feature quantity of the uttered sentence as multimedia data in the estimation stage. 7. The interactive device according to claim 1, wherein the user's degree of interest is output. In a program that operates a computer installed in a device that interacts with a user using a dialogue learning engine selected from a plurality of dialogue learning engines,
Vocabulary extraction means for acquiring dialogue scenarios from all dialogue learning engines and extracting a plurality of vocabularies based on predetermined conditions from all dialogue scenarios;
a thesaurus dictionary creating means for creating a thesaurus dictionary in which the plurality of extracted vocabularies are classified into clusters having similar semantic attributes, and marking the vocabularies corresponding to the content of the current dialogue;
user data acquisition means for acquiring multimedia data based on an interacting user;
an interest level estimation engine for estimating a user's level of interest in current dialogue content from multimedia data;
dialogue learning engine selection means for selecting a dialogue learning engine having a dialogue scenario with the highest degree of similarity to the marker vocabulary of the thesaurus dictionary;
The computer functions as marker vocabulary movement control means for moving the marker to another vocabulary belonging to the same cluster as the marker vocabulary when the interest is high, and to the vocabulary belonging to a cluster different from the marker vocabulary when the interest is low. program to do. In a dialogue method for a device that dialogues with a user using a dialogue learning engine selected from a plurality of dialogue learning engines,
The device
Acquire dialogue scenarios from all dialogue learning engines, extract multiple vocabularies based on predetermined conditions from all dialogue scenarios, and create a thesaurus dictionary that classifies the multiple extracted vocabularies into clusters with similar semantic attributes. and a thesaurus dictionary that marks the vocabulary corresponding to the content of the current dialogue;
an interest level estimation engine for estimating the user's level of interest in current dialogue content from multimedia data based on the user during the dialogue;
a first step of obtaining multimedia data based on an interactive user;
a second step of estimating the user's interest in the current dialogue content from the acquired multimedia data using an interest estimation engine;
a third step of moving the marker to other vocabularies belonging to the same cluster as the marker vocabulary with higher interest and to vocabularies belonging to different clusters from the marker vocabulary having lower interest;
a fourth step of selecting the dialogue learning engine having the dialogue scenario that has the highest degree of similarity with the marker vocabulary of the thesaurus dictionary;
A device interaction method characterized by repeatedly executing

Images