JP6630304B2 - Dialogue destruction feature extraction device, dialogue destruction feature extraction method, program - Google Patents

Description

  The present invention relates to a chat dialogue technology, and more particularly to a dialogue destruction force estimating technology for estimating a degree of inadequacy of a certain utterance to break a dialogue.

  2. Description of the Related Art In recent years, there has been an increasing need for a chat dialogue system for performing an open-domain chat that does not aim at a specific task. In the chat dialogue system, for example, a method of uttering using dialogue knowledge manually constructed (Non-Patent Document 1) and a method of constructing utterance using Web information (Non-Patent Document 2) are used. .

Toyomi Meguro, Hiroaki Sugiyama, Ryuichiro Higashinaka, "Construction of Dialogue System Based on Fusion of Rule-Based Utterance Generation and Statistical Utterance Generation," Proceedings of the Japan Society for Artificial Intelligence, pp.1-4, 2014. Hiroaki Sugiyama, Toyomi Meguro, Ryuichiro Higashinaka, Yasuhiro Minami, "Generating Response Sentences Using Dependencies and Examples for User Utterances with Arbitrary Topics," Transactions of the Japanese Society for Artificial Intelligence, vol.30, no.1, pp .183-194, 2015.

The chat dialogue system needs to respond to a very wide range of topics contained in the user's utterance. Therefore, it is difficult to continuously output an appropriate response, and in the current chat dialogue system, an utterance that breaks down the dialogue is often generated (Reference Non-Patent Document 1).
(Reference non-patent document 1: Ryuichiro Higashinaka, Kotaro Funakoshi, "Collecting chat conversation data and conversation failure annotation in the Project Next NLP dialog task", The 72nd meeting of the Japanese Society for Artificial Intelligence Language and Speech Understanding and Dialogue Processing, pp. 45-50, 2014.)
If it is possible to detect such an utterance that may break the dialog before actually uttering it and suppress the output, it is considered that the continuation of the dialog in the chat dialog system becomes easy.

  Therefore, an object of the present invention is to provide a technique for extracting a dialogue destruction feature amount that can be used to estimate a dialogue destructive force indicating a degree at which an utterance breaks a dialogue.

  According to one embodiment of the present invention, J is an integer equal to or greater than 1 and a j-th feature amount (1 ≦ j ≦ J) is a feature amount indicating whether or not a dialogue has failed. A dialogue destruction feature amount extraction unit that extracts a dialogue destruction feature amount that is a combination of the j-th type feature amount (1 ≦ j ≦ J). The unit includes, for each j that satisfies 1 ≦ j ≦ J, a j-th feature amount calculating unit that calculates the j-th feature amount from the dialogue, and sets the frequent word string feature amount to a predetermined frequency or more within the dialogue. The feature amount using the character string of the appearing word Ngram as an element, the inter-utterance similarity feature amount, the feature amount indicating how similar the last utterance in the dialogue is to other utterances, the sentence length feature amount, The features representing the word length and character length of the last utterance in the dialogue, the number of turns feature, and the features representing the number of turns elapsed since the start of the dialogue , The word vector feature, the feature expressed as a vector generated from the vector representing the word N-gram of each utterance included in the dialog, the question type feature, the utterance immediately before the last utterance in the dialog If it is a question, the features representing the question type to be estimated, the question class features, and the word that is presumed to be required for the answer if the utterance immediately before the last utterance in the dialogue is a question Any one of a vector representing a class, a vector representing a word class included in the last utterance, a difference vector between them, and a boolean indicating whether or not the word classes represented by the two vectors match. The feature amount and the topic repetition number feature amount described above are set as the feature amount representing the number of repetitions of the topic in the dialog, and the dialogue destruction feature amount is a frequent word string feature amount, an inter-utterance similarity feature amount, a sentence length feature. amount, Over emissions number feature quantity, word vector feature quantity, question type feature amount, QCLASS feature amount, out of the topic repetition number feature amount, a combination comprising any one or more feature values.

  According to one embodiment of the present invention, J is an integer equal to or greater than 1 and a j-th feature amount (1 ≦ j ≦ J) is a feature amount indicating whether or not a dialogue has failed. A dialogue destruction feature amount extraction unit that extracts a dialogue destruction feature amount that is a combination of the j-th type feature amount (1 ≦ j ≦ J). The unit includes, for each j that satisfies 1 ≦ j ≦ J, a j-th feature amount calculation unit that calculates the j-th feature amount from the dialogue, and the dialogue act feature amount is a dialogue represented by an utterance included in the dialogue. The features generated from the action, the similarity between utterances, the amount of similarity between the last utterance in the dialogue and the other utterances, and the word feature amount are calculated for each utterance included in the dialogue. The feature amount expressed as a Bag-of-words vector in which word N-grams are arranged and the word class feature amount are calculated for each utterance included in the dialogue. A feature expressed as a Bag-of-classes vector in which word classes corresponding to words are arranged, and a word combination feature are co-occurred between the last utterance in a dialog and other utterances or in the last utterance The feature amount expressed as a Bag-of-words vector whose element is the appearance result of any combination of the word Ngram, word class Ngram, word set, and predicate term structure, Is a feature amount having a character string of a word Ngram appearing at a predetermined frequency or more as an element, and the dialogue destruction feature amount includes a dialogue act feature amount, an utterance similarity feature amount, a word feature amount, a word class feature amount, The combination does not include at least one of the word combination feature amount and the frequent word string feature amount.

  According to one embodiment of the present invention, J is an integer equal to or greater than 1 and a j-th feature amount (1 ≦ j ≦ J) is a feature amount indicating whether or not a dialogue has failed. A dialogue destruction feature amount extraction unit that extracts a dialogue destruction feature amount that is a combination of the j-th type feature amount (1 ≦ j ≦ J). The unit includes, for each j that satisfies 1 ≦ j ≦ J, a j-th feature amount calculating unit that calculates the j-th feature amount from the dialogue, and calculates the similarity feature between utterances in the last utterance in the dialogue. The feature amount that represents how similar the utterance is and the other utterances, the frequent word string feature amount, the feature amount of the word Ngram that appears in the dialog at a predetermined frequency or more as an element, the dialogue act feature amount, The feature amount and sentence length feature amount generated from the dialogue action represented by the utterance included in the dialogue are converted to the word length and sentence of the last utterance in the dialogue. Word Ngram, word class Ngram, word set, predicate term structure that co-occurs a feature representing length and a word combination feature between the last utterance in a dialog and other utterances or in the last utterance , And the dialogue destruction features are the utterance similarity feature, the frequent word string feature, and the dialogue act feature. , Sentence length feature, and word combination feature.

  According to one embodiment of the present invention, J is an integer equal to or greater than 1 and a j-th feature amount (1 ≦ j ≦ J) is a feature amount indicating whether or not a dialogue has failed. A dialogue destruction feature amount extraction unit that extracts a dialogue destruction feature amount that is a combination of the j-th type feature amount (1 ≦ j ≦ J). The unit includes, for each j that satisfies 1 ≦ j ≦ J, a j-th feature amount calculation unit that calculates the j-th feature amount from the dialogue, and the dialogue act feature amount is a dialogue represented by an utterance included in the dialogue. The features generated from the action, the similarity between utterances, the amount of similarity between the last utterance in the dialogue and the other utterances, and the word feature amount are calculated for each utterance included in the dialogue. The feature amount expressed as a Bag-of-words vector in which word N-grams are arranged and the word class feature amount are calculated for each utterance included in the dialogue. The feature amount is expressed as a Bag-of-classes vector in which word classes corresponding to the words are arranged. The dialogue destruction feature amount includes a dialogue act feature amount, an inter-utterance similarity feature amount, and further includes a word feature amount, a word This is a combination that includes at least one of the class feature amounts.

  According to the present invention, it is possible to extract a dialogue destruction feature amount that can be used for estimating a dialogue destructive force indicating a degree at which an utterance breaks a dialogue.

The figure which shows an example of a structure of the dialog destruction model learning apparatus 100. The figure which shows an example of operation | movement of the dialog destruction model learning apparatus 100. The figure which shows an example of a structure of the dialog destructive force estimation apparatus 200. The figure which shows an example of operation | movement of the dialog destructive force estimation apparatus 200. The figure which shows an example of the input / output of the dialog destructive force estimation apparatus 200.

  Hereinafter, embodiments of the present invention will be described in detail. Note that components having the same functions are given the same numbers, and overlapping descriptions are omitted.

<Term>
First, terms used in each embodiment will be briefly described.
The dialogue refers to a series of utterances in the past N (N is an integer of 1 or more). That is, the utterance is a time series of the utterance by the user and the system.

  The dialogue destructive power is the degree to which the utterance breaks down the dialogue. In a series of past N utterances (dialogs), the last utterance breaks the dialogue. Note that the last utterance may be unuttered. The dialogue destructive power may be expressed as a label indicating whether the bankruptcy is not bankrupt, not bankrupt, or neither, or is the probability of bankruptcy, not bankrupt, and the others as random variables. It may be expressed as a distribution. Further, the dialog destructive power may be expressed as a real numerical value indicating the degree of the collapse of the dialog. When expressing the dialog destructive power as a real value, a state in which the dialog has failed, not failed, or neither may be determined by comparing with a threshold value.

<Feature>
There are various factors that affect the failure of the dialogue, and the features associated with each factor also vary. Hereinafter, the feature amount indicating the feature of whether or not the dialogue has failed will be described. At this time, in addition to the definition of each feature, a description is also given of the viewpoint from which each feature affects the dialog destructive power. Note that, in general, the feature amount is extracted from some utterances included in the dialogue, and is expressed as a vector.

  Hereinafter, specific examples may be given in the description of each feature amount, but these are merely examples, and other vector expressions may be used.

[Topic unity]
In the current dialog system, utterance of a topic unrelated to the utterance of the user is generated, and the dialog may be broken. Conversely, by sticking to one topic and repeating the utterance of the same content or topic many times, the dialogue may be broken.

  Therefore, by measuring the appearance frequency of the topic transition pattern, the closeness of the user's utterance topic and the utterance topic of the system, the number of repetitions of the topic, etc., it becomes possible to estimate the dialog destructive power considering the cohesion of the topic. . Hereinafter, the three feature amounts of the word combination feature amount, the inter-utterance similarity feature amount, and the topic repetition number feature amount will be described.

(1-1) Word combination feature amount The word combination feature amount is a word Ngram, word class Ngram, word set co-occurring between the last utterance in a dialog and another utterance or in the last utterance. , A feature amount expressed as a Bag-of-words vector having an appearance result of any combination of the predicate-argument structures (hereinafter referred to as a word combination) as an element.

  The word Ngram, word class Ngram, word set, and predicate term structure can be obtained by performing morphological analysis to divide the utterance included in the dialog into words. The utterances to be subjected to the morphological analysis are the M utterances immediately before and including the last utterance to be the target of estimating the dialog destructive power. In addition, M is preferably about 1 to 4.

  N of the word Ngram and the word class Ngram is preferably about 1 to 4. A word class is a set of word vectors obtained as a result of clustering word vector expressions obtained using word2vec (details will be described later), or a word assigned in a dictionary such as the Japanese vocabulary system. May be represented by abstraction. For example, the word class of a car is represented as a vehicle / artificial object. Also, unlike an Ngram that considers the order, the word set may be separated from words included in a sentence (for example, the immediately preceding M utterances).

  Usually, the number of such word combinations obtained is extremely large, which hinders efficient model learning. Therefore, the number of word combinations used to define the word combination features is increased to the top K (K is about tens to tens of thousands) using the number of appearances in the corpus consisting of the dialogues to be learned and the TF-IDF value. It should be limited. Further, the range of possible word combinations may be limited to those having a dependency relation in each utterance and co-occurrence only in the predicate term structure. Furthermore, words to be considered may be limited to only content words, and words that do not relate to the topic, such as particles and punctuation, may be excluded. In addition, the words to be considered may be limited to only predicates having fewer types than nouns. In this way, it is possible to robustly estimate the diversity of nouns.

  As described above, by using the word combination feature amount, it becomes possible to capture a failure pattern peculiar to a certain dialogue system, and conversely, a general transition pattern of a topic that can occur.

An example of the word combination feature amount will be described using Dialog Example 1 below.
(Interaction example 1)
1 user: Hello /. / Travel / is / like / is / does /?
2 System: Yes /. / The other day / Kyoto / To / Go / Most / Ta /.
Here, the symbol “/” indicates a word delimiter.
For example, in an interactive Example 1, from the user utterance "Hello", ".", "Travel", "may", "love", "is", "do", "?" Is eight words of the obtained, "Yes, from the system utterance If the nine words “,” “.”, “The other day”, “Kyoto”, “ni”, “go”, “masashi”, “ta”, and “.” Are obtained, the combination of the word Ngram (N = 1) between the system utterance "Yes - Hello""Yes-." - "? .-""Yestravel" ... a combination of ways 9x8 = 72 is obtained, and in the system utterance _{9 C 2 = 9x8 / 2 =} 36 The following combinations are obtained. That is, 72 combinations of word Ngrams co-occurring between the last utterance in the dialogue and other utterances are obtained, and 36 combinations of word Ngrams co-occurring in the last utterance are obtained.

  In this manner, all combinations appearing in the training data used for learning are listed, and a vector corresponding to each dimensional element is formed for each combination. The value of the element of each dimension of the vector is set to 1 if a certain combination appears, and 0 if it does not appear. For example, when the number of dimensions of the vector configured as described above is 5, and only a combination corresponding to the element of the second dimension appears, the obtained word combination feature amount is (0,1,0,0,0) It becomes.

(1-2) Inter-utterance similarity feature amount The inter-utterance similarity feature amount is a feature amount indicating how similar the last utterance in a dialog is to other utterances. Is a vector in which one or more similarities are arranged as elements. Each similarity used here measures the degree of similarity between utterances.

  Unlike the word combination feature that captures a specific transition pattern, the inter-utterance similarity feature can evaluate the relevance between utterances based on the similarity, even for a combination that does not appear during a dialogue.

  If the inter-utterance similarity feature amount is calculated between the immediately preceding user's utterance and the last utterance, a discontinuous topic transition can be detected. When the inter-utterance similarity feature amount is calculated between the utterance by the immediately preceding system and the last utterance, persistence on a specific topic can be detected.

  The similarity can be calculated using the word cosine distance, the average vector distance of word2vec (Reference Non-Patent Document 2), and the WordMoversDistance distance (Reference Non-Patent Document 3). Further, a combination of words (word Ngram in the case of a BLEU score and a ROUGE score), such as a BLEU score and a ROUGE score (reference non-patent document 4), may be used. In other words, the degree of similarity can be said to be the distance between utterances calculated based on the distance between words or the co-occurrence relationship between words.

word2vec is a method of converting words into semantic vectors, and the meaning vectors corresponding to each word are calculated so that words that co-occur in the corpus (here, dialogue) are similar so that the distance between the vectors is short. Is done. This makes the average vector-to-vector distance of word2vec more robust than word cosine distance to fluctuations and small differences in notations such as cats and cats and cats and kittens.
(Reference non-patent document 2: T. Mikolov, K. Chen, G. Corrado, J. Dean, “Efficient Estimation of Word Representations in Vector Space”, arXiv preprint arXiv: 1301.3781, 2013.)

WordMoversDistance examines the distance d (w, v) between a word w contained in a certain sentence S and a word v contained in another sentence S ′ from the sentence S, and finds the closest distance d ′ (w). It is calculated and the sum Σd '(w) of all the words of the sentence S is obtained. WordMoversDistance can evaluate the similarity of individual words in more detail than the average vector-to-vector distance of word2vec.
(Reference non-patent document 3: MJ Kusner, Y. Sun, NI Kolkin, KQ Weinberger, “From Word Embeddings To Document Distances”, Proceedings of the 32nd International Conference on Machine Learning (ICML-15), pp.957-966, 2015.)

The BLEU score and the ROUGE score are used in machine translation or the like to calculate the distance between two sentences using the matching rate of the word Ngram.
(Non-Patent Document 4: Tsutomu Hirao, Hideki Isozaki, Katsuhito Sudo, Duh Kevin, Moto Tsukada, Masaaki Nagata, "Automatic Evaluation Method of Machine Translation Based on Word Order Correlation", Natural Language Processing, vol.21, no.3 , pp.421-444, 2014.)

Instead of calculating the similarity using the words as they are, using a dictionary such as Japanese Vocabulary Taiyo (Reference Non-Patent Document 5), the words are abstracted into word classes, and the similarity is calculated. May be.
(Reference non-patent document 5: Satoru Ikehara, Masahiro Miyazaki, Satoshi Shirai, Akio Yokoo, Hiromi Nakaiwa, Kentaro Ogura, Yoshifumi Oyama, Yoshihiko Hayashi, "Japanese vocabulary system", Iwanami Shoten, 1997.)

  Furthermore, words to be considered when calculating the similarity may be limited to content words.

An example of the inter-utterance similarity feature amount will be described using Dialog Example 2 below.
(Interaction example 2)
1 user: Hello /. / Travel / is / like / is / does /?
2 System: Yes /. / The other day / Kyoto / To / Go / Most / Ta /.
For example, in Dialogue Example 2, consider the average vector-to-vector distance of word2vec limited to content words (nouns, verbs, adjectives, independent words, etc.). From the user utterance, "Hello", "travel" three content words "like" is obtained, the system is from the speech of four content words "YES" and "the other day", "Kyoto", "go" is obtained. Convert the word obtained using word2vec to a vector. For example, as a three-dimensional vector, "Hello" is (0.1,0.7,0.2), "travel" is (0.8,0.1,0.1), "like" the (0.3,0.4,0.3), 'Yes' (0.2 , 0.6, 0.2), "the other day" is (0.1, 0.1, 0.8), "Kyoto" is (0.6, 0.3, 0.1), and "going" is (0.7, 0.2, 0.1). At this time, the average vector of the user utterance is ((0.1, 0.7, 0.2) + (0.8, 0.1, 0.1) + (0.3, 0.4, 0.3)) / 3 = (0.4, 0.4, 0.2), and the average vector of the system utterance Is ((0.2,0.6,0.2) + (0.1,0.1,0.8) + (0.6,0.3,0.1) + (0.7,0.2,0.1)) / 4 = (0.4,0.3,0.3). By calculating the cosine similarity ($ 0.97) or the like, the similarity between the user utterance 1 and the system utterance 2 can be obtained. A vector (0.97,...) Obtained by arranging one or more similarities obtained in this manner is the inter-utterance similarity feature amount.

(1-3) Topic repetition count feature The topic repetition count feature is a feature representing the number of repetitions of a topic in a dialogue. Here, the topic refers to a focused word and a focused predicate-argument structure.

  Consecutive utterances of a particular topic are generally unnatural behaviors, and may cause the user to feel uncomfortable or reduce the user's willingness to interact, and as a result, the dialogue may fail. Many. Therefore, by examining the number of repetitions of a certain topic, it is possible to detect the breakdown of the conversation.

An example of the topic repetition number feature amount will be described using Dialog Example 3 below.
(Interaction example 3)
1 system: Hello. Watch out for heat stroke.
2 Users: Yes. Thank you. Be careful too.
3 System: Do you be careful about heat stroke?
4 Users: I drink a little water and watch out.
5 System: Do you want to be careful about heat stroke?
In the case of Dialogue Example 3, the system repeatedly utters the word “heat stroke” and the predicate-argument structure of “watch out for heat stroke”. At this time, the topic repetition number feature amount in the last system utterance of 5 is calculated as 3.

[Connection of dialogue act]
The dialogue act feature is a feature generated from a dialogue act represented by an utterance included in the dialogue. Here, the dialogue act refers to a user's utterance intention such as a question, a greeting, a self-disclosure, a praise, an apology, etc. The dialogue action can be represented as a Bag-of-words vector as described later.
(Reference Non-Patent Document 6: T. Meguro, Y. Minami, R. Higashinaka, K. Dohsaka, “Controlling listening-oriented dialogue using partially observable Markov decision processes”, Proceedings of the 23rd international conference on computational linguistics. Association for Computational Linguistics (COLING 10), pp.761-769, 2010.)

  The dialogue act feature includes a dialogue act sequence feature and a predictive dialogue act feature described below.

(2-1) Dialogue act sequence feature amount The dialogue act sequence feature amount is a feature expressed as a vector having elements of a result of estimating a dialogue act represented by each utterance included in the dialogue (hereinafter referred to as an estimated dialogue act). Quantity. The estimation result (estimated dialogue action) can be represented as a Bag-of-words vector. Specifically, the Bag-of-words vector corresponding to the estimated dialogue action of each utterance is a 1-of-K vector in which the value of 1best dialogue action is 1 and the values of other dialogue actions are 0. Or a probability distribution (probability distribution vector) representing the estimated dialogue act. The utterance including the last utterance for estimating the dialogue action is M utterances immediately before including the last utterance. In addition, M is preferably about 1 to 4.

  For example, if the dialogue actions to be estimated are five questions, greeting, self-disclosure, praise, and apology, and the dialogue action sequence is generated from the four immediately preceding utterances including the last utterance, 1best of the dialogue actions of the four utterances is When “greeting ⇒ greeting ⇒ self-disclosure ⇒ praise”, the dialogue action sequence feature quantity, which is a vector of the Bag-of-words vector, is ((0,1,0,0,0), (0,1,0 , 0,0), (0,0,1,0,0), (0,0,0,1,0)).

  To generate a Bag-of-words vector representing an estimated dialogue action of each utterance, an SVM (Support Vector Machine) using words as feature amounts is used. In addition, it is necessary for a person to learn SVM in advance using a dialog database to which a dialog act corresponding to an utterance is given in advance.

(2-2) Predictive dialogue act feature amount The predictive dialogue act feature amount is a prediction result vector representing a result of predicting a dialogue action that the last utterance should have from an utterance included in the dialogue (hereinafter, referred to as a predictive dialogue action), A vector in which the prediction result vector and the estimation result vector representing the result of estimating the dialogue action represented by the last utterance are arranged, the difference vector between the prediction result vector and the estimation result vector, and whether the 1best of the prediction result vector and the estimation result vector are the same. This is a feature quantity composed of one or more of the true / false values of no. A match between the prediction result vector and 1best of the estimation result vector means that the dimension of the largest element among the elements of each vector matches.

  The prediction result (prediction dialogue action) can be represented as a Bag-of-words vector, similarly to the estimation result of (2-1). Specifically, the Bag-of-words vector corresponding to the predicted dialogue act of the last utterance is a 1-of-K vector in which the value of the dialogue act of 1best is 1 and the values of the other dialogue acts are 0. Or a probability distribution (probability distribution vector) representing the likelihood of a predicted dialogue action.

  A Bag-of-words vector (prediction result vector) representing the predicted dialogue action of the last utterance is generated using SVM or POMDP (Partially Observable Markov Decision Process) that features the dialogue action of a word or the immediately preceding utterance. (Reference non-patent document 6). It is necessary for a person to learn SVM and POMDP in advance by using a dialog database to which a dialog act corresponding to an utterance has been given.

  For example, if there are five dialogue actions: question, greeting, self-disclosure, praise, and apology, and if a “question” is predicted as a dialogue action from the last utterance, a Bag-of-words vector and a final The vector in which the Bag-of-words vectors representing the results of estimating the dialogue action represented by the utterance of the utterance are ((1,0,0,0,0), (0,0,0,1,0)) . The difference vectors are (1,0,0, -1,0), and the true / false value of the match is false (0). For example, assuming that a vector obtained by combining these vectors is a predictive dialogue action feature, the vector (1,0,0,0,0,0,0,0,1,0,1,0,0, -1,0 , 0) is obtained as the predictive dialogue action feature quantity.

(2-3) Character string co-occurrence feature A character string co-occurrence feature is a character string Ngram (where N is 3 or more) that co-occurs between the last utterance in a dialog and other utterances. ) Is a feature quantity expressed as a Bag-of-words vector having the appearance result of the combination of
Since the ending character string often represents a dialogue act, the co-occurrence relationship of the dialogue act can be grasped by looking at their co-occurrence.

An example of a character string co-occurrence feature amount will be described using Dialog Example 4 below.
(Interaction example 4)
1 User: Where did you come from?
2 System: Which are you interested in, Forest Adventure or Takeda Castle Ruins?
For example, if a character string Ngram is extracted with N = 3, “somewhere”, “from here”, “from”, “Suka?” Can be obtained from the user's utterance, and “Fore”, “Oresu”,. Do you? " Here, if the co-occurrence is focused particularly on the endings, the combination "Suka? -Suka?" Is obtained.

  In this manner, all combinations appearing in the training data used for learning are listed, and a vector corresponding to each dimensional element is formed for each combination. The value of the element of each dimension of the vector is set to 1 if a certain combination appears, and 0 if it does not appear. For example, when the number of dimensions of the vector configured as described above is 5, and only a combination corresponding to the element of the second dimension appears, the obtained character string co-occurrence feature amount is (0, 1, 0, 0, 0).

[Logical connection]
(3-1) Question type feature quantity The question type feature quantity is a feature quantity representing a question type estimated when the utterance immediately before the last utterance in the dialogue is a question. Examples of question types include personality questions asking specific tastes and experiences of speakers, factoid questions asking specific things, and questions asking 5W1H of certain events (such as news). Further, a question type may be defined in a form linked to a topic, such as “restaurant place”.

  For estimating the question type, an SVM using a word as a feature amount is used. In addition, it is necessary for a person to learn SVM in advance using a database in which question types are classified in advance.

  Some dialog systems cannot respond to some of the above question types (they are not good at responding). Therefore, using question type features makes it possible to estimate dialog destructive power reflecting such system characteristics.

  For example, an interactive system that provides weather guidance can answer questions about the weather at a specific place, but cannot answer questions about the tourist information of the place or the personality of the system itself. Therefore, the question type is defined as two types of “weather-related questions” and “other questions”, and the question from the user is estimated as one of the question types, and 1-of as in (1,0). The question type feature amount is obtained by vectorizing using the -K expression.

(3-2) Question class feature quantity The question class feature quantity is a vector representing a word class that is presumed to require an answer when the utterance immediately before the last utterance in the dialogue is a question, From one or more of the vectors representing the word classes included in the answer (the last utterance), their difference vectors, and boolean values indicating whether the word classes represented by these two vectors match or not. Characteristic amount. A vector representing the estimated word class and a vector representing the word class included in the answer can be represented as a probability distribution vector or a 1-of-K vector.

  By using ENE (extended named entity) extraction technology to check whether the estimated word class (that is, the word class represented by the question class feature) is included in the last utterance, the correspondence between the question and the answer Can examine relationships.

An example of the question class feature amount will be described using Dialog Example 5 below.
(Example 5 of dialogue)
1 User: Where did you come from?
2 System: From Kyoto For example, in Dialogue Example 5, it is estimated that the word class requested by the user utterance for the answer is “place”, and the system utterance includes the “place” word class. Consider the case where it is estimated that Given a set of word classes as unique, place, and quantity, the 1-of-K vector obtained from the user's utterance (that is, the vector representing the word class that is presumed to require the answer to the question) is (0 , 1,0), and the 1-of-K vector obtained from the system utterance (that is, the vector representing the word class included in the answer (the last utterance)) is (0,1,0). For example, if a vector obtained by combining these vectors is used as a question class feature, a vector (0,1,0,0,1,0) is obtained as a question class feature.

  A technique called question classification is often used for estimating a word class from a user utterance, and a technique called ENE extraction is often used for estimating a word class from a system utterance.

[Suitability of the utterance itself]
(4-1) Perplexity feature amount The perplexity feature amount is a feature amount representing perplexity calculated using a language model for each utterance included in a dialog. Perplexity expresses the naturalness of a series of words, and can detect grammatically unnatural utterances. As the language model, a model using a word Ngram or a character Ngram (N is often about 1 to 7) and a model using a Recurrent Neural Network are known. Any language model that can calculate perplexity may be used. The perplexity feature amount may be a method in which the perplexity value itself is directly used as the feature amount, or a 1-of-K vector quantized to an appropriate number may be used as the feature amount.

  In addition, instead of the perplexity calculated as described above, perplexity was normalized by the appearance probability of each word in order to express with emphasis on sentence fluency regardless of the appearance probability of the word itself. A value (a value obtained by dividing perplexity by the probability of occurrence of a word, hereinafter, referred to as normalized perplexity) may be used.

  For example, while the perplexity of the system utterance "Where are you from?" And the perplexity of the system utterance "Where are you from?" Are you from? ”Is a more fluent expression, and it is expected that perplexity will decrease.

(4-2) Word feature quantity The word feature quantity is a feature quantity expressed as a Bag-of-words vector in which words N-grams (N is about 1 to 5) of each utterance included in the dialog are arranged. .
By using the word feature amount, it becomes possible to capture an error pattern that is easily output by a certain interactive system.

  The word used for the word feature amount may be cut off to the top N words using the number of occurrences in the dialog or the TF-IDF value. Further, the words to be considered may be limited to only the content words, and words not related to the topic such as particles may be excluded. In addition, the words to be considered may be limited to only predicates having fewer types than nouns. In this way, it is possible to robustly estimate the diversity of nouns.

An example of the word feature amount will be described using Dialog Example 6 below.
(Example 6 of dialogue)
1 user: Hello /. / Travel / is / like / is / does /?
2 System: Yes /. / The other day / Kyoto / To / Go / Most / Ta /.
For example, in an interactive Example 6, from the user utterance, "Hello", ".", "Travel", "may,""love,""is,""or""?" From the eight words is obtained, the system utterance of " If nine words of "Yes", ".", "The other day", "Kyoto", "Nii", "Go", "Mashi", "Ta" and "." Are obtained, Bag-of-words of the word Ngram (N = 1) vector, from the user utterance "Hello", ".", "travel", "may,""love,""is,""or""?" dimension of the elements 1 corresponding to, vector word feature other than that it becomes 0 Obtained as a quantity. On the other hand, from the system utterance, a vector whose dimension element corresponding to "Yes", ".", "The other day", "Kyoto", "Nii", "Go", "Better", "Ta", and "." It is obtained as a word feature.

(4-3) Word Class Feature The word class feature is a feature expressed as a Bag-of-classes vector in which word classes corresponding to the words of each utterance included in the dialog are arranged. The word class represents the general meaning of the word.

  The word class construction method is a dictionary-based method using class information given to dictionaries such as Wordnet or Japanese vocabulary, a method of clustering Word2vec vectors with K-means and generating a set of words, etc. There is.

  The word class used for the word class feature may be cut off to the top N using the number of occurrences in the dialogue and the TF-IDF value.

An example of the word class feature amount will be described using the following interactive example 7.
(Example 7 of dialogue)
1 User: Where did you come from?
2 System: From Kyoto For example, in Dialogue Example 7, consider the case where the word classes are limited to person names, places, and amounts. At this time, since the user utterance does not include a word that can be converted to a word class, (0,0,0) is obtained as a word class feature amount. On the other hand, since it is estimated that the system utterance includes the word class of “place”, (0,1,0) is obtained as the word class feature amount.

(4-4) Word Vector Feature The word vector feature is a feature expressed as a vector generated from a vector representing a word N-gram (N is about 1 to 5) of each utterance included in the dialogue. is there. For example, it can be generated using a weighted average or multiplication for each element.

  When the weighted average is used, the word vector feature amount is a feature amount represented by a vector configured as a weighted average of a vector representing the word N-gram of each utterance included in the dialogue.

  The vector representing the word N-gram may be extracted by, for example, Word2vec. As the weight of the weighted average, a TF-IDF value may be used, or all may be equal.

  Alternatively, the number of vectors representing the word N-gram used for calculating the weighted average may be truncated using the TF-IDF value, and only the top M words may be used.

  When the multiplication for each element of the vector is used, the word vector feature amount is a feature amount represented by a vector configured by multiplying each element of the vector representing the word N-gram of each utterance included in the dialogue.

An example of the word vector feature amount will be described using Dialog Example 8 below.
(Example 8 of dialogue)
1 user: Hello /. / Travel / is / like / is / does /?
2 System: Yes /. / The other day / Kyoto / To / Go / Most / Ta /.
For example, in Dialogue Example 8, an average vector of word2vec limited to content words (noun, verb, adjective, independent word, etc.) is considered as a word vector feature. From the user utterance, "Hello" three content words "travel", "love" is obtained, from the system utterance, four of content words "YES" and "the other day", "Kyoto", "go" is obtained . Convert the word obtained using word2vec to a vector. For example, as a three-dimensional vector, "Hello" is (0.1,0.7,0.2), "travel" is (0.8,0.1,0.1), "like" the (0.3,0.4,0.3), 'Yes' (0.2 , 0.6, 0.2), "the other day" is (0.1, 0.1, 0.8), "Kyoto" is (0.6, 0.3, 0.1), and "going" is (0.7, 0.2, 0.1). At this time, the average vector (word vector feature amount) of the user utterance is ((0.1, 0.7, 0.2) + (0.8, 0.1, 0.1) + (0.3, 0.4, 0.3)) / 3 = (0.4, 0.4, 0.2) ), The average vector (word vector feature) of the system utterance is ((0.2, 0.6, 0.2) + (0.1, 0.1, 0.8) + (0.6, 0.3, 0.1) + (0.7, 0.2, 0.1)) / 4 = (0.4,0.3,0.3).

(4-5) Sentence length feature amount The sentence length feature amount is a feature amount representing the word length and character length of the last utterance in a dialogue.
In the current interactive system, it is difficult to estimate the consistency between the contents of the user utterance and the contents of the system utterance without error. Therefore, there is a problem that the longer the system utterance is, the more likely that an extraneous part is included. In order to reflect this in the estimation of the dialog destructive power, a sentence length feature can be used.
For example, when the last utterance in the dialogue is “shopping is fun,” the word length is 7, the character length is 13, and the sentence length feature amount is (7, 13).

(4-6) Number-of-turns feature amount The number-of-turns feature amount is a feature amount representing the number of turns that have elapsed since the start of the dialogue.
This is because both dialog systems generate relatively appropriate utterances at the beginning of the dialogue, but the proportion of inappropriate utterances tends to increase as the dialogue progresses. belongs to.
For example, in the case of the dialog shown in the input of FIG. 5, the number of turns is 5, and therefore, the number-of-turns feature amount is (5).

[In scenario]
(5-1) Frequent Word String Feature Amount The frequent word string feature is a feature having a character string of a word Ngram appearing at a predetermined frequency T or more in a dialog as an element. Here, N is preferably about 4 to 7, and T is preferably 10 or more.

  This feature value takes into account a case where a dialogue system is constructed by inserting a system utterance that leads to a scenario assumed in advance, such as which one of sentence A and sentence B is uttered. Immediately after leading to a scenario, the system can respond in a relatively disconnected manner from the context so far, so it is easy to generate an appropriate response. Immediately after inducing the scenario, it is assumed that the evaluation tendency differs from the other parts. Therefore, a frequent word string feature is used as a feature for estimating whether or not it is immediately after a scenario is guided.

  For example, if the word “2 gram” “shopping” appears in the dialogue at a predetermined frequency of 3 or more, the frequent word string feature amount includes “shopping” as an element, and is a vector such as (“shopping”). Is expressed.

<First embodiment>
[Dialogue destruction model learning device 100]
Hereinafter, the dialogue destruction model learning device 100 will be described with reference to FIGS. As shown in FIG. 1, the dialogue destruction model learning device 100 includes a dialogue destruction feature amount extraction unit 110, a model generation unit 120, and a recording unit 190. The recording unit 190 is a component that appropriately records information necessary for the processing of the dialogue destruction model learning device 100. For example, a dialogue destruction model during learning (dialogue destruction model parameter) is recorded.

Also, interactive fracture feature amount extraction unit 110, the one feature quantity calculating unit 110 1, _..., including a first J species feature quantity calculator 110 _J (although, J is an integer of 1 or more). The j-th feature amount calculation unit 110 _j (1 ≦ j ≦ J) calculates the j-th feature amount from a dialog. The j-th feature amount is any of the feature amounts described in <feature amount>.

  Prior to the start of learning, a plurality of training data, which are a set of a dialog made up of a series of utterances and correct data indicating whether or not the dialog has failed, are prepared. The correct answer data indicating whether or not the dialogue has failed may be a label indicating whether the dialogue has failed, has not failed, or has neither, or has failed, has not failed, or has not. The probability distribution may be a random variable. Further, the correct answer data may be a real numerical value indicating the degree of failure. As the training data, dialogue failure data to which a label indicating the failure of the dialogue or a probability distribution is manually added in advance in Reference Non-Patent Document 1 can be used.

  The dialogue destruction model learning device 100 learns a dialogue destruction model from a set of a dialogue and correct answer data which are training data. The dialogue destruction model is used for estimating a dialogue destructive power indicating a degree of whether or not a dialogue has failed.

  The operation of the dialog destructive model learning device 100 will be described with reference to FIG. The dialogue destruction feature extraction unit 110 extracts a dialogue destruction feature that indicates whether or not the dialogue is broken by the last utterance from the input dialogue (S110). The dialog destruction feature is a combination of the first type feature,..., The J type feature. The following is an example of the dialogue destruction feature amount which is a combination of the feature amounts.

1) One of the frequent word string feature, inter-utterance similarity feature, sentence length feature, turn number feature, word vector feature, question type feature, question class feature, and topic repetition feature A combination that includes one or more features.
The inter-utterance similarity feature amount uses a similarity other than the word cosine distance as a vector element.

2) A combination that includes a dialogue act feature, an inter-utterance similarity feature, and does not include at least one of a word feature, a word class feature, a word combination feature, and a frequent word string feature.
This combination does not include at least one of the four feature amounts of the word feature amount, the word class feature amount, the word combination feature amount, and the frequent word string feature amount. For example, a combination including three features of a word feature, a word class feature, and a word combination feature in addition to the dialogue feature and the inter-utterance similarity feature is appropriate as the dialogue destruction feature. A combination including all four of the word feature, the word class feature, the word combination feature, and the frequent word string feature in addition to the feature and the inter-utterance similarity feature is not appropriate. Therefore, by using this combination, it is possible to effectively estimate the failure due to the unnatural transition of the dialogue action and the failure due to the rapid transition of the topic while suppressing the number of feature values.

3) Includes dialogue action features, inter-utterance similarity features, and sentence length features, and calculates at least one of the word features, word class features, word combination features, and frequent word string features. Not including combinations.
Using this combination, it is possible to effectively estimate failures due to unnatural transitions in dialogue actions, failures due to sudden transitions in topics, and failures due to the inclusion of irrelevant topics due to too long utterances, while suppressing the number of feature values. Will be possible. In other words, a combination having more preferable characteristics than the case 2) is obtained.

4) Including dialogue action features, inter-utterance similarity features, sentence length features, and turn number features, and at least one of word features, word class features, word combination features, and frequent word string features A combination that does not include one feature.
When this combination is used, it reflects the difficulty of failure at the beginning of the dialogue and suppresses the number of features, while the failure due to an unnatural transition of the dialogue action, the failure due to a sudden transition of the topic, and the irrelevant topic due to an excessively long utterance It is possible to effectively estimate a failure due to the incorporation of. In other words, a combination having characteristics more preferable than 3) is obtained.

5) Combinations that include dialogue action features, inter-utterance similarity features, sentence length features, and turn number features, but do not include word features, word class features, word combination features, and frequent word string features .
This combination further restricts the combination of 4).

6) Includes dialogue act features, inter-utterance similarity features, sentence length features, turn number features, and character string co-occurrence features, while word features, word class features, word combination features, and frequent words Combinations that do not include column features.
This combination further restricts the combination of 5).

7) Including dialogue act features, inter-utterance similarity features, sentence length features, turn number features, perplexity features, word features, word class features, word combination features, frequent word string features Combination that does not include at least one of the feature quantities of this quantity When this combination is used, it reflects the difficulty of failure at the start of the dialogue and the naturalness of the syntax of the utterance itself and suppresses the number of feature quantities, It is possible to effectively estimate a failure due to an unnatural transition, a failure due to a sudden transition of a topic, or a failure due to mixing of an unrelated topic due to an utterance that is too long. In other words, the combination having characteristics more preferable than 4) is obtained.

8) Combinations including inter-utterance similarity features, frequent word string features, dialogue act features, sentence length features, and word combination features.
This combination uses frequently occurring word string features to capture the unique behavior of a dialog system that operates based on a scenario, and word combination features to capture efficient topic transitions.

9) A combination that includes a dialogue act feature, an inter-utterance similarity feature, and further includes at least one of a word feature and a word class feature.
By including at least one of the word feature amount and the word class feature amount as the feature amount, this combination can further improve the performance when the data amount can be sufficiently secured.

  The model generation unit 120 generates a dialogue destruction model using the dialogue destruction feature amount extracted in S110 and the input correct data (S120). Here, any algorithm may be used as the learning algorithm of the dialogue destruction model. For example, a Deep Neural Network (DNN), SVM, ExtraTreesClassifier can be used.

  The dialogue destruction force estimating devices configured using the dialogue destruction models obtained by these learning algorithms have the following features, respectively. The use of DNN is superior in that it can automatically take into account the combination of features (in fact, it shows good results experimentally), but the behavior is small when the amount of training data is small. There is a problem that it is not stable. When the SVM is used, although the peak value of the estimation accuracy is inferior to DNN, it is possible to efficiently learn even a small amount of training data. When ExtraTreesClassifier is used, estimation can be performed in consideration of the combination of features even when the amount of training data is small.However, when the amount of features increases, model learning takes time and the accuracy of estimation decreases. .

  When the correct answer data is given as a probability distribution (that is, when the dialogue destruction force estimating device estimates the failure of the dialogue as a probability distribution), the Support Vector Regressor is used instead of the SVM, and the ExtraTreesRegressor is used instead of the ExtraTreesClassifier. Good.

  The generated dialogue destruction model is fed back and used for learning using the next training data.

  The model generation unit 120 is a component that executes a calculation based on a learning algorithm. Therefore, the dialogue destruction model learning device 100 sets the initial value of the dialogue destruction model recorded in the recording unit 190 in the model generation unit 120 before learning starts. Further, the dialogue destruction model learning apparatus 100 sets the generated dialogue destruction model in the model generation unit 120 each time the model generation unit 120 generates the dialogue destruction model during learning.

  The dialogue destruction model learning apparatus 100 repeats the processing of S110 to S120 by the number of training data, and outputs a finally generated dialogue destruction model as a learning result.

  Note that the dialogue destruction feature amount extraction unit 110 that extracts the dialogue destruction feature amount may be handled as an independent device (hereinafter, a dialogue destruction feature amount extraction device) instead of as a part of the dialogue destruction model learning device 100. In this case, the dialogue destruction feature amount extraction device includes a dialogue destruction feature amount extraction unit 110 and a recording unit 190. The dialogue destruction feature value extraction device extracts and outputs a dialogue destruction feature value which is a combination of feature values indicating characteristics of whether or not the dialogue has failed, using the dialogue as an input.

[Interaction destructive force estimation device 200]
Hereinafter, the dialog destructive force estimation device 200 will be described with reference to FIGS. As shown in FIG. 3, the dialogue destruction force estimation device 200 includes a dialogue destruction feature amount extraction unit 110 and a dialogue destruction force calculation unit 220.

  Further, the dialog destructive force estimation device 200 is connected to the learning result recording unit 290. The learning result recording unit 290 records the dialogue destruction model learned by the dialogue destruction model learning device 100. The learning result recording unit 290 may be a component included in the dialog destructive force estimation device 200.

  The dialog destructive force estimating apparatus 200 estimates a dialog destructive force that is the degree to which the last utterance of the dialog breaks the dialog, from the dialog that is a series of utterances. FIG. 5 shows an example of input and output of the interactive destructive force estimation device 200. A series of utterances (from utterances by the system "shopping is easy for one person" to utterances by the system "shopping is fun together") is input. The utterance made by the system "shopping is fun together" is the last utterance and is the target for estimating the destructive power of the dialogue. In addition, the dialog destructive power is estimated as a probability distribution of not failing (）), failing (x), or neither (△). Here, since the probability value is the largest (that is, 1best is ×), it is considered that the dialogue is broken by the last utterance.

  The dialog destructive force estimation device 200 sets the dialog destructive model recorded in the learning result recording unit 290 in the dialog destructive force calculation unit 220 before the estimation starts.

  The operation of the dialog destructive force estimation device 200 will be described with reference to FIG. The dialogue destruction feature extraction unit 110 extracts a dialogue destruction feature that indicates whether or not the dialogue is broken by the last utterance from the input dialogue (S110). The last utterance may be an utterance of the user or an utterance by the system.

  The dialog destructive power calculation unit 220 calculates a dialog destructive power that is such that the last utterance breaks the dialog from the dialog destructive feature amount extracted in S110 (S220). At that time, the dialog destruction model learned by the dialog destruction model learning device 100 is used. The dialogue destructive power as the estimation result is calculated as one of a label, a probability distribution, and a real value according to the generated dialogue destruction model.

  According to the invention of this embodiment, the dialogue destruction feature amount can be calculated as a combination of feature amounts based on various factors that cause the dialogue to fail. Further, by setting the dialogue destruction feature amount in consideration of the feature of each feature amount used for the combination, it becomes possible to learn the dialogue destruction model from a small amount of training data.

  By estimating the dialog destructive power for the utterance by the system, it is possible to suppress the output of an utterance having a high dialog destructive power. As a result, the continuation of the dialogue becomes easy.

  In addition, by estimating the dialog destructive power for a dialog between an actual person and the system, it is possible to find a place where the dialog is likely to be broken, which can be used for improving the system.

  Further, in speech dialogue, since it is difficult to detect a speech recognition error only from the acoustic feature of the utterance by the user, the system may generate an utterance from the utterance including the recognition error. In such a case, by estimating the naturalness of the interactive connection between the speech recognition result of the user utterance and the immediately preceding system utterance as the dialog destructive power of the user utterance, detection of a recognition error and retrieval of speech recognition candidates are performed. Ranking becomes possible, and smooth voice dialogue becomes possible.

<Modification>
The present invention is not limited to the above embodiment, and it goes without saying that changes can be made as appropriate without departing from the spirit of the present invention. The various processes described in the above embodiment may be performed not only in chronological order according to the order described, but also in parallel or individually according to the processing capability of the device that executes the process or as necessary.

<Supplementary note>
The device of the present invention is, for example, a single hardware entity, an input unit to which a keyboard or the like can be connected, an output unit to which a liquid crystal display or the like can be connected, a communication device (for example, a communication cable) that can communicate outside the hardware entity. , A communication unit, a CPU (which may include a central processing unit, a cache memory and a register, etc.), a RAM and a ROM as a memory, an external storage device as a hard disk, and an input unit, an output unit, and a communication unit thereof. , A CPU, a RAM, a ROM, and a bus connected so that data can be exchanged between the external storage devices. If necessary, the hardware entity may be provided with a device (drive) that can read and write a recording medium such as a CD-ROM. A physical entity provided with such hardware resources includes a general-purpose computer.

  The external storage device of the hardware entity stores a program necessary for realizing the above-described functions, data necessary for processing the program, and the like. It may be stored in a ROM that is a dedicated storage device). Data obtained by the processing of these programs is appropriately stored in a RAM, an external storage device, or the like.

  In the hardware entity, each program stored in the external storage device (or ROM or the like) and data necessary for processing of each program are read into the memory as needed, and interpreted and executed / processed by the CPU as appropriate. . As a result, the CPU realizes a predetermined function (each of the components described as the above-mentioned... Section,... Means).

  The present invention is not limited to the above-described embodiment, and can be appropriately modified without departing from the spirit of the present invention. Further, the processes described in the above embodiments may be executed not only in chronological order according to the order of description, but also in parallel or individually according to the processing capability of the device that executes the processes or as necessary. .

  As described above, when the processing function of the hardware entity (the device of the present invention) described in the above embodiment is implemented by a computer, the processing content of the function that the hardware entity should have is described by a program. Then, by executing this program on a computer, the processing functions of the hardware entities are realized on the computer.

  A program describing this processing content can be recorded on a computer-readable recording medium. As the computer-readable recording medium, for example, any recording medium such as a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory may be used. Specifically, for example, a hard disk device, a flexible disk, a magnetic tape, or the like is used as a magnetic recording device, and a DVD (Digital Versatile Disc), a DVD-RAM (Random Access Memory), or a CD-ROM (Compact Disc Read Only) is used as an optical disk. Memory), CD-R (Recordable) / RW (ReWritable), magneto-optical recording media, MO (Magneto-Optical disc), EEP-ROM (Electronically Erasable and Programmable-Read Only Memory) as semiconductor memory, etc. Can be used.

  The distribution of the program is carried out, for example, by selling, transferring, lending, or the like, a portable recording medium such as a DVD or a CD-ROM on which the program is recorded. Further, the program may be stored in a storage device of a server computer, and the program may be distributed by transferring the program from the server computer to another computer via a network.

  A computer that executes such a program first stores, for example, a program recorded on a portable recording medium or a program transferred from a server computer in its own storage device. Then, when executing the process, the computer reads the program stored in its own recording medium and executes the process according to the read program. Further, as another execution form of the program, the computer may directly read the program from the portable recording medium and execute processing according to the program, and further, the program may be transferred from the server computer to the computer. Each time, the processing according to the received program may be sequentially executed. A configuration in which the above-described processing is executed by a so-called ASP (Application Service Provider) type service which realizes a processing function only by executing an instruction and acquiring a result without transferring a program from the server computer to the computer. It may be. It should be noted that the program in the present embodiment includes information used for processing by the computer and which is similar to the program (data that is not a direct command to the computer but has characteristics that define the processing of the computer).

  Further, in this embodiment, a hardware entity is configured by executing a predetermined program on a computer. However, at least a part of these processing contents may be realized by hardware.

Claims (13)

J is an integer of 1 or more, and a j-th feature quantity (1 ≦ j ≦ J) is a feature quantity indicating a feature indicating whether or not the dialogue has failed.
A dialogue destruction feature extraction unit that extracts a dialogue destruction feature that is a combination of the j-th type feature (1 ≦ j ≦ J) from a dialogue consisting of a series of utterances. ,
The dialog destruction feature amount extraction unit includes a j-th feature amount calculation unit that calculates the j-th feature amount from the dialog for each j satisfying 1 ≦ j ≦ J,
A dialogue act feature is defined as a feature generated from a dialogue act represented by an utterance included in the dialogue,
The similarity feature between utterances is a feature quantity indicating how similar the last utterance in the dialog is to other utterances,
A word feature is expressed as a Bag-of-words vector in which words N-grams of each utterance included in the dialog are arranged,
A word class feature is expressed as a Bag-of-classes vector in which word classes corresponding to words of each utterance included in the dialog are arranged,
The word combination feature is defined as the combination of any one of the word Ngram, word class Ngram, word set, and predicate term structure co-occurring between the last utterance and the other utterances in the dialog or in the last utterance. Feature quantity expressed as a Bag-of-words vector with the appearance result as an element,
The frequent word string feature amount is a feature amount having a character string of a word Ngram appearing at a predetermined frequency or more in a dialog as an element,
The sentence length feature is a feature representing the word length and character length of the last utterance in the dialogue,
The dialogue destruction feature includes a dialogue act feature, an inter-utterance similarity feature, and a sentence length feature, and is at least one of a word feature, a word class feature, a word combination feature, and a frequent word string feature. A dialogue destruction feature quantity extraction device characterized by a combination not including one feature quantity. The dialogue destruction feature quantity extraction device according to claim 1 ,
The number-of-turns feature is a feature representing the number of turns elapsed since the start of the dialogue,
The dialogue destruction feature quantity extraction device, wherein the dialogue destruction feature quantity is a combination further including a turn number feature quantity. The dialogue destruction feature quantity extraction device according to claim 2 ,
The dialogue destruction feature amount extraction apparatus, wherein the dialogue destruction feature amount is a combination that does not include any of a word feature amount, a word class feature amount, a word combination feature amount, and a frequent word string feature amount. The dialogue destruction feature quantity extraction device according to claim 3 ,
Character string co-occurrence features are expressed as a Bag-of-words vector whose elements are the appearance results of combinations of character strings Ngram co-occurring between the last utterance in the dialog and other utterances Feature
The dialogue destruction feature quantity extraction device, wherein the dialogue destruction feature quantity is a combination further including a character string co-occurrence feature quantity. The dialogue destruction feature quantity extraction device according to claim 2 ,
The perplexity feature amount is a feature amount representing perplexity calculated using a language model for each utterance included in the dialogue,
The dialogue destruction feature quantity extraction device, wherein the dialogue destruction feature quantity is a combination further including a perplexity feature quantity. J is an integer of 1 or more, and a j-th feature quantity (1 ≦ j ≦ J) is a feature quantity indicating a feature indicating whether or not the dialogue has failed.
A dialogue destruction feature extraction unit that extracts a dialogue destruction feature that is a combination of the j-th type feature (1 ≦ j ≦ J) from a dialogue consisting of a series of utterances. ,
The dialog destruction feature amount extraction unit includes a j-th feature amount calculation unit that calculates the j-th feature amount from the dialog for each j satisfying 1 ≦ j ≦ J,
The similarity feature between utterances is a feature quantity indicating how similar the last utterance in the dialog is to other utterances,
A frequently occurring word string feature amount is a feature amount having a character string of a word Ngram appearing at a predetermined frequency or more in a dialogue as an element,
A dialogue act feature is defined as a feature generated from a dialogue act represented by an utterance included in the dialogue,
The sentence length feature is defined as a feature representing the word length and character length of the last utterance in the dialogue,
The word combination feature is defined as the combination of any one of the word Ngram, word class Ngram, word set, and predicate term structure co-occurring between the last utterance and the other utterances in the dialog or in the last utterance. A feature quantity expressed as a Bag-of-words vector with the appearance result as an element,
The dialogue destruction feature amount extraction device, wherein the dialogue destruction feature amount is a combination including an inter-utterance similarity feature amount, a frequent word string feature amount, a dialogue action feature amount, a sentence length feature amount, and a word combination feature amount. . The dialogue destruction feature quantity extraction device according to any one of claims 1 to 6 , wherein:
When the dialogue destruction feature amount includes an inter-utterance similarity feature amount, the inter-utterance similarity feature amount is a similarity calculated using an average vector distance of word2vec, and a similarity calculated using a WordMoversDistance distance. , A dialogue destruction feature quantity extraction device characterized by being a vector in which at least one of the similarity calculated using a BLEU score and the similarity calculated using a ROUGE score is arranged as an element. . J is an integer of 1 or more, and a j-th feature quantity (1 ≦ j ≦ J) is a feature quantity indicating a feature indicating whether or not the dialogue has failed.
A dialogue destruction feature extraction unit that extracts a dialogue destruction feature that is a combination of the j-th type feature (1 ≦ j ≦ J) from a dialogue consisting of a series of utterances. ,
The dialog destruction feature amount extraction unit includes a j-th feature amount calculation unit that calculates the j-th feature amount from the dialog for each j satisfying 1 ≦ j ≦ J,
A dialogue act feature is defined as a feature generated from a dialogue act represented by an utterance included in the dialogue,
The similarity feature between utterances is a feature quantity indicating how similar the last utterance in the dialog is to other utterances,
A word feature is expressed as a Bag-of-words vector in which words N-grams of each utterance included in the dialog are arranged,
The word class feature is a feature expressed as a Bag-of-classes vector in which word classes corresponding to words of each utterance included in the dialog are arranged,
The interactive fracture characteristic amount dialogue act characteristic quantity includes an utterance similarity between the feature amount, Ri combination der containing further word feature, one or more feature quantities any of the word class characteristic quantity,
When the dialogue destruction feature amount includes an inter-utterance similarity feature amount, the inter-utterance similarity feature amount is a similarity calculated using an average vector distance of word2vec, and a similarity calculated using a WordMoversDistance distance. , A dialogue destruction feature quantity extraction device characterized by being a vector in which at least one of the similarity calculated using a BLEU score and the similarity calculated using a ROUGE score is arranged as an element. . The dialogue destruction feature quantity extraction device according to any one of claims 1 to 6 , wherein:
A dialogue action sequence feature is defined as a feature expressed as a vector having elements obtained by estimating a dialogue action represented by each utterance included in the dialogue,
The predicted dialogue act feature amount is a prediction result vector representing a result of predicting a dialogue action that the last utterance should have from the utterance included in the dialogue, and represents a result of estimating the dialogue act represented by the prediction result vector and the last utterance. Any one of a vector in which the estimation result vectors are arranged, a difference vector between the prediction result vectors and the estimation result vectors, and a boolean value indicating whether or not 1best of the prediction result vectors and the estimation result vectors match. The feature amount consisting of the above,
The dialogue act feature amount is a dialogue act sequence feature amount or a predictive dialogue act feature amount. J is an integer of 1 or more, and a j-th feature quantity (1 ≦ j ≦ J) is a feature quantity indicating a feature indicating whether or not the dialogue has failed.
A dialogue destruction feature amount extracting apparatus for extracting a dialogue destruction feature amount which is a combination of the j-th type feature amount (1 ≦ j ≦ J) from a dialogue made up of a series of utterances. A destructive feature extraction method,
The dialog destruction feature amount extraction step includes a j-th type feature amount calculation step of calculating the j-th type feature amount from the dialog for each j satisfying 1 ≦ j ≦ J,
A dialogue act feature is defined as a feature generated from a dialogue act represented by an utterance included in the dialogue,
The similarity feature between utterances is a feature quantity indicating how similar the last utterance in the dialog is to other utterances,
A word feature is expressed as a Bag-of-words vector in which words N-grams of each utterance included in the dialog are arranged,
A word class feature is expressed as a Bag-of-classes vector in which word classes corresponding to words of each utterance included in the dialog are arranged,
The word combination feature is defined as the combination of any one of the word Ngram, word class Ngram, word set, and predicate term structure co-occurring between the last utterance and the other utterances in the dialog or in the last utterance. Feature quantity expressed as a Bag-of-words vector with the appearance result as an element,
The frequent word string feature amount is a feature amount having a character string of a word Ngram appearing at a predetermined frequency or more in a dialog as an element,
The sentence length feature is a feature representing the word length and character length of the last utterance in the dialogue,
The dialogue destruction feature includes a dialogue act feature, an inter-utterance similarity feature, and a sentence length feature, and is at least one of a word feature, a word class feature, a word combination feature, and a frequent word string feature. A method for extracting dialogue destruction features, which is a combination that does not include one feature .   J is an integer of 1 or more, and a j-th feature quantity (1 ≦ j ≦ J) is a feature quantity indicating a feature indicating whether or not the dialogue has failed.
  A dialog destruction feature extracting step of extracting a dialog destruction feature, which is a combination of the j-th type feature (1 ≦ j ≦ J), from a dialog consisting of a series of utterances;
  Dialogue feature extraction method including
  The dialog destruction feature amount extraction step includes a j-th type feature amount calculation step of calculating the j-th type feature amount from the dialog for each j satisfying 1 ≦ j ≦ J,
  The similarity feature between utterances is a feature quantity indicating how similar the last utterance in the dialog is to other utterances,
  A frequently occurring word string feature amount is a feature amount having a character string of a word Ngram appearing at a predetermined frequency or more in a dialogue as an element,
  A dialogue act feature is defined as a feature generated from a dialogue act represented by an utterance included in the dialogue,
  The sentence length feature is defined as a feature representing the word length and character length of the last utterance in the dialogue,
  The word combination feature is defined as the combination of any one of the word Ngram, word class Ngram, word set, and predicate term structure co-occurring between the last utterance and the other utterances in the dialog or in the last utterance. A feature quantity expressed as a Bag-of-words vector with the appearance result as an element,
  The dialogue destruction feature amount extraction method, wherein the dialogue destruction feature amount is a combination including an utterance similarity feature amount, a frequent word string feature amount, a dialogue action feature amount, a sentence length feature amount, and a word combination feature amount. .   J is an integer of 1 or more, and a j-th feature quantity (1 ≦ j ≦ J) is a feature quantity indicating a feature indicating whether or not the dialogue has failed.
  A dialog destruction feature extracting step of extracting a dialog destruction feature, which is a combination of the j-th type feature (1 ≦ j ≦ J), from a dialog consisting of a series of utterances;
  Dialogue feature extraction method including
  The dialog destruction feature amount extraction step includes, for each j satisfying 1 ≦ j ≦ J, a j-th feature amount calculation step of calculating the j-th feature amount from the dialog.
  A dialogue act feature is defined as a feature generated from a dialogue act represented by an utterance included in the dialogue,
  The similarity feature between utterances is a feature quantity indicating how similar the last utterance in the dialog is to other utterances,
  A word feature is expressed as a Bag-of-words vector in which words N-grams of each utterance included in the dialog are arranged,
  The word class feature is a feature expressed as a Bag-of-classes vector in which word classes corresponding to words of each utterance included in the dialog are arranged,
  The dialogue destruction feature amount includes a dialogue act feature amount, an utterance similarity feature amount, and further includes a word feature amount, a combination including one or more feature amounts of word class feature amounts,
  When the dialogue destruction feature amount includes an inter-utterance similarity feature amount, the inter-utterance similarity feature amount is a similarity calculated using an average vector distance of word2vec, and a similarity calculated using a WordMoversDistance distance. A dialogue destruction feature amount extraction method, characterized in that the vector is a vector in which at least one of the similarity calculated using a BLEU score and the similarity calculated using a ROUGE score is arranged as an element. . Program for causing a computer to function as the interactive fracture feature extraction equipment according to any one of claims 1 to 9.

Images