JP2013117842A - Knowledge amount estimation information generating device, knowledge amount estimating device, method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a knowledge amount information generating device for accurately estimating a knowledge amount of an inquirer with respect to a subject which becomes an object of conversation, on the basis of a conversation result, and to provide a knowledge amount estimating device, a method, and a program.SOLUTION: A knowledge amount information generating device comprises: an inquiry extraction part 2 extracts an inquiry speech string which is a speech string from text data of a speech string in conversation between an inquirer and a respondent; a feature quantity extraction part 3 extracts inquiry feature quantity indicating feature quantity regarding a generation state of the inquiry speech string in the text data on the basis of the inquiry speech string; an estimation information generation part 4 uses the inquiry feature quantity, and knowledge amount information indicating a knowledge amount of the inquirer, which is preliminarily given to the text data considered as an object of extraction of the inquiry feature quantity, as learning data to generate estimation information to be used for estimation of the knowledge amount corresponding to the text data; and a knowledge amount estimation part 5 estimates the knowledge amount corresponding to text data for estimation on the basis of the inquiry feature quantity extracted from the text data for estimation and the estimation information generated by the estimation information generation part 4.

Description

  The present invention relates to a knowledge amount estimation information generation device, a knowledge amount estimation device, a method, and a program, and in particular, a knowledge amount related to the content of an inquirer's dialogue from text data of an utterance string in the dialogue between the inquirer and the respondent. The present invention relates to a knowledge amount estimation information generation device, a knowledge amount estimation device, a method, and a program suitable for accurately estimating the amount of information.

  For example, Patent Documents 1 and 2 disclose conventional techniques for estimating the amount of human knowledge using a programmed computer. In these Patent Documents 1 and 2, a technique for estimating a background such as a knowledge amount and a knowledge depth for each field of a user who performs a document search is described.

  In document search, in order to obtain information that the user is searching for, a search is performed by specifying a keyword that is likely to be related to the information. In order to improve the accuracy of the search, it is known that acquisition of information for knowing the background of the user who performs the search is important. For this reason, in Patent Document 3, for example, the keyword history specified by the user or In addition, a technique for estimating a favorite field such as which field is interested from the history of a browsed electronic document is described.

  However, with the preference estimation technique described in Patent Literature 3, even if the field of information that the user frequently examines can be estimated, it is not possible to estimate how familiar the user is in that field.

  On the other hand, in Patent Documents 1 and 2, a query log that is a part of a user log of a search system is used, a degree of expertise of a query is calculated for each field from a query of the query log, and a background relating to a user's knowledge amount It is possible to estimate how familiar the user is in the field.

  Further, Non-Patent Document 1 discloses a technique similar to that of Patent Documents 1 and 2 described above. That is, paying attention to the expertise of the vocabulary used, the expertise of each word is calculated from its rarity, etc., and the average value of the expertise of the word used as the query from the history of the search query entered by a user Is described, and a technique for estimating the knowledge amount of the user is described.

  Non-Patent Document 2 describes a technique for estimating the amount of knowledge of a user for a user of a bus operation information guidance system in order to perform a collaborative dialogue with the user in a voice dialogue system. Here, a technique is described in which the ratio of bus stop attributes (bus stops used only by citizens / others) is determined from the history of bus stops searched by a certain user, and the amount of knowledge about the area of the user is estimated. .

JP 2011-170699 A JP 2011-221872 A JP 2000-148773 A

Daisuke Sato, Yoshihito Yasuda, Takayuki Mochizuki, Tomoya Suzuki, Yumiko Matsuura, Ryoji Kataoka, “Knowledge Estimation by Search System User Field”, DEIM Forum 2010 (2010) Junichi Ueno, Kazunori Komatani, Tatsuya Kawahara, Hiroshi Okuno, “Generating Adaptive Response Using User Model in Bus Information Guidance System”, Spoken Language Information Processing 2002, vol.65, pp.5-10 (2002)

  However, with the knowledge amount estimation techniques described in Patent Documents 1 and 2 and Non-Patent Document 1, it is not possible to estimate the speaker's knowledge amount for speakers in a dialogue between humans.

  In order to be able to estimate the amount of knowledge even for a conversation speaker, it is necessary to be able to handle the speech recognition result as an input for knowledge amount estimation. However, when the speech recognition technology is used in the technology for estimating the amount of knowledge of the user using the above-described vocabulary expertise, there is a problem that it is vulnerable to misrecognition included in the speech recognition result.

  As a specific problem, if some highly specialized words appear in the speech recognition result of a speaker's utterance, the knowledge amount of the speaker may be presumed to be erroneously high. is there.

  Examples of situations where knowledge amount estimation technology is used for such conversational speakers include contact center call analysis and voice dialogue systems. In either case, speech recognition is indispensable. Therefore, the knowledge amount estimation technique used in such a scene is required to be robust against misrecognition included in the speech recognition result.

  In the above-mentioned Non-Patent Document 2, when estimating the amount of knowledge of the user in the bus operation information guidance system, the voice information input by the user in a certain area by telephone or the like is recognized by the voice dialogue system, and A technique for estimating the amount of knowledge of the user is described.

  However, also in this technology, the voice information input by the user by telephone or the like is specific information such as a bus stop, a boarding place, and a getting-off place, and further, attributes of these specific information, that is, only citizens use it. Similar to the above-mentioned Patent Documents 1 and 2 and Non-Patent Document 1, such as bus stops and other bus stops, formal bus stop names, bus stop designations with the nearest facility names, etc. The amount of knowledge of the user in the area is estimated, and if some highly specialized words appear in error in the speech recognition result of the user's utterance, the amount of knowledge of the user is erroneously high. It may be estimated that.

  The problem to be solved by the present invention is that, when the technology for estimating the knowledge amount of the user is used for estimating the knowledge amount for the speaker of the dialogue by voice, focusing on the expertise of the conventional user's vocabulary There is a possibility that the knowledge amount of the speaker cannot be estimated correctly due to misrecognition included in the speech recognition result.

  The present invention has been made to solve the above-mentioned problems, and based on the result of dialogue, it is possible to estimate the knowledge amount of the inquirer with respect to the subject matter of the dialogue with high accuracy. An object is to provide an information generation device, a knowledge amount estimation device, a method, and a program.

  In order to achieve the above object, the knowledge amount estimation information generating apparatus according to claim 1 is a question utterance sequence which is an utterance sequence of a question to the other party from text data of an utterance sequence in a dialogue between an inquirer and a respondent. Based on the question utterance string extracted by the question extraction means, feature quantity extraction means for extracting a question feature quantity indicating a feature quantity related to the occurrence state of the question utterance string in the text data, and The question feature amount extracted by the feature amount extraction means and the knowledge amount of the inquirer assumed from the dialogue indicated by the text data with respect to the text data targeted for extraction of the question feature amount Knowledge that generates presumed information used for estimating the knowledge amount corresponding to the text data by using knowledge amount information given in advance as learning data It includes a quantity estimation information generating means.

  According to the knowledge amount estimation information generating apparatus according to claim 1, the question utterance sequence that is the utterance sequence of the question to the other party is obtained from the text data of the utterance sequence in the dialogue between the inquirer and the respondent by the question extraction unit. Based on the question utterance string extracted by the question extracting means, the extracted feature quantity indicating the feature quantity related to the occurrence state of the question utterance string in the text data is extracted by the feature quantity extracting means, and the knowledge amount estimation information Knowledge of the inquirer assumed from the dialogue indicated by the text data for the question feature quantity extracted by the feature quantity extraction means by the generation means and the text data targeted for extraction of the question feature quantity By using, as learning data, data to which knowledge amount information has been assigned in advance as an indication of the amount, an estimate used for estimating the amount of knowledge corresponding to text data Information is generated.

  That is, in the present invention, a question exchanged between the inquirer and the respondent in the dialogue is applied to the estimation of the knowledge amount of the inquirer, thereby focusing on the expertise of the conventional vocabulary used. It is possible to solve problems associated with misrecognition in the technology for estimating the amount of knowledge.

  The learning data includes a voice recognition process based on text data that transcribes the dialogue between the inquirer and the respondent who are the target of the question feature extraction, and voice data of the dialogue between the inquirer and the respondent. The text data created by performing is included. For example, the knowledge amount information is manually given in advance using at least one of text data and voice data of dialogue between the inquirer and the respondent.

  Thus, according to the knowledge amount estimation information generating device according to claim 1, the question utterance sequence that is the utterance sequence of the question to the other party is extracted from the text data of the utterance sequence in the dialogue between the inquirer and the respondent. Then, based on the extracted question utterance sequence, a question feature amount indicating a feature amount related to the occurrence state of the question utterance sequence in the text data is extracted, and the extracted question feature amount and the question feature amount are extracted. Knowledge amount corresponding to the text data by using, as learning data, knowledge amount information given in advance as indicating the knowledge amount of the inquirer assumed from the dialogue indicated by the text data The estimation information used for the estimation of the inquiries is generated. By using this estimation information, it is possible to accurately estimate the inquirer's knowledge amount regarding the subject matter of the dialogue. It can be carried out well.

  On the other hand, in order to achieve the above object, a knowledge amount estimation device according to claim 2 includes a storage unit that stores in advance the estimation information generated by the knowledge amount estimation information generation device according to claim 1; Question extraction means for extracting a question utterance string, which is an utterance string of a question to the other party, from text data of an utterance string in the dialogue between the inquirer and the respondent of the estimation target, and the question utterance extracted by the question extraction means A feature amount extracting means for extracting a question feature amount indicating a feature amount relating to an occurrence state of the question utterance sequence in the text data based on the sequence; a question feature amount extracted by the feature amount extracting means; Knowledge amount estimation means for estimating the knowledge amount corresponding to the text data to be estimated using the estimated information.

  According to the knowledge amount estimation device described in claim 2, the estimation information generated by the knowledge amount estimation information generation device according to claim 1 is stored in advance by the storage means, and the knowledge amount estimation target is determined by the question extraction means. The question utterance sequence that is the utterance sequence of the question to the other party is extracted from the text data of the utterance sequence in the dialogue between the inquirer and the respondent, and the question utterance extracted by the question extraction unit by the feature amount extraction unit Based on the column, a question feature amount indicating a feature amount related to the occurrence state of the question utterance sequence in the text data is extracted, and the question feature amount extracted by the feature amount extraction unit by the knowledge amount estimation unit and the storage unit in advance A knowledge amount corresponding to text data to be estimated is estimated using the stored estimation information.

  As described above, according to the knowledge amount estimation device according to claim 2, the estimation information generated by the knowledge amount estimation information generation device according to claim 1 is stored in advance, A question utterance string, which is the utterance string of the question to the other party, is extracted from the text data of the utterance string in the dialogue of the user, and based on the extracted question utterance string, a question indicating a feature amount regarding the occurrence state of the question utterance string in the text data The feature quantity is extracted, and the knowledge quantity corresponding to the text data to be estimated is estimated using the extracted question feature quantity and the stored estimated information. Knowledge amount can be estimated with high accuracy.

  The question extraction means and the feature quantity extraction means in the knowledge amount estimation apparatus according to claim 2 may be used in common with the question extraction means and the feature quantity extraction means in the knowledge amount estimation information generation apparatus according to claim 1. it can.

  Further, according to the present invention, as in the invention described in claim 3, the question feature amount includes the number of questions of the inquirer, the number of questions of the respondent, the number of questions of the inquirer, and the number of questions of the respondent. , The time in the text data where the question of the inquirer occurred, and the time in the text data where the question of the respondent occurred. Since all of these question feature quantities can be extracted relatively easily and with high precision, they are subject to dialogue more easily and with higher precision than when other feature quantities are applied. It is possible to estimate the knowledge amount of the inquirer for the matter.

  On the other hand, in order to achieve the above object, the knowledge amount estimation information generating method according to claim 4 is a method for generating a question that is an utterance string of a question to a partner from text data of an utterance string in a dialogue between an inquirer and an answerer. A question extraction step for extracting an utterance string, and a feature quantity extraction step for extracting a question feature quantity indicating a feature quantity regarding the occurrence state of the question utterance string in the text data based on the question utterance string extracted in the question extraction step And the question feature amount extracted in the feature amount extraction step, and the knowledge amount of the inquirer assumed from the dialogue indicated by the text data with respect to the text data targeted for extraction of the question feature amount Is used for estimating the knowledge amount corresponding to the text data by using knowledge amount information given in advance as learning data. And a, and knowledge estimation information generation step of generating estimated information.

  Therefore, according to the knowledge amount estimation information generation method according to the fourth aspect, since it operates in the same manner as the invention according to the first aspect, the generated estimation information is used in the same manner as the invention according to the first aspect. Thus, it is possible to accurately estimate the amount of knowledge of the inquirer with respect to the matter that is the subject of the dialogue.

  On the other hand, in order to achieve the above object, a knowledge amount estimation method according to claim 5 includes a storage step of previously storing the estimation information generated by the knowledge amount estimation information generation method according to claim 4 in a storage device; A question extraction step for extracting a question utterance sequence, which is an utterance sequence of a question to the other party, from text data of an utterance sequence in a dialogue between an inquirer and a respondent as an object of estimation of knowledge amount, and extraction in the question extraction step A feature amount extracting step for extracting a question feature amount indicating a feature amount relating to an occurrence state of the question utterance sequence in the text data based on the question utterance sequence, the question feature amount extracted in the feature amount extracting step, and A knowledge amount estimation step of estimating the knowledge amount corresponding to the text data to be estimated using the estimation information stored in the storage step. There.

  Therefore, according to the knowledge amount estimation method described in claim 5, since it operates in the same manner as in the invention described in claim 2, as in the invention described in claim 2, an inquiry about the matter subject to dialogue is performed. The amount of knowledge of the person can be estimated with high accuracy.

  On the other hand, in order to achieve the above object, according to the program according to claim 6, the computer operates in the same manner as the knowledge amount estimation information generating device according to claim 1 or the knowledge amount estimation device according to claim 2. Therefore, similarly to the knowledge amount estimation information generation device or the knowledge amount estimation device, it is possible to accurately estimate the inquirer's knowledge amount with respect to the subject matter of the dialogue.

  The vocabulary used and the content of the conversation content are estimated by estimating the amount of knowledge of the inquirer regarding the subject matter of the dialogue using the feature amount related to the occurrence state of the question extracted from the question in the dialogue between the inquirer and the respondent. Regardless, it is possible to estimate the knowledge amount of the inquirer with high accuracy. In addition, since the feature amount related to the occurrence state of the question that does not depend on the vocabulary used is used, the feature amount is not easily affected by a speech recognition error. There is no need to prepare learning data for each field of vocabulary or conversation content.

It is a block diagram (partial flowchart) which shows the functional structural example of the knowledge amount estimation apparatus which concerns on embodiment. It is a block diagram which shows the computer structural example of the knowledge amount estimation apparatus which concerns on embodiment. It is a flowchart which shows the flow of a process of the learning process program performed by the knowledge amount estimation apparatus which concerns on embodiment. It is a flowchart which shows the flow of a process of the estimation process program performed by the knowledge amount estimation apparatus which concerns on embodiment. It is a schematic diagram which shows an example of the question pattern information used with the knowledge amount estimation apparatus which concerns on embodiment. It is a figure with which it uses for description of operation | movement of the knowledge amount estimation apparatus which concerns on embodiment, and is a schematic diagram which shows an example of the text data which consists of an utterance row | line | column. It is a figure with which it uses for description of operation | movement of the knowledge amount estimation apparatus which concerns on embodiment, and is a schematic diagram which shows an example of a user's utterance content information. It is a figure with which it uses for description of operation | movement of the knowledge amount estimation apparatus which concerns on embodiment, and is a schematic diagram which shows an example of an operator's utterance content information. It is a figure with which it uses for description of operation | movement of the knowledge amount estimation apparatus which concerns on embodiment, and is a schematic diagram which shows an example of criteria information. It is a figure with which it uses for description of operation | movement of the knowledge amount estimation apparatus which concerns on embodiment, and is a schematic diagram which shows an example of evaluation result information.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In this example, an explanation will be given of an example in which the amount of knowledge of a user is estimated by using a voice recognition result of an interaction between an operator and a user in a contact center (hereinafter referred to as a contact center call) as an input.

  The knowledge amount estimation apparatus 10 according to the present embodiment has the functional configuration shown in FIG. 1, and has the computer configuration shown in FIG. First, the configuration of the computer will be described with reference to FIG.

  As shown in FIG. 2, the knowledge amount estimation device 10 according to the present embodiment includes a CPU (Central Processing Unit) 22 that controls the operation of the entire knowledge amount estimation device 10, and when the CPU 22 executes various programs. RAM (Random Access Memory) 24 used as a work area, a ROM (Read Only Memory) 26 in which various control programs and various parameters are stored in advance, and a hard disk 28 (see FIG. The external interface 30 (referred to as “External I / F” in the figure) is responsible for exchanging various types of information among the “HDD” 28), the keyboard 14, mouse 16, display 18, and externally connected devices. These are connected to each other by a system bus BUS. Note that the external interface 30 is connected to a communication device 50 such as a headset for inputting / outputting voice data exchanged through dialogue, for example.

  The CPU 22 accesses the RAM 24, the ROM 26 and the hard disk 28, acquires various information via the keyboard 14 and the mouse 16, displays various information on the display 18, and exchanges via the call device 50 connected to the external interface 30. Input / output of dialogue information consisting of voice data can be performed.

  Each process in the knowledge amount estimation device 10 according to the present invention shown in FIG. 1 is executed by the CPU 22 reading a program for controlling the processing as the knowledge amount estimation device according to the present invention stored in the hard disk 28 into the RAM 24 and executing it. Part functions are executed.

  As shown in FIG. 1, in the knowledge amount estimation device 10 according to the present embodiment, an utterance string extraction unit 1, a question extraction unit 2, a feature amount extraction unit 3, an estimated information generation unit 4, and a knowledge amount estimation unit 5 are included. Provided as a function realized by programmed computer processing.

  The processing unit in the knowledge amount estimation apparatus 10 according to the present embodiment configured as described above is roughly divided into a “learning unit” and an “estimation unit”.

  The learning unit includes an utterance string extraction unit 1, a question extraction unit 2, a feature amount extraction unit 3, and an estimated information generation unit 4, and is prepared for learning in advance, and is a voice recognition result for a dialogue between an inquirer and a respondent ( When the conversation information (learning call) 7 is input, the utterance string extraction unit 1 first extracts text data composed of the utterance string.

  Next, the question extraction unit 2 extracts a question utterance sequence that is an utterance sequence of the question to the other party from the text data of the utterance sequence in the dialogue between the inquirer and the respondent extracted by the utterance sequence extraction unit 1. Based on the question utterance sequence extracted by the question extraction unit 2, the feature amount extraction unit 3 extracts a question feature amount indicating a feature amount related to the occurrence state of the question utterance sequence in the text data.

  Then, in the estimation information generation unit 4, the question feature amount 4a extracted by the feature amount extraction unit 3 (described as “feature amount 4a” in the figure), and the text data that is the target of extraction of the question feature amount 4a Use knowledge amount information 4b (denoted as “knowledge amount label 4b” in the figure) given in advance as an indication of the knowledge amount of the inquirer assumed from the dialogue indicated by the text data as learning data Thus, estimation information used to estimate the knowledge amount corresponding to the text data is generated by the model generation function 4c (described as “model generation 4c for associating the knowledge amount with the feature amount” in the drawing).

  Thus, the learning unit of this example corresponds to the knowledge amount estimation information generation device according to the present invention. Then, the learning unit performs such processing on a relatively large number of previously prepared dialogs, thereby generating estimation information as model information 5a (denoted as “model 5a” in the figure), and a hard disk It memorize | stores in memory | storage devices, such as 28.

  On the other hand, the estimation unit includes an utterance string extraction unit 1, a question extraction unit 2, a feature amount extraction unit 3, and a knowledge amount estimation unit 5. The estimation unit is generated in a learning unit (knowledge amount estimation information generation device) and stored in the hard disk 28 or the like. Using the model information 5a stored in the apparatus, the amount of knowledge related to the matters of the dialogue of the inquirer in the individual dialogue to be estimated is estimated.

  That is, when a speech recognition result (estimation target) 6 of a call i, which is an individual conversation to be estimated, is input, first, the utterance string extraction unit 1 extracts text data including the utterance string. Note that the speech recognition result (estimation target) 6 of the call i may not be used if the recognition accuracy is too low.

  Next, in the question extraction unit 2, a question utterance sequence that is an utterance sequence of a question to the other party from the text data of the utterance sequence in the dialogue between the inquirer to be estimated and the respondent extracted by the utterance sequence extraction unit 1. The feature amount extraction unit 3 extracts a question feature amount indicating a feature amount related to the occurrence state of the question utterance sequence in the text data based on the question utterance sequence extracted by the question extraction unit 2.

  Then, in the knowledge amount estimation unit 5, the question feature amount extracted by the feature amount extraction unit 3 by the collation function 5 c (described as “estimate the knowledge amount 5 c by collating with the model” in the figure), and the learning unit The estimated amount of knowledge corresponding to the text data to be estimated is estimated using the estimated information generated by the estimated information generating unit 4 and stored as the model information 5a in the storage device such as the hard disk 28. The knowledge amount corresponding to the text data estimated in this way is described as the knowledge amount 8 of the speaker (user) of the call i inputted as the estimation target (in the figure, “output: knowledge amount label 8 of the call i”). ) Is output.

  In this example, the same utterance string extraction unit 1, question extraction unit 2, and feature amount extraction unit 3 are used in the learning unit and the estimation unit, but each of the learning unit and the estimation unit is used. The utterance string extraction unit 1, the question extraction unit 2, and the feature amount extraction unit 3 may be different.

  Further, in the knowledge amount estimation apparatus 10 according to the present embodiment, voice recognition processing is performed on voice data input / output by the call device 50 shown in FIG. Then, the speech recognition result (estimation target) 6 and the speech recognition result (learning call) 7 of the call i are generated.

  Then, the utterance string extraction unit 1 extracts only the portion corresponding to the utterance from the speech recognition result (learning call) 7 which is a text file. Usually, the speech recognition result often includes time information, information indicating recognition accuracy, and the like, so this processing is necessary.

  If the data input to the utterance string extraction unit 1 is not a text file indicating a speech recognition result, but text data including an utterance string is input, processing in the utterance string extraction unit 1 is not necessary. .

  Further, in this example, the question extraction unit 2 determines whether or not the utterance is a “question” for all utterances of the operator and the user in the dialogue.

  In addition, the feature quantity extraction unit 3 extracts the number of questions extracted by the question extraction unit 2, the ratio of the number of user-to-operator questions, and the occurrence time (timing) of the question in the dialogue as question feature quantities. To do.

  The modeling of the association between the question feature quantity and the knowledge quantity in the estimation information generation unit 4 and the automatic estimation of the knowledge quantity of the speaker on the user side in the knowledge quantity estimation unit 5 are known techniques, for example. For example, “Support Vector Machine (SVM)” which is one of identification methods of “Supervised learning” is used.

  Hereinafter, using FIG. 3 and FIG. 4, the utterance string extraction unit 1, the question extraction unit 2, the feature amount extraction unit 3, the estimated information generation unit 4, and the knowledge amount of the knowledge amount estimation device 10 according to the present embodiment. The details of the processing performed by the estimation unit 5 will be described.

  FIG. 3 shows the processing contents in the learning unit of the knowledge amount estimation apparatus 10 according to the present embodiment. First, it is determined whether or not a predetermined amount of speech data has been input (step S101). When input, the voice data is reproduced (step S102), voice recognition processing is performed to transcribe the voice data into text data (step S103), and the speech string extraction unit 1 transcribes the voice recognition result. The text data of the speech string corresponding to the speech is extracted from the text data (step S104).

  For example, if the speech recognition result is an xml file, and the part corresponding to the utterance is enclosed in <TEXT> and </ TEXT>, <TEXT> and </ TEXT If the part surrounded by> is specified, the utterance can be extracted.

  Next, in the question extraction unit 2, each utterance exchanged between the user and the operator exchanged in the dialogue indicated by the input voice data with respect to the text data composed of the utterance sequence extracted by the utterance sequence extraction unit 1. Is determined as “question”, and the text data of the utterance string determined as “question” is extracted as a question utterance string (step S105).

  Whether each utterance exchanged in the dialogue here is a “question” is determined by, for example, “Kenji Imamura, Atsuko Izumi, Genichiro Kikui, Satoshi Sato,“ Semantic Label Tagger of Predicate Functional Expression ”, Language Processing Society of Japan It may be automatically performed by a statistical method using a known technique described in “17th Annual Conference Papers (2011)” or a pattern matching technique with a language expression that represents a question prepared in advance. You may go by.

  In this example, the latter pattern matching technique is adopted, and a question utterance string is extracted using a pattern such as “Is it?” Or “Is it?” As illustrated as question pattern information in FIG. Note that the pattern information shown in FIG. 5 is stored in advance in the hard disk 28 or the like in FIG.

  Next, the feature amount extraction unit 3 uses the question utterance sequences extracted by the question extraction unit 2 and uses the question utterance sequences of the user and the operator, and three types (“1. And “number of questions of each operator”, “2. question ratio of user to operator”, “3. timing of user and operator questions (timing)”) (step S106). Such a plurality of feature amounts are called feature amount vectors.

  Then, in the estimated information generation unit 4, knowledge amount information (knowledge amount label 4b) as a correct answer label is input to the dialogue indicated by the sound data in accordance with the sound data reproduced in the process of step S102. (Step S107), the question feature quantity (feature quantity vector) extracted by the feature quantity extraction unit 3 and the input knowledge quantity information (knowledge quantity label 4b) are made into a one-to-one set as learning data, Model information 5a (estimated information) used to estimate the amount of knowledge is generated from the features by learning the model using “Support Vector Machine (SVM)”, which is a well-known pattern identification method, with learning data as input (Step S108), and then the present process is terminated.

  “Support Vector Machine (SVM)” is a binary discrimination method. When the knowledge amount is estimated in two stages using “Support Vector Machine (SVM)”, a model is created to determine whether the knowledge amount is “high knowledge amount” or not. When creating a model, for example, learning data corresponding to “high knowledge amount” is set as positive learning data, and other learning data is negative learning data (learning data corresponding to “small knowledge amount”). Do.

  When the knowledge amount is estimated in three stages using “Support Vector Machine (SVM)”, three models are created for each stage by learning. Model 1 that discriminates whether “knowledge amount is large” or not, Model 2 that discriminates whether “knowledge amount is medium” or other, and “knowledge amount is small” Or model 3 for discriminating whether it is other than that. For example, in the model 1, learning is performed with learning data corresponding to “a large amount of knowledge” as positive learning data, and other learning data as negative learning data. In the model 2, learning is performed with learning data corresponding to “medium amount of knowledge” as positive learning data and other learning data as negative learning data. In the model 3, learning is performed with learning data corresponding to “small knowledge amount” as positive learning data and other learning data as negative learning data.

  In this example, model learning is performed using “Support Vector Machine (SVM)” for pattern identification. However, the present invention is not limited to this. For example, a neural network based on back-propagation learning is used. It is also possible to do learning. Alternatively, a method of creating a feature amount vs. knowledge amount table and determining the knowledge amount based on the table may be used. The generated model information 5a is stored in a storage device such as the hard disk 28 in FIG.

  On the other hand, FIG. 4 shows the processing contents in the estimation unit of the knowledge amount estimation apparatus 10 according to the present embodiment, and a dialogue (call) between the user and the operator at the contact center to be estimated (hereinafter referred to as “call”). When the speech data of “estimation target dialogue” is input (step S201), the speech recognition process is performed on the speech data (step S203), and then the speech sequence is the same as in steps S104 to S106 of FIG. Each process of the extraction unit 1, the question extraction unit 2, and the feature amount extraction unit 3 (steps S204, S205, and S206) is performed, and thereafter, in the knowledge amount estimation unit 5, as in the estimated information generation unit 4, “Support Vector Machine” (SVM) "is used to generate the question feature quantity (feature quantity vector) obtained by the process of step S206 by the process by the learning unit described above (FIG. 3). By collating with the model information 5a, the knowledge amount of the user (speaker) who is performing the estimation target dialogue is estimated by specifying which knowledge amount the estimation target dialogue corresponds to. (Step S209). As for the knowledge amount estimation unit 5, similarly to the estimated information generation unit 4, the knowledge amount is estimated using “Support Vector Machine (SVM)”. Any estimation method may be used. For example, when the knowledge amount is estimated using the above-described models 1 to 3 using the “Support Vector Machine (SVM)”, the determination is made using each of the models 1 to 3, and the positive data of the most suitable model is used. Let the amount of knowledge corresponding to be the speaker's amount of knowledge.

  Next, the three types of question feature quantities extracted by the feature quantity extraction unit 3, that is, “1. User and operator question count”, “2. User-to-operator question ratio”, and “3. User The feature amount vector consisting of “time of operator's question (timing)” will be described in detail.

  First, “1. The number of questions for each of the user and the operator” will be described below.

  Of the roles played by questions in dialogue (calls) at the contact center, the most basic is to request information from the other party. Specifically, a question that a user issues is for the operator to provide knowledge that the user lacks, and a question that the operator issues is information about the user's requirements and situation. It is for having you offer.

  In other words, it can be said that the user's number of questions reflects the amount of knowledge that the user lacks, and the operator's number of questions reflects the effort required to grasp the user's requirements and situation. Here, when the operator tries to grasp the user's requirement / situation, it is considered that more labor is required in the dialogue with the user having a small amount of knowledge, so if the number of questions of the user and the operator is used, The amount of knowledge of the user can be estimated.

  In this example, the values of the user's question count and the operator's question count are used as feature values. For example, in the case of the dialogue content illustrated by the text data 500 including the utterance string of FIG. 6, the utterance including the part with the underline is “question”. In the example of FIG. Means two times.

  Accordingly, here, the value used as the feature amount is “2” for both the number of questions by the operator and the number of questions by the user. 6 is stored in a storage device such as the RAM 24, the hard disk 28, or the like.

  Next, “2. User-to-operator question ratio” will be described below.

  A user with a small amount of knowledge often cannot accurately interpret a question asked by an operator, and “question return” in which a question (an operator) is further returned as a question frequently occurs. Therefore, if the ratio of the number of questions of the user to the number of questions of the operator is high, it can be estimated that the knowledge amount of the user is small.

  Here, the user-to-operator question ratio is a value that can represent the ratio by simply dividing the number of questions of one speaker by the number of questions of the other, or by a normalized value such as a percentage. It ’s fine. In this example, the value calculated by the following equation is used as the feature amount.

User-to-operator question ratio = user question count / operator question count Using the dialog content shown in FIG. 6 as an example, the question ratio is calculated to be “2 ÷ 2 = 1”, and the value (ratio) used as the feature amount is “1”.

  Next, “3. Timing of user and operator questions (timing)” will be described below.

  In the time series of the dialogue, the area where the user asks the most questions (time (timing)) depends on the amount of knowledge of the user.

  Specifically, in the case of a user with a small amount of knowledge, many questions are asked at the beginning of the dialogue, and in subsequent dialogues, the explanation of the operator is often heard from beginning to end. That is, a user's question with a small amount of knowledge often appears biased toward the beginning of the dialogue.

  On the other hand, users with a lot of knowledge often start conversations from explanations of their own requirements and situations, and because the number of questions that can be asked through the conversation is relatively small, questions are asked at specific locations in the conversation. It does not appear so much.

  Therefore, when the peak of the number of questions of the user appears at the beginning of the dialogue, it can be estimated that the amount of knowledge of the user is small.

  On the other hand, regarding the timing when the operator asks questions, for example, when the peak of the number of questions appears at the beginning, it is reflected that it is difficult to grasp the user's requirements and situation It can be estimated that the user who is the conversation partner of the operator has a small amount of knowledge.

  On the other hand, if the peak of the number of questions asked by the operator appears in the second half of the dialogue, the operator cannot answer the inquiry content that requires a high level of knowledge from the user who is the partner of the dialogue. Can be seen as answering questions.

  Therefore, when the peak of the number of questions of the operator appears in the second half of the dialogue, it can be estimated that the knowledge amount of the user who is the partner of the dialogue is large.

  As can be seen from the above example, the amount of knowledge of the user can be estimated by grasping the generation time (timing) of each question of the user and the operator in the dialogue.

  Furthermore, in this example, after capturing a dialogue time series in units of the number of utterances, it is extracted as a feature quantity of the timing (timing) when a question is uttered by two types of methods. The first type uses the number of questions in the first 10% of the dialogue as a feature amount. The second type divides the dialogue into five equal parts, and in which dialogue category the most questions appear. Is a feature quantity.

  Specifically, the value of the feature value corresponding to the dialogue category in which most questions appear is “1”, and the value corresponding to the remaining dialogue category is “0”.

  Taking the user's utterance content information shown in FIG. 7 as an example, the number of questions in the 10% section at the beginning of the dialogue is 3, and the value used as the feature value is “3”. Note that the user utterance content information shown in FIG. 7 is stored in a storage device such as the RAM 24 and the hard disk 28.

  Further, taking the utterance content information of the operator shown in FIG. 8 as an example, the most questions are uttered by the dialogue category (1), which corresponds to each dialogue category (1) to (5). The value of the feature amount to be “1”, “0”, “0”, “0”, “0” in order. Note that the utterance content information of the operator shown in FIG. 8 is stored in a storage device such as the RAM 24 and the hard disk 28.

  In addition, about the method of capturing the timing of utterance in the dialogue, in addition to the method of dividing the dialogue into several parts by the number of utterances as in this example, it may be divided into some parts by time information, It suffices as long as the time series of utterances can be expressed, such as how many seconds after the start of the conversation, how many utterances, or what percentage of the entire conversation is uttered.

  Accordingly, the plurality of question feature amounts (feature vector) used in this example are “1: operator question count”, “2: user question count”, “3: user to operator question ratio”, “ 4: “Number of user questions in the first 10% of the dialogue”, “5-9: timing of operator questions”, and “10-14: timing of user questions”.

  Note that the values of these feature vectors change depending on whether the operator is a veteran and has a high experience value, or is a newcomer and has a low experience value. For this reason, it is also effective to perform weighting according to the level of the experience value of the operator in advance.

  For example, if the operator is experienced and has a high experience value, the user can be asked questions accurately, and the number of questions from the user for the questions from the operator is reduced. In such a case, since the values of the question feature values of “1: operator's question count” and “2: user's question count” are both smaller than the other cases, the value of the question feature value. Similarly, since the value of the question feature value of “3: question ratio of user to operator” becomes small, it is possible to perform weighting so as to increase the value of the question feature value. . In the following description, in this example, it is assumed that such weighting is not performed.

  Next, an evaluation experiment example for the knowledge amount estimation apparatus 10 according to the present embodiment will be described. Here, the knowledge amount was estimated for each user role in a call at a plurality of contact centers, and an evaluation experiment was performed.

  Here, as the input to the knowledge amount estimation apparatus 10 according to the present embodiment, the speech recognition result of both the operator and the user of the call at each contact center is used, and the speech sequence obtained as the speech recognition result is used. The various question feature amounts described above are extracted using text data.

  In addition, as a correct label (knowledge amount information) used for evaluation of the estimation result of the knowledge amount for each call, each user in each call is classified as “small knowledge amount” or “large knowledge amount” in human subjective evaluation. Whether it is applicable or not is determined according to the criterion shown as criterion information in FIG. 9, and knowledge amount information (knowledge amount label) is assigned to each call. 9 is stored in advance in the hard disk 28, read from the hard disk 28, and stored in the RAM 24 for use.

  In this evaluation experiment example, 115 pairs of voice recognition results and knowledge amount labels of operators and users were prepared. Among them, 54 calls are given a “small knowledge” label and 61 calls are given a “high knowledge” label.

  The learning of the model that associates the knowledge amount label given to each call with the question feature amount extracted from each call, and the evaluation of the estimation of the knowledge amount are four of the data divided into five, and the remaining 1 This was carried out by 5-fold cross-validation in which five patterns of data sets were prepared by using one as evaluation data.

  In this example, the question feature used to estimate the knowledge amount is (i) the number of questions for the operator role, (ii) the number of questions for the user role, (iii) the question ratio of the user role to the operator role, (iv) The number of user role questions within 10% of the beginning, (v) the timing at which the operator role questions most frequently appear, and (vi) the timing at which the user role questions most frequently appear are six types. In addition, “Support Vector Machine (SVM)” was used for learning the correspondence between the question feature amount extracted from each call and the knowledge amount label, and for estimating the knowledge amount.

  On the other hand, in the accuracy evaluation of this example, “accuracy rate” that is an accuracy index is used as an evaluation index. This index indicates how many of the estimated results coincide with the correct knowledge amount label.

  In this way, there is a reason to place importance on the precision. That is, the most characteristic feature of the call at the contact center, which is the target of knowledge amount estimation in this example, is that the number of data is enormous. Considering that it is a major purpose of using the present invention to accurately narrow down the objects to be analyzed from the enormous amount of call data, it can be said that the relevance rate has an important meaning.

  In this evaluation example, it was regarded as a problem of estimating “small amount of knowledge”, and the precision was calculated using the following formula.

Relevance rate = number of “small amount of knowledge” that can be correctly estimated ÷ total number estimated as “small amount of knowledge” In this experimental example, in addition to this relevance rate, “reproducibility” and “ The F value, which is the harmonic average of the “reproducibility” and the “matching rate”, was also evaluated and used as a reference. The recall was calculated using the following formula.

Recall rate = the number of “small amount of knowledge” that can be estimated correctly ÷ the number of “small amount of knowledge” in all data. Further, the F value was calculated using the following equation.

  F value = (2 x precision x reproduction rate) / (precision + reproduction rate)

  In the 5-fold cross-validation, five learning data / evaluation data pairs are created from the data for all 115 calls, and learning and accuracy evaluation are performed. In this experimental example, the evaluation was performed by the micro average of the precision calculated from the estimation results of each of the five data.

  In the evaluation of this example, a label “small amount of knowledge” is given to 54 calls out of the prepared 115 data calls. Then, if the accuracy when the knowledge amount label of “small knowledge amount” is assigned to all 115 prepared calls is set as the baseline, in this case, the “accuracy rate” and “reproduction” in the above calculation method are set. The calculation results of “rate” and “F value” are “0.47”, “1.00”, and “0.64”, respectively, as shown in the evaluation result information of FIG.

  On the other hand, as a result of evaluating the performance using only (i) the number of questions of the operator role and (ii) the number of questions of the user role as the question feature amount, as shown in correspondence with the “number of times” in FIG. The calculation results of “Rate”, “Recall Rate”, and “F Value” are “0.64”, “0.43”, and “0.51”, respectively, and “Fit rate” is “0” compared to the baseline. .17 "improved.

  In addition to the “number of times”, (iii) as a result of evaluating the performance as a question feature amount together with the question ratio of the user role to the operator role, as shown in correspondence with “number of times + ratio” in FIG. ”,“ Reproducibility ”, and“ F value ”are“ 0.68 ”,“ 0.43 ”, and“ 0.52 ”, respectively. 21 "improved.

  Furthermore, together with “number of times” and “ratio”, (iv) the number of questions for the user role within 10% of the beginning, (v) the timing at which the questions for the operator role appear most frequently, and (vi) the question for the user role is the most As a result of evaluating the performance together with the frequently appearing timing as a question feature amount, as shown in correspondence with “number of times + ratio + timing” in FIG. 10, “accuracy rate”, “reproduction rate”, “F value” "0.70", "0.43", and "0.53", respectively, and the "precision" improved by "0.23" compared to the baseline.

  Thus, as shown in FIG. 10, even when only the “number of times” feature quantity is used, the precision is improved by 0.17 compared to the baseline, and all the question feature quantities proposed in this example are obtained. In the case of “number of times + ratio + timing” used, the precision was improved by 0.23 compared to the baseline. When the F value is compared, the proposed technique of this example has a decrease of 0.11 to 0.13 compared to the baseline. However, this technique can improve the relevance rate that is important in call analysis. An example is useful.

  In addition, in order to investigate whether there is a difference in the correct / incorrect answer ratios of the estimation results using each question feature, we performed the “McNemar's Test”. It was shown that there was a significant difference (p <0.05) between the estimation result using “number of times + ratio + timing” and the baseline estimation result.

  In this evaluation example, an example using only the number of questions of the inquirer and the respondent, an example of using this number of questions, and a combination of the ratio of the number of questions of the inquirer and the number of questions of the respondent, and these In addition to the number of questions and the ratio, the example in which the occurrence time in the text data of the inquirer and the respondent's question is used is shown. For example, only the ratio between the number of questions of the inquirer and the number of questions of the respondent Or using only the time of occurrence in the text data of the question of the inquirer and the respondent, or the occurrence of the number of questions of the inquirer and the number of questions of the respondent, and the text data of the question of the inquirer and the respondent You may make it perform learning, estimation, and evaluation using any one of each, or a combination, such as using a combination of times.

  As described above, the knowledge amount estimation apparatus 10 according to the present embodiment uses the “question” in the dialogue between the operator and the user in the contact center or the like, thereby obtaining the knowledge amount of the speaker (user). Is estimated.

  Specifically, as a function implemented by programmed computer processing, at least a question extraction unit 2, a feature amount extraction unit 3, an estimation information generation unit 4, and a knowledge amount estimation unit 5 are provided. A question utterance sequence, which is an utterance sequence of questions to the other party, is extracted from the text data of the utterance sequence in the dialogue between the inquirer and the respondent, and the feature quantity extraction unit 3 extracts the question utterance sequence extracted by the question extraction unit 2 The question feature quantity indicating the feature quantity related to the occurrence state of the question utterance string in the text data is extracted based on the question feature quantity, and the estimation information generation unit 4 extracts the question feature quantity extracted by the feature quantity extraction unit 3 and the question feature quantity Knowledge amount information given in advance to indicate the knowledge amount of the inquirer assumed from the dialogue indicated by the text data for the text data to be extracted It is used to generate an estimate information used to estimate the amount of knowledge corresponding to the text data.

  The estimation information generated by the estimation information generation unit 4 in this way is stored in a storage device such as the hard disk 28.

  In addition, the question extraction unit 2 extracts a question utterance sequence, which is a utterance sequence of questions to the other party, from text data of an utterance sequence in a dialogue between an inquirer and a respondent whose estimation is a knowledge amount, and extracts a feature amount The unit 3 extracts a question feature amount indicating a feature amount related to the occurrence state of the question utterance sequence in the text data based on the question utterance sequence extracted by the question extraction unit 2, and the knowledge amount estimation unit 5 extracts the feature amount. The knowledge amount corresponding to the text data to be estimated is estimated using the question feature amount extracted by the unit 3 and the estimation information stored in advance in the storage device.

  Here, the question feature amount includes the number of questions of the inquirer, the number of questions of the respondent, the ratio of the number of questions of the inquirer and the number of questions of the respondent, the occurrence time in the text data of the question of the inquirer, and the respondent The date of occurrence in the text data of the question.

  As described above, in this example, the “question” in the dialogue is used to estimate the knowledge amount of the speaker from the dialogue, and in the case where the speech recognition result is handled as the input of the knowledge amount estimation in the prior art. The problem can be avoided.

  In other words, when speech recognition results using conventional vocabulary expertise are treated as input for estimation of the amount of knowledge, when some highly specialized words appear erroneously in the speech recognition results of a speaker's utterance However, in this example, using the “question” in the dialogue, the knowledge amount of the speaker is estimated from the dialogue. Such problems can be avoided.

  Thereby, by using the knowledge amount estimation apparatus 10 according to the present embodiment, it is possible to accurately estimate the knowledge amount of the speaker (inquirer) for the item that is the subject of the dialogue based on the dialogue result. it can.

  In addition, this invention is not limited to the example mentioned above, A various deformation | transformation and application are possible within the range which does not deviate from the summary of this invention.

  For example, in this example, the dialogue (call) between the user and the operator in the contact center has been described as an example of the dialogue to be processed. However, the dialogue is not limited to this, for example, in the non-patent document 2 bus operation information guidance system described above. The present invention can be applied to a system that estimates a user's knowledge amount in a dialog using other speech recognition, such as a dialog with a responder to a query from a user.

  The dialogue may be any system (respondent) versus human (inquirer), human (respondent) versus human (inquirer), or any conversation between two parties.

  The text data of the dialog may be anything as long as it is a text-converted dialog, such as a speech recognition result of the dialog, a transcription, or a text chat.

  In the computer configuration shown in FIG. 2, a program for realizing the function of each processing unit according to the present invention is recorded on a computer-readable recording medium, and the program recorded on the recording medium is stored in a computer system. By reading and executing, the processing by each configuration may be executed, or the program may be read by using a communication function not shown.

  The computer-readable recording medium refers to a portable medium such as a flexible disk, a magneto-optical disk, a ROM, and a CD-ROM, and a storage device such as a hard disk built in the computer system.

  The program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line.

  The program may be for realizing a part of the functions described above. Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, what is called a difference file (difference program) may be sufficient.

  As described above, the embodiment for carrying out the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and the scope of the present invention is not deviated. Design etc. are also included.

DESCRIPTION OF SYMBOLS 1 Speech sequence extraction part 2 Question extraction part 3 Feature-value extraction part 4 Estimated information generation part 4a Question feature-value 4b Knowledge-amount information (knowledge-amount label)
4c Model generation function 5 Knowledge amount estimation unit 5a Model information 5b Feature amount 5c of call i Collation function 6 Speech recognition result of call i (estimation target)
7 Speech recognition results (call for learning)
8 Knowledge amount of speaker (user) of call i 10 Knowledge amount estimation device 14 Keyboard 16 Mouse 18 Display 22 CPU
24 RAM
26 ROM
28 Hard Disk 30 External I / F (Interface)
50 Communication equipment

Claims (6)

A question extraction means for extracting a question utterance string, which is an utterance string of a question to the other party, from text data of an utterance string in the dialogue between the inquirer and the respondent;
Based on the question utterance string extracted by the question extraction means, feature quantity extraction means for extracting a question feature quantity indicating a feature quantity related to the occurrence state of the question utterance string in the text data;
The question feature amount extracted by the feature amount extraction means and the knowledge amount of the inquirer assumed from the dialogue indicated by the text data with respect to the text data targeted for extraction of the question feature amount Knowledge amount estimation information generating means for generating estimation information used for estimation of the knowledge amount corresponding to the text data by using knowledge amount information given in advance as learning data;
A knowledge amount estimation information generating apparatus comprising: Storage means for storing in advance the estimation information generated by the knowledge amount estimation information generation device according to claim 1;
A question extraction means for extracting a question utterance string, which is an utterance string of a question to the other party, from text data of an utterance string in a dialogue between an inquirer and a respondent who is an estimation target of a knowledge amount;
Based on the question utterance string extracted by the question extraction means, feature quantity extraction means for extracting a question feature quantity indicating a feature quantity related to the occurrence state of the question utterance string in the text data;
Knowledge amount estimation means for estimating the knowledge amount corresponding to the text data to be estimated using the question feature amount extracted by the feature amount extraction means and the estimation information stored in the storage means;
A knowledge amount estimation device. The question feature amount is the number of questions of the inquirer, the number of questions of the respondent, the ratio of the number of questions of the inquirer and the number of questions of the respondent, the occurrence time of the question of the inquirer in the text data, The knowledge amount estimation information generation device according to claim 1, comprising: at least one of occurrence time of the question of the respondent in the text data. A question extraction step of extracting a question utterance sequence that is a utterance sequence of a question to the other party from text data of an utterance sequence in the dialogue between the inquirer and the respondent;
Based on the question utterance sequence extracted in the question extraction step, a feature amount extraction step for extracting a question feature amount indicating a feature amount relating to the occurrence state of the question utterance sequence in the text data;
The question feature quantity extracted in the feature quantity extraction step and the knowledge quantity of the inquirer assumed from the dialogue indicated by the text data with respect to the text data targeted for extraction of the question feature quantity A knowledge amount estimation information generation step for generating estimation information used for estimation of the knowledge amount corresponding to the text data by using knowledge amount information given in advance as learning data;
A knowledge amount estimation information generation method comprising: A storage step for storing the estimation information generated by the knowledge amount estimation information generation method according to claim 4 in a storage device;
A question extraction step for extracting a question utterance sequence, which is a utterance sequence of a question to the other party, from text data of an utterance sequence in the dialogue between the inquirer and the respondent as an estimation target of the knowledge amount;
Based on the question utterance sequence extracted in the question extraction step, a feature amount extraction step for extracting a question feature amount indicating a feature amount relating to the occurrence state of the question utterance sequence in the text data;
A knowledge amount estimation step for estimating the knowledge amount corresponding to the text data to be estimated using the question feature amount extracted in the feature amount extraction step and the estimation information stored in the storage step;
A knowledge amount estimation method comprising:   A program for causing a computer to function as the knowledge amount estimation information generation device according to claim 1 or the knowledge amount estimation device according to claim 2.

Images