JP5574842B2 - FAQ candidate extraction system and FAQ candidate extraction program - Google Patents

Description

  The present invention relates to a natural language processing technique, and more particularly, to a FAQ candidate extraction system and a FAQ candidate extraction program for extracting FAQ (Frequently Asked Questions) candidates from the result of analyzing the structure of discourse data obtained by speech recognition processing. It is related to effective technology when applied.

  For example, in a call center operation, an operator or a communicator needs to perform a series of operations such as creating a response record and storing it in a response management system after receiving a call. At this time, there is a problem that it takes time for the operator or the like to create a response record based on his / her memory.

  Further, quality (content, uniformity, completeness, conciseness, etc.) varies according to the skill of each operator. In this way, the quality of the response records varies or the quality is low, so it is not possible to accurately grasp the customer's voice, or from the accumulation of response records, new technologies such as text mining are used. Problems such as inability to obtain knowledge also occur.

  In order to solve such a problem, a solution using a speech recognition technology is being studied. By storing the voice information of the response as text using the speech recognition technology, it is possible to ensure the completeness of the response record. It is also possible to automatically create response records by applying natural language processing technology to the text data and summarizing the contents, reducing the cost of response record creation, uniformity, It is possible to improve the usability of response records by ensuring conciseness.

  As such a technique, for example, in Non-Patent Document 1, in voice conversation at a call center, voice data is converted into text, and rules are recursively obtained by inductive learning from past conversation contents and business daily reports. It describes a technology that automatically generates daily business reports by determining and extracting important points for unknown dialogue contents using rules and performing style conversion.

  On the other hand, there are various technologies that can be used to obtain new knowledge through analysis by text mining or the like from the accumulation of voice data that has been converted into text, depending on the type of voice data interaction. From the contents of the response, it is assumed that frequently asked questions are automatically extracted, and answers (FAQs) are prepared to prepare FAQ (Frequently Asked Questions) efficiently and effectively.

  Conventionally, a question-answer pair to be used as a FAQ has been extracted from a set of data classified and stored in advance as questions and answers. In addition, frequently asked questions and answers are extracted from text data of response records by matching with specific patterns or extracted patterns and statistical processing. For example, in Japanese Patent Laid-Open No. 2001-134575 (Patent Document 1), for a large amount of text data, an FAQ is automatically created by detecting an important pattern having a high appearance frequency from a set of data structures obtained by parsing the text data. The technology to be enabled is described.

  Further, for example, in Non-Patent Document 2, for a television program with closed captions that has been increasing in recent years, a question sentence is identified using CRF (Conditional Random Fields) from among closed captions in the television program. Describes a technique for extracting a question-answer pair by estimating the center of an answer sentence corresponding to, and further identifying information for introducing and supplementing the answer sentence using CRF.

JP 2001-134575 A

Junji Yano, Kenji Araki, “Performance evaluation of daily report generation method using inductive learning for voice conversation in call center”, Information Processing Society of Japan Research Report 2007-NL-178, March 28, 2007, p. 21-28 Masahiro Kawano and 6 others, “Question-answer pair automatic extraction from TV program closed caption using CRF”, Proc. Of the 14th Annual Conference of the Language Processing Society, March 20, 2008, p. 915-918

  When using data obtained by converting a person's discourse into text by speech recognition (hereinafter sometimes referred to as “discourse data”), the sentences in the discourse data are usually recorded in spoken language, so follow the correct grammar. Japanese language sentences and existing natural language processing technology based on written language may not be able to cope well. Moreover, the accuracy of the current speech recognition engine cannot eliminate recognition errors, and there are many noises such as typographical errors and omissions due to erroneous recognition. Therefore, it is difficult to analyze and effectively use discourse data using natural language processing.

  On the other hand, in order to obtain new knowledge such as a question sentence that is a candidate for FAQ from the accumulation of voice data that has been converted into text, for example, pattern extraction and matching as well as statistical techniques such as the technique of Patent Document 1 and the like are used. It may be possible to extract frequently asked questions and answers by processing. However, these technologies are based on information such as the frequency of appearance of words and syntax patterns, and the meaning content of each sentence of discourse data (not only having a specific expression pattern, but also from past sentences). This does not take into account the meaning that is determined in consideration of the flow and development of Therefore, depending on the content of the data, for example, it may be assumed that it is not possible to sufficiently extract only the sentence corresponding to the question with high accuracy and efficiency.

  In the technique of Non-Patent Document 2 described above, a question-answer pair can be extracted from a closed caption of a television program. Here, the content of the target closed caption is annotated in the schema, that is, specified in advance. It is applied to the framework of the sentence structure, and questions and answers are extracted based on this. However, for example, it is possible to identify and extract a question sentence from discourse data that cannot be applied to the framework or schema of a predetermined sentence structure, such as text data of telephone reception at a call center. It is difficult with the technique as described in Non-Patent Document 2.

  Therefore, the object of the present invention is strong in the characteristics of discourse data such as spoken words and noise, and based on the information on the semantic content obtained from the result of analyzing the structure of discourse data without prescribing the framework of the discourse sentence structure, An object of the present invention is to provide a FAQ candidate extraction system and a FAQ candidate extraction program for extracting question sentences as FAQ candidates from discourse data. The above and other objects and novel features of the present invention will be apparent from the description of this specification and the accompanying drawings.

  Of the inventions disclosed in this application, the outline of typical ones will be briefly described as follows.

  The FAQ candidate extraction system according to a representative embodiment of the present invention is a discourse data in which the content of a discourse is converted into text by speech recognition processing, and an analysis result of the discourse structure in the discourse data. A FAQ candidate extraction system that inputs discourse semantics, which is information for grasping meaning content, and extracts and outputs a question sentence as a FAQ candidate from the discourse data, and has the following characteristics.

  That is, in the FAQ candidate extraction system, the discourse semantics includes flow information including flow information indicating the semantic content of each statement in the discourse data, and the FAQ candidate extraction system is uttered by the customer from the discourse data. A question extraction unit that extracts a question / request statement in which a flow indicating that it is a question sentence or a request sentence in the flow information of the discourse semantics, and the question / request statement extracted by the question extraction unit, A candidate extraction unit that extracts a keyword including the specified keyword, and the question / request statement that is representative of each cluster by clustering the question / request statements extracted by the candidate extraction unit into the same contents As a FAQ candidate It is characterized in that it has a clustering unit for outputting.

  The present invention can also be applied to a program that causes a computer to function as the FAQ candidate extraction system as described above.

  Among the inventions disclosed in the present application, effects obtained by typical ones will be briefly described as follows.

  According to the exemplary embodiment of the present invention, the semantic content is obtained from the result of analyzing the structure of discourse data, without having to define the framework of the discourse sentence structure, strong in the characteristics of discourse data such as spoken words and noise. Based on such information, it is possible to extract a question sentence as a FAQ candidate from the discourse data, and it is possible to support the creation of an efficient and effective FAQ.

It is the block diagram shown about the outline | summary of the structural example of the FAQ candidate extraction system which is one embodiment of this invention. It is the block diagram shown about the outline | summary of the structural example of the discourse structure analysis system in one embodiment of this invention. It is the figure shown about the example of the discourse in one embodiment of this invention, and the concept of discourse semantics. It is the figure shown about the flow of discourse data and the example of a context in one embodiment of this invention. It is the figure shown about the example of the flow analysis rule of the discourse structure analysis system in one embodiment of this invention. It is the flowchart which showed the example of the flow of the flow analysis process in the flow analysis part of the discourse structure analysis system in one embodiment of this invention. It is the figure shown about the example of the specific expression analysis rule of the discourse structure analysis system in one embodiment of this invention. It is the flowchart which showed the example of the flow of the specific expression analysis process in the specific expression analysis part of the discourse structure analysis system in one embodiment of this invention. It is the figure shown about the example of cohesion analysis in the discourse structure analysis system in one embodiment of this invention. It is the flowchart which showed the example of the flow of the cohesiveness analysis process in the cohesiveness analysis part of the discourse structure analysis system in one embodiment of this invention. It is the figure shown about the example of the screen which displays the extracted FAQ candidate in one embodiment of this invention. It is the flowchart which showed the example of the flow of the FAQ candidate extraction process in the FAQ candidate extraction system in one embodiment of this invention.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. Note that components having the same function are denoted by the same reference symbols throughout the drawings for describing the embodiment, and the repetitive description thereof will be omitted.

  The FAQ candidate extraction system according to one embodiment of the present invention is based on discourse semantics (information related to discourse structure for grasping the semantic content of the entire discourse), which is the analysis result of discourse structure for discourse data including spoken words. Then, the sentence corresponding to the question from the customer is extracted from the discourse data, and the importance based on the appearance frequency of the word is calculated. Furthermore, by extracting questions that include a specified keyword from a set including question sentences in an existing FAQ, and clustering the extracted questions into similar ones, the priority (importance) of answer creation is high The system sequentially outputs the FAQ candidates including the information on the completed / unanswered answers.

  Here, discourse semantics means, for example, a flow as described below for the entire discourse data by inferring the semantic content from character strings and morphemes in the discourse data, and also a discourse such as cohesiveness (discourse segment), proper expression, etc. It is the analysis result obtained by analyzing the structure. This discourse semantics may be generated from discourse data by using a discourse structure analysis system, which will be described later, or may be generated using a sentence structure analysis system using other natural language processing, etc. . It is also possible to manually create a tag by tagging the discourse data. In the present embodiment, description will be made assuming that the generation is performed using a discourse structure analysis system described later.

  The discourse semantics in this embodiment do not have to be the result of strict analysis of the content and structure of Japanese sentences. For example, the discourse semantics have a level of accuracy that does not impede practical use, such as creating a response record at a call center. I just need it. In this specification, “discussion” refers to an utterance action by one or more speakers, for example, a speech at a lecture where one person speaks, a call center operator as described above, and a customer. Includes communication, multi-person conference calls and video conferencing, and other multi-person conversations and dialogues.

  FIG. 3 is a diagram showing an example of discourse and the concept of discourse semantics in the present embodiment. The example of FIG. 3 shows an example of a discourse consisting of interaction between a call center operator and a customer. The utterance on the left side is the utterance of the operator, and the right side shows the utterance of the customer. In the present embodiment, one sentence of each speaker is called a “statement”. In addition, a group of consecutive statements for each speaker is referred to as a “block” (even if consecutive statements of the same speaker are separated in time, they become different blocks).

  In FIG. 3, “flow” is information that simply indicates the semantic content of each statement. By tracing this information, the flow of discourse can be grasped. For example, in the operator statement of FIG. 3, the first “Yes.” Indicates that the flow is “greeting”, while the next “Yes.” Indicates that the flow is “confusion”. In this way, depending on the discourse situation (context), the flow may be different even for statements of the same wording.

  In FIG. 3, “proprietary expression” is an expression that individually characterizes the content of the discourse. For example, in the discourse of exchange at the call center in FIG. 3, the speaker ID, person name, company name, telephone number, Applicable to proper nouns such as location. Further, the “segment” is a segmented set of common (united) contents of the discourse for each block. The information about the above “flow”, “specific expression”, and “segment” is a constituent element of “Discourse Semantics” as the analysis result of the discourse structure.

[System configuration (Discourse structure analysis system)]
FIG. 2 is a block diagram showing an outline of a configuration example of the discourse structure analyzing system according to the embodiment of the present invention. The discourse structure analysis system 2 is configured by a computer system such as a server or a PC (Personal Computer). For example, the preprocessing unit 10, the flow analysis unit 20, the morpheme analysis unit 30, and the proper expression analysis unit 40 are implemented by software programs. The discourse structure having each unit of the cohesiveness analysis unit 50, each table of the flow analysis rule 22 and the specific expression analysis rule 42, and the discourse data 100 in which the content of the discourse is converted into text by the speech recognition engine 3 is input. And the discourse semantics 200 is output.

  Note that the existing voice recognition engine 3 can be used. As described above, the discourse structure analysis system 2 according to the present embodiment is characterized by being resistant to noise (such as recognition errors by the speech recognition engine 3) of the discourse data 100, but the recognition rate of the speech recognition engine 3 is higher. Needless to say, it is possible to analyze the discourse structure with higher accuracy.

  The preprocessing unit 10 of the discourse structure analysis system 2 has a function of receiving input of the discourse data 100 and performing preprocessing such as data layout conversion so that processing can be performed in other units as necessary. . Here, for example, continuous statements for each speaker are grouped into blocks, sequence numbers (IDs) indicating the order are assigned to each block and each statement, and speaker data of each block is extracted and set. Perform the process. The discourse data 100 is represented as a data structure using, for example, XML (eXtended Markup Language).

  The flow analysis unit 20 has a function of analyzing a rule-based flow based on the flow analysis rule 22 on the discourse data 100 after the preprocessing by the preprocessing unit 10 and outputting the flow information 21 as an analysis result. . The flow information 21 holds the ID of each statement in the discourse data 100 and the flow information corresponding thereto, and is represented as a data structure using XML or the like, for example.

  This flow information 21 is included in the discourse semantics 200 as an output of the discourse structure analysis system 2 and is an essential component. That is, the discourse semantics 200 includes flow information 21 as a flow analysis result for at least the discourse data 100. Details of the flow analysis rule 22 and details of processing in the flow analysis unit 20 will be described later.

  The morpheme analysis unit 30 performs a so-called morpheme analysis on the discourse data 100 after the preprocessing by the preprocessing unit 10 and divides the discourse data into columns of morphemes (the smallest unit having meaning in the language) to determine the part of speech. As a result, the morpheme information 31 is output. The morpheme information 31 holds the ID of each statement in the discourse data 100 and morpheme string information corresponding thereto, and is represented as a data structure using, for example, XML.

  Similar to the flow information 21, the morpheme information 31 is included in the discourse semantics 200 as an output of the discourse structure analysis system 2, and becomes an input to the specific expression analysis unit 40 and the cohesiveness analysis unit 50 described later. In other words, when the specific expression analysis by the specific expression analysis unit 40 and the cohesiveness analysis (segmentation) by the cohesiveness analysis unit 50 are not performed, the morpheme analysis by the morpheme analysis unit 30 may be unnecessary. In the present embodiment, since the morpheme information 31 is used in the FAQ candidate extraction system described later, the morpheme analysis by the morpheme analysis unit 30 is essential. Note that an existing morpheme analysis engine (for example, MeCab (Japanese cloth candy) or ChaSen (tea candy)) can be used for the morpheme analysis unit 30.

  The specific expression analysis unit 40 is a rule-based specific expression based on the specific expression analysis rule 42 based on the discourse data 100 after the preprocessing by the preprocessing unit 10 and the morpheme information 31 output from the morpheme analysis unit 30. And the specific expression information 41 is output as an analysis result. That is, the analysis process in the specific expression analysis unit 40 is based on the premise that at least morpheme analysis by the morpheme analysis unit 30 is performed. The specific expression information 41 holds information of each statement ID in the discourse data 100 and a list of specific expressions corresponding to the statement ID, and is represented as a data structure using, for example, XML.

  The specific expression information 41 is included in the discourse semantics 200 as an output of the discourse structure analysis system 2, as with the flow information 21, but the usage form by the application that uses the discourse semantics 200 thereafter and the required discourse semantics are included. It may be unnecessary depending on accuracy. Therefore, the output of the specific expression information 41 is not essential, and the user may be able to select whether or not to output. Details of the specific expression analysis rule 42 and details of the processing in the specific expression analysis unit 40 will be described later.

  The cohesiveness analysis unit 50 determines each block based on the discourse data 100 after the preprocessing by the preprocessing unit 10, the flow information 21 output from the flow analysis unit 20, and the morpheme information 31 output from the morpheme analysis unit 30. The cohesiveness is analyzed and segmented, and the cohesiveness information 51 is output as an analysis result. That is, the analysis processing in the specific expression analysis unit 40 is premised on that at least morpheme analysis by the morpheme analysis unit 30 and flow analysis by the flow analysis unit 20 are performed. The cohesiveness information 51 holds segment ID information corresponding to each block ID in the discourse data 100, and is represented as a data structure using XML or the like, for example.

  The cohesiveness information 51 is included in the discourse semantics 200 as an output of the discourse structure analysis system 2 as in the case of the specific expression information 41. However, the usage form by the application using the discourse semantics 200 after that and the required discourse semantics are included. Depending on the accuracy, etc., it may be unnecessary. Therefore, the output of the cohesiveness information 51 is not essential, and the user may be able to select whether or not to output. Details of processing in the cohesiveness analysis unit 50 will be described later.

  As described above, the discourse semantics 200 output as the analysis result by the discourse structure analysis system 2 includes the flow information 21, the morpheme information 31, the unique expression information 41, and the cohesiveness information 51. At this time, a set of these data may simply be handled as the discourse semantics 200, or these data may be combined into one data structure using XML, for example.

[System configuration (FAQ candidate extraction system)]
FIG. 1 is a block diagram showing an outline of a configuration example of a FAQ candidate extraction system according to an embodiment of the present invention. The FAQ candidate extraction system 1 is configured by a computer system such as a server or a PC, similar to the discourse structure analysis system 2, and includes, for example, a question extraction unit 60, a scoring unit 70, a candidate extraction unit 80, and a software program. Each part of the clustering unit 90 and an existing FAQ 82 table, the discourse data 101 in which the content of the discourse is converted into text, and all or part of the discourse semantics 200 that is the analysis result of the discourse structure of the discourse data 101 (particularly This is a system that receives flow information 21 and morpheme information 31), extracts a question sentence that is a FAQ candidate from the discourse data 101, and outputs it as a FAQ candidate 300.

  Note that the discourse data 101 is equivalent to that preprocessed by the preprocessing unit 10 of the discourse structure analysis system 2 described above, that is, for example, continuous statements for each speaker are collected as blocks. A sequence number (ID) indicating an order is assigned to each statement, and the speaker data of each block is set. The discourse semantics 200 is output as a result of analyzing the discourse structure (flow, specific expression, cohesiveness, and morpheme) of the discourse data 101 by the discourse structure analysis system 2 described above, for example. is there.

  In the present embodiment, the analysis result of the discourse structure output by the discourse structure analysis system 2 is input to the FAQ candidate extraction system 1. For example, these systems are configured on one computer system. The processing from the discourse structure analysis processing to the FAQ candidate 300 extraction may be executed continuously.

  The question extraction unit 60 of the FAQ candidate extraction system 1 has a function of extracting a question statement by a customer from the discourse data 101 based on the discourse data 101 and the flow information 21 of the discourse semantics 200. The scoring unit 70 has a function of calculating the degree of importance for each statement extracted by the question extraction unit 60 based on the appearance frequency information of the included words and outputting the question information 71, for example. The question information 71 holds a statement of a question by a customer or a list of statement IDs and importance lists.

  The candidate extraction unit 80 has a function of extracting a question statement or a question sentence including the keyword 81 from a set including a question sentence of the existing FAQ held in the existing FAQ 82 in the question statement included in the question information 71 Have Here, the keyword 81 may be specified by an input from the user, or may be acquired and specified, for example, the latest rapid keyword extracted by the FAQ candidate extraction system 1 or another system. The clustering unit 90 clusters the question statements and the question sentences extracted by the candidate extraction unit 80 into the same ones, and collects the question statements that are representative of each cluster (not the existing FAQ question sentences). , And a function of outputting the FAQ candidate 300 in the order of importance calculated by the scoring unit 70.

  In the present embodiment, as described above, it is assumed that the FAQ candidates with higher importance have higher priority to be prepared and maintained. If there is a question sentence held in the existing FAQ 82 in the cluster to which the FAQ candidate belongs, the answer sentence corresponding to the existing FAQ 82 has already been registered, and thus the answer has been prepared for the FAQ candidate. Treat as. Details of processing by each unit of the FAQ candidate extraction system 1 will be described later.

[Flow analysis]
Below, the detail of the process of the flow analysis by the flow analysis part 20 of the discourse structure analysis system 2 shown in FIG. 2 is demonstrated. FIG. 4 is a diagram illustrating an example of a flow and context in the discourse data 100. In the present embodiment, a context is set for each block of discourse. In the example of FIG. 4, the context of the first block (speaker: operator) is “start”, and the next block (speaker: customer) and the next block (speaker: operator) are both “declaration”. It has become.

  As shown in the example of FIG. 4, in this embodiment, in principle, the flow set in the last statement of the block immediately before a different speaker is set as the context of the target block. Since there is no immediately preceding block for the first block of the discourse data 100, “start” is always set as the context. By determining the context in this way, it is possible to analyze the flow along the discourse context even if the statements have the same wording.

  FIG. 5 is a diagram illustrating an example of the flow analysis rule 22 of the discourse structure analysis system 2. The flow analysis rule 22 is held in a file, database, or the like, and includes an expression for determining a flow in a statement (hereinafter sometimes referred to as “flow expression”) and an expression that matches the flow expression. It is a table that holds a list designated in advance with respect to correspondence with a flow pattern (hereinafter, sometimes referred to as “flow definition”) set in accordance with the context of a statement it has.

  As shown in the example of FIG. 5, the flow expression is specified by a regular expression of a character string (for example, “/Yes./” or “/ ?? /”) that serves as a clue for determining the flow. Accordingly, the flow analysis unit 20 or the discourse structure analysis system 2 needs to be able to perform processing necessary for regular expressions by introducing, for example, an existing regular expression library.

  Note that the character string specified as the flow expression should preferably specify, for example, a greeting at the beginning of a sentence, a reply, a particle, a conjunction, a sentence ending expression, etc. Specify in descending order. As for the priority, for example, the complexity of the regular expression may be calculated according to a predetermined condition, and the priority may be set in descending order of the complexity. Note that the flow expression pattern to be specified may differ depending on the type of discourse targeted by the discourse structure analysis system 2 (for example, exchange at a call center or regular meeting in a company).

  For each flow expression, for example, a flow definition is specified by a label “flow”. In the flow definition, set “greeting” as the flow if the context is “start” in the statement that matches the context of the target statement and the flow that is set in the statement (for example, “/Yes./”) Specify). A plurality of flow definitions by the label “flow” can be specified according to the type of context assumed in each flow expression.

  In addition, for example, a default flow (for example, a flow expression of “/Yes./”, which is a flow to be set when the context of the target statement does not correspond to any “flow” label by the label “default”. In the statement that matches, if the context of the statement does not match any of the “flow” labels, the flow is set to “contrast”. Note that one default flow is always set in each flow expression.

  For the flow definition, it is desirable to specify a main pattern as a general discourse flow pattern according to the type of discourse targeted by the discourse structure analysis system 2. For example, in the exchange at the call center, the flow definition can be specified based on a pattern such as “apology” if the customer makes a “complaint”.

  In the flow analysis rule 22, as described above, the flow expression to be specified and the content of the corresponding flow definition may differ depending on the type of discourse targeted by the discourse structure analysis system 2. A plurality of flow analysis rules 22 to be applied can be held so that the user can be switched, so that it is possible to flexibly cope with the characteristics of the discourse.

  As described above, in this embodiment, the part of speech is not specified in the flow analysis rule 22. That is, since the flow analysis unit 20 does not determine the part of speech, it is not necessary to perform the morpheme analysis by the morpheme analysis unit 30 in advance in the analysis process in the flow analysis unit 20. In order to improve accuracy, the morpheme information 31 is used, for example, a regular expression in the flow expression of the flow analysis rule 22 is applied to a morpheme sequence (including part-of-speech information) of the morpheme information 31 as described later. It is also possible to perform matching under more detailed conditions as applied.

  FIG. 6 is a flowchart showing an example of the flow of flow analysis processing in the flow analysis unit 20 of the discourse structure analysis system 2. When starting the flow analysis process, the flow analysis unit 20 first starts a loop process that repeats the process for each statement of the discourse data 100 after the preprocessing by the preprocessing unit 10 (S101). When loop processing for each statement is started, loop processing for repeating processing for each flow expression of the flow analysis rule 22 is started (S102).

  When the loop processing for each flow expression is started, it is determined whether or not the target statement matches the target flow expression (S103). If there is no match, the process proceeds to the next flow expression process (S108, S102). On the other hand, if there is a match, first, for the target statement, the flow set in the last statement in the block immediately before the block to which the statement belongs is set as a context (S104). At this time, as shown in FIG. 4, when the block to which the statement belongs is the first block of the discourse data 100, “start” is set in the context.

  Next, with respect to the context set in step S104, it is determined whether or not there is a context that matches the context specified by the “flow” label of the flow definition in the target flow expression (S105). If there is a match, the flow corresponding to the context in the matching flow definition is set as the flow of the target statement (S106). On the other hand, if there is no match, the default flow is set as the flow of the target statement (S107). Thereafter, the loop processing for each flow expression is exited.

  When the loop processing for each flow expression is completed, it is determined whether or not a flow is set in the target statement (S109). If the flow is set, nothing is performed. If the flow is not set, the “statement” flow is set in the target statement (S110), and the process proceeds to the next statement (S111). , S101).

  Here, the “statement” flow is set when the target statement does not include a word that matches the flow expression specified in the flow analysis rule 22. In other words, it is not a statement (statement that includes a flow expression) that corresponds to a predetermined pattern or a typical pattern in the framework of discourse, but generally a statement that describes specific details about matters specific to discourse. It is believed that there is.

  When all the loop processes for each statement are completed, the ID of each statement and the flow information corresponding thereto are output as the flow information 21 and the flow analysis process is terminated.

  In this way, flow analysis that considers spoken language is used to match each statement, and by setting an appropriate flow according to the context of the matched statement, flow analysis that is less susceptible to spoken language and noise for the entire discourse data 100 is performed. It can be carried out.

[Specific expression analysis]
Hereinafter, the details of the specific expression analysis processing by the specific expression analysis unit 40 of the discourse structure analysis system 2 shown in FIG. 2 will be described. FIG. 7 is a diagram showing an example of the specific expression analysis rule 42 of the discourse structure analysis system 2. The specific expression analysis rule 42 is held in a file, a database, or the like, and a regular expression for specifying a specific expression in a statement and a rule for specifying and extracting a specific expression from a portion that matches the regular expression It is a table that holds a list designated in advance for correspondence with (hereinafter may be described as “specific expression rules”).

  As shown in the example of FIG. 7, the specific expression in the statement is specified by a regular expression. Therefore, the specific expression analysis unit 40 or the discourse structure analysis system 2 needs to be able to perform processing necessary for regular expressions by introducing, for example, an existing regular expression library. In the specific expression analysis, it is necessary to determine not only the regular expression for the normal character string but also the expression of the morpheme string of each statement held in the morpheme information 31. It is expanded so that the information can be judged.

  For example, in the example of FIG. 7, the range of the morpheme string is specified by “()” in each regular expression, and this part is matched with the information of the morpheme string instead of the character string. In the first regular expression example, the part of “(<f: organization> +)” indicates one or more morpheme sequences analyzed that the part of speech (f) is “organization”. (?: Of)? "Indicates a part of speech of 0 or more and 1 or less analyzed that the surface shape is" no ". The part of “(<f: person name, surname>)” indicates one morpheme string that has been analyzed with the part of speech being “person name, surname”. This regular expression matches expressions such as “AAA Shoji Suzuki” and “AAA Shoji Suzuki”. Note that the representation of the morpheme string may vary depending on the output specifications of the morpheme analysis engine or the like in the morpheme analysis unit 30.

  At this time, in the corresponding specific expression rule, the first element in the morpheme string (the morpheme string whose part of speech is “organization”, for example, “AAA Shoji”) is specified as a specific expression having the attribute “company name”. The second element in the column (the morpheme string whose part of speech is “person name, surname”, for example “Suzuki”) is specified to be specified as a specific expression having the attribute “person name”. It should be noted that a portion (for example, “no”) enclosed by “(?:)” Is treated as not included in the element of the target morpheme string.

  In the example of the second regular expression of FIG. 7, the part of speech is specified by a normal regular expression without particular consideration, and matches the expression of a telephone number such as “03-1234-5678”. At this time, in the corresponding specific expression rule, the first element in the morpheme string (in this example, there is only one element such as “03-1234-5678”) is defined as a specific expression having the attribute “phone number”. It is specified to be specified.

  It is desirable that the character string of the unique expression specified by the regular expression is, for example, a proper noun that individually characterizes the content of the discourse or a character string made up of numerical values. In addition, for example, a character string composed of expressions (for example, complaint expression or gratitude expression) indicating the emotion of a speaker (customer, etc.) is specified and specified as a specific expression having a corresponding emotion attribute. May be. In addition, as a result of analysis, a certain character string may be included in a plurality of unique expressions.

  In the specific expression analysis rule 42, as described above, a list of correspondence between the regular expression for specifying the specific expression and the specific expression rule is specified in advance. As with the flow analysis rule 22, the normal expression to be specified is specified. For the expression and the corresponding specific expression rule, different ones are set according to the type of discourse targeted by the discourse structure analysis system 2, or the user switches the specific expression analysis rule 42 to be applied according to the type of discourse. It is possible to respond flexibly according to the characteristics of discourse.

  FIG. 8 is a flowchart showing an example of the flow of the specific expression analysis process in the specific expression analysis unit 40 of the discourse structure analysis system 2. When the specific expression analysis unit 40 starts the specific expression analysis process, first, the specific expression analysis unit 40 starts a loop process that repeats the process for each statement of the discourse data 100 (S201). When the loop processing for each statement is started, loop processing for repeating the processing for each regular expression for specifying the specific expression of the specific expression analysis rule 42 is started (S202).

  When the loop processing for each regular expression is started, it is determined whether or not each morpheme sequence in the morpheme information 31 corresponding to the target statement matches the target regular expression (S203). If there is no match, the processing proceeds to the next regular expression (S205, S202). On the other hand, if a match is found, a specific expression is identified and extracted by applying a specific expression rule corresponding to the target regular expression (S204), and the process proceeds to the next regular expression process (S205, S202).

  When all the loop processes for each regular expression are completed, the process proceeds to the next statement (S206, S201). When all the loop processing for each statement is completed, the ID of each statement and the information of the list of specific expressions corresponding thereto are output as the specific expression information 41, and the specific expression analysis process is ended.

  In this way, by performing matching with a regular expression considering the part of speech of the morpheme string and extracting the morpheme string, it is possible to perform a specific expression analysis considering a grammar peculiar to spoken words such as omission of a particle.

[Cohesion analysis]
In the following, details of cohesiveness analysis processing by the cohesiveness analysis unit 50 of the discourse structure analysis system 2 shown in FIG. 2 will be described. FIG. 9 is a diagram showing an example of cohesiveness analysis in the discourse structure analysis system 2. In the cohesiveness analysis process, first, as shown in the upper diagram, all the blocks in the discourse data 100 are set as one segment.

  Thereafter, segment division processing is performed first. In the dividing process, it is determined for each statement that a flow of a specific type is set in the flow information 21, and a segment is divided in a block to which the statement belongs. Here, it is desirable that the type of the flow to be identified is a flow that is estimated to appear when the discontinuity or content changes.

  For example, in the case of multi-person discourse such as exchanges at a call center, it is often the case that the discourse progresses depending on the response of the question and the response to it, so the content is often switched in the “question” statement of the flow. It is done. Therefore, in the present embodiment, the statement of the flow “question” (hereinafter sometimes referred to as “question statement”) is determined based on the flow information 21, and as shown in the middle diagram of FIG. Each time a question statement appears, the segment shall be split.

  Here, the question statement is a statement that is identified in the flow analysis unit 20 of the discourse structure analysis system 1 as having a flow expression indicating a question sentence for the other speaker. As a flow expression for specifying a question statement, for example, the sentence end expression of a question sentence such as “/?./” or “/like./” can be considered.

  After dividing the segments, segment integration processing is performed. In the integration process, when there is a question statement corresponding to a predetermined integration condition for each segment, the segment having the question statement is integrated with the immediately preceding segment. Here, the integration condition is designation of an expression that causes the segment having the question statement to be inferred that the content continues from the previous segment (highly cohesive with the previous segment).

  Examples of the integration condition include a case where the question statement has a word (proprietary noun) common to the immediately previous question statement or a case where a directive is included. A segment having a question statement corresponding to these integration conditions is determined to be highly cohesive with the previous segment, and is integrated with the immediately preceding segment as shown in the lower diagram of FIG. It should be noted that the integration process is repeated until there are no segments that can be integrated.

  FIG. 10 is a flowchart showing an example of the flow of cohesiveness analysis processing in the cohesiveness analyzing unit 50 of the discourse structure analyzing system 2. When the cohesiveness analysis unit 50 starts cohesiveness analysis processing, first, all the blocks of the discourse data 100 are set as one segment (S301). Specifically, for example, the initial value of the segment ID is assigned to all the blocks. After that, as a segment dividing process, a loop process for repeating the process for each block of the discourse data 100 is started (S302).

  When the loop processing for each block is started, it is determined whether or not a question statement is included in the target block (whether or not the flow has a “question” statement) (S303). If a question statement is included, the segment is divided between the target block and the immediately preceding block (S304). Specifically, for example, the next segment ID is assigned to all blocks after the target block. Thereafter, the processing moves to the next block (S305, S302).

  When the loop processing (division processing) for each block is completed, next, loop processing that repeats the processing for each segment divided by the above-described division processing is started as segment combination processing (S306). When the loop processing for each segment is started, it is determined whether or not there is a question statement corresponding to the integration condition in the target segment (S307).

  If there is a question statement corresponding to the integration condition, the target segment and the previous segment are integrated (S308). Specifically, for example, the integration is performed by updating the segment ID of each block included in the target segment to the segment ID of the immediately preceding segment. At this time, the segment ID of each block included in each subsequent segment may be sequentially incremented. Thereafter, the process proceeds to the next segment (S309, S306). When the loop processing (integration processing) for each segment is completed, information on a list of segment IDs corresponding to the ID of each block (including each block) is output as the cohesiveness information 51 for cohesiveness analysis. The process ends.

  Thereby, each statement in the discourse data 100 can be segmented based on the flow information 21 obtained by the flow analysis, and can be used as useful information for various applications using the discourse semantics 200.

  The flow information 21, the unique expression information 41, and the cohesiveness information 51 included in the discourse semantics 200 output by the processes of the flow analysis, the specific expression analysis, and the cohesiveness analysis described above are described in the discourse structure analysis system 2. An interface that allows the user to refer to the contents and make corrections as appropriate may be provided.

[Faq candidate extraction]
Details of the FAQ candidate extraction process by the FAQ candidate extraction system 1 shown in FIG. 1 will be described below. FIG. 11 is a diagram showing an example of a screen that displays the FAQ candidate 300 extracted by the FAQ candidate extraction system 1. In the upper part of the screen, there is an input field for designating a keyword as a reference when extracting the FAQ candidate 300. When the user designates a keyword, the FAQ candidate extraction system 1 extracts the contents of the FAQ candidate 300 that is extracted from the statement corresponding to the question by the customer in the discourse data 101 and the question sentence held in the existing FAQ 82 and clustered. Displayed as a list.

  In the example of FIG. 11, the contents of FAQ candidates 300 extracted and clustered are displayed for each keyword such as “XXX property” and “contract”. At this time, for the FAQ candidates in which the same question text already registered in the existing FAQ 82 is in the same cluster, the corresponding answer text is already registered in the existing FAQ 82 and is displayed as “completed”. On the other hand, FAQ candidates for which similar question texts in the cluster are not registered in the existing FAQ 82 are displayed as “undeveloped” because the corresponding answer text is not maintained in the existing FAQ 82.

  A supervisor such as a supervisor needs to prepare an answer for the “undeveloped” FAQ candidate. Therefore, for example, by instructing a hyperlink set in the sentence portion of each FAQ candidate, the screen is changed to a screen (not shown) for inputting an answer to the question so that the answer can be prepared. In this embodiment, the priority is given to the priority of answer preparation in the order of importance calculated by the scoring unit 70. For example, “not ready” is given higher priority. The display order can be arbitrarily set.

  Further, in the example of FIG. 11, “question” or “request” flow information is displayed for each FAQ candidate. In this embodiment, when extracting the FAQ candidate 300, the question extraction unit 60 extracts the statement corresponding to the question from the customer from the discourse data 101, and the flow is “question” in the flow information 21 of the discourse semantics 200. Alternatively, a statement that is a “request” (hereinafter sometimes referred to as “question / request statement”) is extracted.

  Here, the flow of “request” is set in the flow analysis unit 20 of the discourse structure analysis system 1 for a statement having a flow expression indicating a request for the partner speaker. In the example of the flow analysis rule 22 shown in FIG. 5, as the flow expression corresponding to the “request” flow, for example, an expression indicating an operation request such as “/ I want /” or “/ I want to know /” is considered. It is done.

  FIG. 12 is a flowchart showing an example of the flow of FAQ candidate extraction processing in the FAQ candidate extraction system 1. When the FAQ candidate extraction system 1 starts the FAQ candidate extraction process, the question extraction unit 60 first extracts a question / request statement based on the customer's utterance based on the discourse data 101 and the flow information 21 of the discourse semantics 200. (S401). That is, a statement whose flow is “question” or “request” is extracted from the discourse data 101.

  Next, the scoring unit 70 extracts words based on the morpheme information 31 of the discourse semantics 200 from the question / request statement extracted in step S401, and for each extracted word, the word corresponds to each question / request. Information (transposition index) obtained by counting the numbers included in the statement is created (S402). Further, the importance of each question / request statement is calculated based on this transposed index, and each question / request statement and its importance information are output as question information 71 (S403).

  The method of calculating the importance is not particularly limited. For example, as a commonly performed method, a TF-IDF (Term Frequency-Inverse Document Frequency) value is calculated for each word included in the target question / request statement. A method can be used in which the average value is the importance of the target question / request statement. Further, for example, another algorithm such as Okapi-BM25 may be used.

  Next, the FAQ candidate extraction system 1 accepts input of a keyword 81 for extracting FAQ candidates from the user, for example, via the screen shown in FIG. 11 (S404). This step may be performed at the beginning of the FAQ candidate extraction process. In addition, by the analysis by other techniques including the known technique in the FAQ candidate extraction system 1 and other systems, instead of the input from the user, for example, the rapid increase that has recently occurred frequently from statements in the discourse data 101 and other text data A keyword may be extracted and used as the keyword 81.

  Next, a question including the keyword input in step S404 from the set including the question sentence of the existing FAQ held in the existing FAQ 82 in the question / request statement included in the question information 71 by the candidate extraction unit 80 A request statement or a question sentence (hereinafter, sometimes simply referred to as “question / request statement” in some cases) is extracted (S405). Here, the question / request statement is narrowed down simply by keyword matching. In the present embodiment, a set including the question text of the existing FAQ 82 is targeted for extraction, but only the question information 71 may be extracted.

  Next, the clustering unit 90 clusters the question / request statements extracted in step S405 into similar ones, and outputs representative question / request statements of each cluster as FAQ candidates 300 (S406). finish. The format of the FAQ candidate 300 is not particularly limited, and various formats such as a data structure using XML or a simple text format can be used.

  In step S406, the clustering method is not particularly limited. For example, as a generally performed method, using the transposed index created in step S402, a word based on the appearance frequency of words included in each FAQ candidate A method can be used in which vectors are created and clustered by an algorithm such as a K-average method based on an algebraic distance (similarity) between word vectors in the vector space.

  When selecting a representative question / request statement from each cluster to be a FAQ candidate, the question / request statement included in the question information 71 and the existing FAQ question held in the existing FAQ 82 in the same cluster. When both sentences are included, a representative question / request statement from the question / request statements included in the question information 71 is selected. At this time, the question / request statement is handled as a response sentence corresponding to the existing FAQ 82, for example, to that effect (a question sentence held in the existing FAQ 82 in the same cluster belongs, that is, the answer is It is output so that it can be identified, for example, by setting a flag that is already maintained.

  Further, when outputting the FAQ candidate 300, as described above, the FAQ candidate having a higher importance calculated in step S403 may be ranked higher and output as a higher priority that should be prepared by preparing an answer. Good. The contents of the output FAQ candidate 300 are displayed on a screen as shown in FIG. 11, for example.

  As described above, according to the FAQ candidate extraction system 1 of the present embodiment, the sentence structure of the discourse is obtained by using the flow information 21 for each statement of the discourse data 101, which is the analysis result of the discourse structure. The question / request statement can be extracted from the discourse data 101 without prescribing the framework. In addition to scoring the extracted question / request statement based on importance, clustering based on the distance (similarity) of the word vectors effectively extracts the FAQ candidate 300 from the question / request statement. Can be output. Further, by extracting together with the question text of the existing FAQ 82, the FAQ candidate 300 can be extracted in consideration of duplication with the existing FAQ.

  Further, when extracting the question / request statement, only the necessary part (flow) is referred to and extracted from the contents of the discourse data 101 and the analysis result of the discourse structure, so that an erroneous analysis result due to noise or noise is present in other parts. Even if it is included, it is possible to extract the question / request statement by minimizing the influence from these.

  Thus, it becomes possible to extract FAQ candidates 300 effectively and efficiently, facilitating the maintenance of FAQs by supervisors, etc., and providing these FAQs to customers on, for example, websites, It is also possible to reduce the number of calls and reduce the operating cost of the call center.

  As mentioned above, the invention made by the present inventor has been specifically described based on the embodiment. However, the present invention is not limited to the embodiment, and various modifications can be made without departing from the scope of the invention. Needless to say.

  The present invention can be applied to a FAQ candidate extraction system and a FAQ candidate extraction program that extract FAQ candidates from the result of analyzing the structure of discourse data obtained by voice recognition processing.

1 ... FAQ candidate extraction system, 2 ... Discourse structure analysis system, 3 ... Speech recognition engine,
10: Pre-processing unit,
20 ... Flow analysis unit, 21 ... Flow information, 22 ... Flow analysis rule,
30 ... Morphological analyzer, 31 ... Morphological information,
40 ... proper expression analysis unit, 41 ... proper expression information, 42 ... proper expression analysis rule,
50: Cohesiveness analysis unit, 51 ... Cohesiveness information,
60 ... Question extraction unit,
70 ... scoring part, 71 ... question information,
80 ... candidate extraction unit, 81 ... keyword, 82 ... existing FAQ,
100, 101 ... Discourse data, 200 ... Discourse semantics, 300 ... FAQ candidates.

Claims (6)

With the input of discourse semantics, which is information for grasping the semantic content of the whole discourse, which is the analysis result of the discourse structure in the discourse data, and the discourse data in which the content of the discourse is converted into text by voice recognition processing, A FAQ candidate extraction system that extracts and outputs a question sentence as a FAQ candidate from the discourse data,
The discourse semantics includes flow information including flow information indicating the semantic content of each statement in the discourse data,
A question extraction unit that extracts a question / request statement that is uttered by a customer from the discourse data and has a flow indicating that it is a question sentence or a request sentence in the flow information of the discourse semantics;
A candidate extraction unit that extracts a specified keyword from the question / request statement extracted by the question extraction unit;
The question / request statement extracted by the candidate extraction unit and the question statement in the existing FAQ are clustered into the same content, and the question / request statement that is representative of each cluster is set as a FAQ candidate. A FAQ candidate extraction system comprising: a clustering unit that outputs together with information as to whether or not a question sentence extracted from an existing FAQ is included in a cluster to which the group belongs . The FAQ candidate extraction system according to claim 1 ,
The discourse semantics includes morpheme information including information on a morpheme sequence obtained by dividing each statement into morphemes and determining a part of speech;
Further, for the question / request statement extracted by the question extraction unit, information on the appearance frequency of the included word is acquired based on the morpheme information, and the score for calculating the importance based on the information on the appearance frequency Having a ring part,
The clustering unit outputs the question / request statements included in the FAQ candidate in the order of the importance levels. The FAQ candidate extraction system according to claim 2 ,
The scoring unit calculates a TF-IDF value for each word included in the question / request statement extracted by the question extraction unit, and uses an average value of the TF-IDF values of the words as the importance level. FAQ candidate extraction system characterized by this. With the input of discourse semantics, which is information for grasping the semantic content of the whole discourse, which is the analysis result of the discourse structure in the discourse data, and the discourse data in which the content of the discourse is converted into text by voice recognition processing, A FAQ candidate extraction program for causing a computer to function as a FAQ candidate extraction system that extracts and outputs a question sentence as a FAQ candidate from the discourse data,
The discourse semantics includes flow information including flow information indicating the semantic content of each statement in the discourse data,
A question extraction process for extracting a question / request statement uttered by a customer from the discourse data, and a flow indicating a question sentence or a request sentence is set in the flow information of the discourse semantics;
Candidate extraction processing for extracting the one containing the specified keyword from the set including the question sentence in the existing FAQ in the question / request statement extracted by the question extraction processing;
The question / request statements extracted by the candidate extraction process are clustered to have the same contents, and the question / request statements that are representative of each cluster are set as FAQ candidates , and the existing FAQ in the cluster to which the FAQ candidate belongs. A FAQ candidate extraction program that executes clustering processing that is output together with information indicating whether or not a question sentence extracted from is included . In the FAQ candidate extraction program according to claim 4 ,
The discourse semantics includes morpheme information including information on a morpheme sequence obtained by dividing each statement into morphemes and determining a part of speech;
Further, for the question / request statement extracted by the question extraction process, a score for obtaining information on the appearance frequency of the contained word based on the morpheme information and calculating the importance based on the information on the appearance frequency Perform the ring process,
The FAQ candidate extraction program, wherein the clustering process outputs the question / request statements included in the FAQ candidate in the order of importance. In the FAQ candidate extraction program according to claim 5 ,
In the scoring process, a TF-IDF value is calculated for each word included in the question / request statement extracted by the question extraction process, and an average value of the TF-IDF values of the words is used as the importance level. FAQ candidate extraction program characterized by the above.

Images