KR20190133931A - Method to response based on sentence paraphrase recognition for a dialog system - Google Patents

Abstract

According to an embodiment of the present invention, a sentence paraphrase recognition-based dialogue system answer method relates to a method of searching a question-answer knowledge (question) having a similar meaning and intention of a sentence (query) input through a paraphrasing recognition technique to provide a related answer. The present invention comprises: a technology application domain keyword recognition and pattern extraction step; a question domain/detail category/talk act classification step; a semantically similar question analysis step; an expressively similar question analysis step; and a similar question answer providing step. Through the technology, provided is the paraphrase recognition-based answer method in a dialogue system for an automated Q&A service such as a chatbot for customer consultation. The present invention can convert a natural language query sentence, which can be transformed into various expressions to be input, into a vector containing semantic and structural information, and can determine whether an input query is similar to a sentence of previously constructed knowledge by using the vector. In addition, the present invention can provide an answer to a query by determining whether to provide an answer of a corresponding input according to a similarity score. When it is desired to provide a strictly refined answer depending on an application domain such as a financial domain, which is an embodiment of the present invention, the present invention can provide a more reliable answer than a method using a sentence generation model.

Description

{Method to response based on sentence paraphrase recognition for a dialog system}

The present invention relates to a method of searching for a question-answer knowledge (question) having similar meaning and intention of a sentence (query) input through a paraphrase recognition technique to provide a related answer.

The present invention is an invention that can be utilized in FAQ, customer service, etc., and is applied to a conversation system for automatic Q & A. It utilizes natural language processing and understanding technology based on sentence embedding technology. Through this, it is possible to search for questions similar in meaning and intent to the input questions.

Intelligence question and answering service is a service that provides answers to questions entered by users. Naver intellectuals, Quora, Yahoo! From community-like online platforms that provide answers to experts and other users, such as answers, to intelligent Q & A systems that automatically provide answers based on advanced natural language processing technologies such as IBM Watson and ETRI's Exobrain and built knowledge bases. The system provides a response to the user's query.

A dialogue system is a computer system that is concretely oriented and conversational with a user. In general, a process of inquiring and answering between a user and a computer takes the form of a conversation. This is mainly done by promptly processing the user's query input and responding to the request. Recently, a context-based dialogue analysis, which provides a response by analyzing a conversation over several turns, and an asynchronous response providing method are studied. have. The present invention describes a method for providing an automatic Q & A service using this dialog system.

In order to communicate with users and computers in a conversation system, high-level natural language processing and understanding skills are required. In the present invention, natural language processing such as morphological analysis and POS tagging, object name extraction and recognition, and natural language understanding such as sentence embedding, domain / intention classification, and paraphrase recognition are performed.

In the present invention, the sentence embedding technique is a method used in domain / intention classification, sentence similarity analysis, and paraphrase recognition part of a query sentence. Similarly to the well-known word embedding technique, the vector is represented by a natural language representation to represent a sentence as a real vector of a predetermined size. For this purpose, deep learning based machine learning model is used, and the sentence is embedded as a vector by using domain and speech act classification system according to the applied domain as learning quality. By vectorizing a sentence, semantic / structural information can be expressed as numerical and spatial information, and classification, clustering, and similarity measuring can be performed through this vector.

The present invention provides a paraphrase recognition based answer method in a conversation system for an automatic Q & A service such as a chatbot for customer consultation.

Sentence paraphrase recognition-based dialogue system answer method according to an embodiment of the present invention comprises the steps of recognition of important keywords and pattern extraction of the technology application domain based on the morphological analysis results analyzed in the preprocessing process; Classifying the question domain / detail category / talk act using the morphological analysis results analyzed in the preprocessing process and the extracted important keywords and patterns; A semantic similarity question analysis step of learning a model using the classified question domain subcategory speech act classification feature as a semantic feature of a query sentence and extracting sentence semantic vectors to measure similarity between vectors; A representational similarity question analysis step of learning a language model of the character and extracting a sentence expression vector based on the character to analyze the similarity between the expression and the structure; Based on the analyzed semantic similarity and expressional similarity, a vector containing semantic and expressive information about an input query sentence that can be input in various forms is generated, and a similar sentence sentence in a FAQ knowledge is paraphrase recognition technology. A similar question answer step of finding and providing an answer is included.

According to an embodiment, the technology application domain important keyword recognition and pattern extraction step may include a technology application domain specific entity name recognition step, a compound word extraction step, a synonym / parent word analysis step, and a question expression pattern extraction step.

In the technology application domain important keyword recognition and pattern extraction step, domain-specific entity names and compound word dictionaries can be constructed based on the morphological analysis results analyzed in the preprocessing process, and thesaurus or lexical network such as synonyms and parent words can be constructed.

According to an embodiment, the classification step of the question domain / detail category / act act may include a first learning step and a classification step. The first learning step may include learning data tagging and construction step, question intention classification model learning step, and the like. The classification step uses a learned classification model and may include a question intent classification step, an analysis result textualization, and additional learning data semi-automatic tagging step.

In the classification step of the question domain / detailed category / conversation act, the morphological analysis results analyzed in the preprocessing process and the extracted important keywords and patterns are used to generate training data and learn a classification model to extract corresponding classification features for new query sentences. can do.

According to an embodiment, the semantic similarity question analysis step may include a second learning step and a first similarity analysis step. The second learning step may include sentence morphological tagging, semantic feature (domain, speech act) classification step, and semantic feature based sentence embedding model learning step. The similarity analysis step uses a learned embedding model (encoder), and may include a question intent embedding step and an embedding vector based semantic similarity measure step.

In the semantic similar question analysis step, the model may be trained using the classified question domain subcategory speech act classification feature as the semantic feature of the query sentence, and the similarity between the vectors may be measured by extracting the sentence semantic vector.

According to an embodiment, the expression-like question analysis step may also include a third learning step and a second similarity analysis step. The third learning step may include a step of tokenizing on a letter basis and a learning expression embedding-based embedding model. The second similarity analyzing step may include a trained letter level embedding model, and may include a letter level embedding step and an embedding vector based expressive similarity measure step.

In the expression-like question analysis step, the language model of the letter may be learned, and the sentence expression vector based on the letter may be extracted to analyze the similarity between the expression and the structure.

According to an embodiment of the present disclosure, the providing of the similar question answer may include a semantic similarity question analysis step and an expression similarity analysis step, and search for a question similar to the input question using the analyzed similarity result. And determining whether or not to provide an answer.

In the step of providing a similar question answer, it may be recognized as a paraphrase sentence of a refined FAQ query sentence based on the analyzed semantic similarity and expression similarity, and may provide an answer of the corresponding FAQ query sentence according to a similarity score.

The present invention provides a source technology for providing an answer in a conversation system for an automatic Q & A service such as a chatbot for customer consultation. Through this technology, natural language query sentences that can be transformed into various expressions can be transformed into vectors containing semantic and structural information, and can be used to determine whether they are similar to the sentences of input queries and pre-built knowledge. have. In addition, it is possible to provide an answer to the query by determining whether to provide an answer of the corresponding input according to the similarity score. If you want to provide a strictly refined answer according to the technology application domain, such as the financial domain, there is an advantage that can provide a reliable answer compared to using a sentence generation model.

1 is a flowchart illustrating an answer method of an automatic Q & A conversation system based on paraphrase recognition based on sentence embedding similarity according to an embodiment of the present invention.
2 is a detailed flowchart illustrating a technology application domain important keyword recognition and pattern extraction step according to an embodiment of the present invention.
3 is a detailed flowchart of a question domain / detail category / act act classification step according to an embodiment of the present invention.
4 is a detailed flowchart of a semantic similar question analysis step according to an embodiment of the present invention.
5 is a detailed flowchart of an expression-like question analysis step according to an embodiment of the present invention.
6 is a detailed flowchart of providing a similar question answer step according to an embodiment of the present invention.
7 is a conceptual diagram for a method of answering a sentence paraphrase recognition based conversation system.
8 is a block diagram of a learning model for embedding query sentences used in semantic similar question analysis according to the present invention. In order to generate sentence semantic vector, inputted query sentence, domain, subcategory and speech act information are needed and used as input and output of this model.
9 is a block diagram of a character embedding model used in the expression similarity question analysis in the present invention. Each letter is input to each neural network and learns letters, expressions, and structure information that appear together in a sentence, and reflects this information in the embedding vector.
10 illustrates a similar sentence recognized by the input query sentence and a paraphrase and the similarity measurement result.

DETAILED DESCRIPTION The following detailed description of the invention refers to the accompanying drawings that show, by way of illustration, specific embodiments in which the invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention. It should be understood that the various embodiments of the present invention are different but need not be mutually exclusive. For example, certain shapes, structures, and characteristics described herein may be embodied in other embodiments without departing from the spirit and scope of the invention with respect to one embodiment. In addition, it is to be understood that the location or arrangement of individual components within each disclosed embodiment may be changed without departing from the spirit and scope of the invention. The following detailed description, therefore, is not to be taken in a limiting sense, and the scope of the present invention, if properly described, is defined only by the appended claims, along with the full range of equivalents to which such claims are entitled. Like reference numerals in the drawings refer to the same or similar functions throughout the several aspects.

Hereinafter, a method of searching for a question-answer knowledge (question) having similar meaning and intention of a sentence (query) input through a paraphrase recognition technology according to an embodiment of the present invention will be described with reference to the accompanying drawings. Explain.

1 is a flowchart illustrating an answer method of an automatic Q & A conversation system based on paraphrase recognition based on sentence embedding similarity according to an embodiment of the present invention.

Referring to FIG. 1, the sentence paraphrase recognition-based dialogue system answering method includes a technology application domain key keyword recognition and pattern extraction step (S100), a question domain / detail category / speech act classification step (S200), and a semantic similar question analysis step ( S300), a representation similar question analysis step (S400), may provide a similar question answer step (S500).

7 is a conceptual diagram for a method of answering a sentence paraphrase recognition based conversation system. Referring to FIG. 7, an exemplary embodiment of a sentence paraphrase recognition based dialogue system answering method is presented.

The technology application domain important keyword recognition and pattern extraction step (S100) analyzes the entity name, compound word, synonym, upper word, etc. related to the applied domain in the inputted query sentence, and patterns it to map answers and questions. It is the process of extracting. Domain-specific entity name dictionary and compound word dictionary are built in semi-automatic way through machine learning model for entity name / composite candidate extraction, and synonym and lexical concept information of lexical vocabulary is constructed as lexical network or thesaurus. Through this, it is possible to extract important keywords that can be found in the input query, and some verbal expressions can be expressed as agreement or higher concept.

The question domain / detail category / act act classification step S200 is a process of analyzing the question domain / detail category / act act qualities of the new input query sentence. Along with the pattern information extracted in S100, domain, category, speech act information is used to build a question-answer knowledge or find an answer. In the learning process of this stage, the process of tagging domain, subcategory and speech act information for each learning question data is required. In the embodiment according to the present invention, the question intent classification model may use a deep learning based neural network model. Alternatively, the question intent classification model may replace other learning-based classification models. In addition, the classification result of the new query sentence can be reviewed and used for semi-automatic construction of additional learning data.

The semantic similar question analysis step S300 is a process of performing sentence embedding to vectorize an input query sentence to a predetermined length and measuring semantic similarity with a question of FAQ knowledge. For this, the sentence embedding model learning process may be preceded, and the domain, subcategory, speech act qualities, and query sentences constructed in S200 are learned in the learning model of FIG. 8.

The similarity question analysis step (S400) is a process of embedding a sentence into a vector using only letters and structure information of the input sentence. If the keyword pattern is not extracted from the sentence, or the domain or subcategory qualities are not classified, the answer cannot be provided in the previous step, or the answer or response to the expression similar to the input sentence is provided through this step.

The similar question answer providing step (S500) is a process of determining whether to provide an answer for the inputted question sentence by using similar sentence analysis results analyzed in steps S300 and S400 as similarity scores.

2 is a detailed flowchart of a technology application domain important keyword recognition and pattern extraction step S100 according to an embodiment of the present invention.

Referring to FIG. 2, the technology application domain important keyword recognition and pattern extraction step S100 according to an embodiment may include a technology application domain-specific entity name recognition step S110, a compound word extraction step S120, and synonyms / parent words analysis. A step S130 and a question expression pattern extraction step S140 may be included.

The technology application domain-only entity name recognition step (S110) according to an exemplary embodiment is a process of recognizing the technology application domain-only entity name. It may include date information, non-identifiable expressions, and the like, and may construct a domain-specific entity name dictionary using existing entity name recognition technology.

According to an exemplary embodiment, the compound word extracting step S120 is a process of additionally extracting compound word information specific to an applied domain by using a morphological analysis result, and may be used to catch a specific brand name or an important keyword in an input sentence. Similar to the entity name dictionary, a separate dictionary can be constructed according to the applied domain. In this process, the entity name / compound candidate can be extracted using a machine learning model. The system developer decides whether or not to register in advance.

The synonym / parent word analysis step S130 according to an exemplary embodiment is a process of normalizing a specific term expression or analyzing synonym information or higher concept information of a specific expression. The vocabulary of the entity name and compound word dictionary constructed in the previous step can be constructed by thesaurus or vocabulary network.

The extracting of the question expression pattern according to an exemplary embodiment (S140) may be a process of patterning question expressions such as an entity name, a compound word, a synonym, and a parent word for the input query extracted in the previous step according to a connection relationship.

3 is a detailed flowchart of a question domain / detail category / speech act classification step S200 according to an embodiment of the present invention.

Referring to FIG. 3, the question domain / detail category / talk act classification step S200 according to an exemplary embodiment may be largely divided into a learning step and a classification step. The learning step may include a learning data tagging and building step (S210), a question intention classification model learning step (S220), and the like. The classification step may include a question intent classification step S230, an analysis result textification, and additional learning data semi-automatic tagging step S240.

The learning data tagging and construction step (S210) according to an exemplary embodiment is a process of tagging a query sentence classification system feature for Q & A specific to a system application domain for each question, by tagging features such as domain, subcategory, and act act. Create data for learning and evaluating classification models,

The learning intention classification model training step S220 according to an exemplary embodiment is a process of training the learning data constructed in the previous step S210 in a deep learning based classification model. In the embodiment of the present invention, four learning-based classification models (SVM, RF, FC-MLP, ELM) were implemented to compare classification performance. In the embodiment of the present invention, a classification model based on FC-MLP is employed.

The question intent classification step (S230) according to an exemplary embodiment is a process of performing a feature analysis on a new input query sentence using a learned classification model, which is used to provide rule-based answers and evaluates the classification accuracy of the classifier itself. It is used to In addition, the analysis process was used to determine the threshold for providing answers later based on the classification prediction rate and the accuracy of classification.

The text analysis and semi-automatic tagging of the analysis result according to an exemplary embodiment and the semi-automatic tagging step S240 are processes for additionally constructing learning data. Perform data modification and further build up. More refined learning data are needed to improve the feature extraction performance of machine learning based classification models.

4 is a detailed flowchart of a semantic similar question analysis step according to an embodiment of the present invention.

Referring to FIG. 4, the semantic similar question analysis step may be divided into a learning step and a similarity analysis step. The learning step may include a sentence morphological tagging, a semantic feature (domain, speech act) classification step (S310), and a semantic feature based sentence embedding model learning step (S320). The similarity analyzing step may include a question intent embedding performing step S330 and an embedding vector based semantic similarity measuring step S340.

According to an exemplary embodiment, the sentence morpheme tagging and semantic qualities (domain, speech act) classification step S310 is a process of arranging data to be trained in a model by collecting the results analyzed in the previous step. In this process, we use the classification features analyzed through the word embedding vector and classification model of the morphemes learned from the language analyzer (morphological tagging) as semantic features and structure them as input structures.

The semantic feature-based sentence embedding model training step S320 according to an exemplary embodiment is a process of training a deep learning model combining a Seq2Seq-based encoder-decoder model and a classification model learned in the previous step S200. Through this model, we can build an encoder model that learns the linguistic model of sentences and generates sentence semantic vectors containing semantic information such as domains, categories and speech act classifications.

The question intent embedding performing step S330 according to an exemplary embodiment is a process of embedding a new input query sentence into a sentence semantic vector using the encoder model learned in the previous step S320. In this process, the question is converted into a real vector of the desired dimension for the input query sentence.

The embedding vector-based semantic similarity measuring step (S340) according to an exemplary embodiment may be performed by using a sentence semantic vector transformed in the previous step, and a FAQ sentence having a query sentence and refined answer knowledge through a method of measuring similarity between vectors. The similarity is measured. In an embodiment of the present invention, cosine similarity capable of measuring similarity between vectors is used, and the similarity between similar vectors may be replaced.

8 illustrates an embodiment of a learning model for query sentence embedding used for semantic similar question analysis in the present invention. An input query sentence, domain, subcategory and speech act information is required to generate a sentence semantic vector, and may be used as an input of the learning model illustrated in FIG. 8.

Next, FIG. 5 is a detailed flowchart of an expression-like question analysis step S400 according to an exemplary embodiment of the present invention.

Referring to FIG. 5, the expression-like question analysis step S400 according to an embodiment may also include a learning step and a similarity analysis step. The learning step may include a step of tokenizing in units of letters (S410) and a step of learning a letter expression-based embedding model (S420). The similarity analyzing step may include a character level embedding performing step S430 and an embedding vector based expressive similarity measuring step S440.

According to an exemplary embodiment, the tokenization in units of characters (S410) is a process of creating an input vector at a character level to implement an encoder-decoder model for learning a language model of a character appearing in a query sentence. As one step, unlike the previous step (S300) in which the word embedding vector is used, one-hot encoding vector of a letter vocabulary may be used.

According to an exemplary embodiment, the character expression-based embedding model learning step (S420) may be a process of learning an encoder-decoder model for learning a language model of a letter appearing in a query sentence. In this step, the language model between the letters appearing together in the sentence can be learned through this learning process. Through this, it is possible to vectorize the relationship between the letters constituting the sentence. In addition, the decoder can predict the letters constituting the sentence.

Embedding step of the letter level according to an exemplary embodiment (S430) is similar to the process of embedding a query sentence using the semantic qualities of the previous step (S300) using a letter only sentence representation with only the expression information of the sentence It may be a process of generating a vector.

The expression vector-based expressive similarity measuring step S440 according to an exemplary embodiment may be a process of measuring an explicit similarity such as a character and a structure of an input sentence using the analyzed sentence expression vector. This step was developed to respond to input conversations that are difficult to measure semantic similarity or that are not relevant to the domain to which it is applied, and can provide responses to emotional expressions, such as greetings and abusive language. As in the previous step S300, a method for measuring similarity between vectors may be used, and in an exemplary embodiment, cosine similarity may be used.

9 is an embodiment of a character embedding model used in the expression similarity question analysis in the present invention. Each letter is input to each neural network and learns letters, expressions, and structure information that appear together in a sentence, and reflects this information in the embedding vector.

6 is a detailed flowchart of a similar question answer providing step S500 according to an embodiment of the present invention.

Referring to FIG. 6, the providing of a similar question answer (S500) according to an embodiment may include a semantic similarity question analyzing step S510 and a representational similar question analyzing step S520. The providing similar question answer step S500 may also include searching for similar question FAQ questions similar to the input question using the similarity result analyzed through the preceding step, and determining whether to provide an answer (S530). have.

The semantic similarity question analysis step (S510) and the expression similarity question analysis step (S520) according to the exemplary embodiment exceed the reference score based on the similarity measure (score) analyzed in the previous steps (S300, S400) or It may be a process of providing an answer in step S530 of determining whether or not to provide a similarity answer by determining whether or not. In an exemplary embodiment of the present invention, semantic similarity may be based on, for example, 0.7 for similarity judgment. That is, an answer to an FAQ question showing a similarity level of 0.7 or more may be provided, and in the case of expression similarity, for example, 0.6 may be used as a criterion for determining similarity. That is, it can provide an answer of general conversation knowledge showing a degree of similarity of 0.6 or more.

FIG. 10 illustrates an example of a similar sentence recognized by the input query sentence and a paraphrase and a result of similarity measurement.

As described above, the present invention can provide a source technology for developing a chatbot capable of understanding the contents of counseling in natural language and responding appropriately.

Features, structures, effects, etc. described in the above embodiments are included in one embodiment of the present invention, and are not necessarily limited to one embodiment. Furthermore, the features, structures, effects, and the like illustrated in the embodiments may be combined or modified with respect to other embodiments by those skilled in the art to which the embodiments belong. Therefore, contents related to such combinations and modifications should be construed as being included in the scope of the present invention.

In addition, the above description has been made with reference to the embodiment, which is merely an example, and is not intended to limit the present invention. Those skilled in the art to which the present invention pertains will be illustrated as above without departing from the essential characteristics of the present embodiment. It will be appreciated that various modifications and applications are possible. For example, each component specifically shown in the embodiment can be modified. And differences relating to such modifications and applications will have to be construed as being included in the scope of the invention defined in the appended claims.

Claims (11)

A method executed by a processor of a computer device,
Recognition of important keywords and pattern extraction of the technology application domain based on the morphological analysis results analyzed in the preprocessing;
Classifying the question domain / detail category / talk act using the morphological analysis results analyzed in the preprocessing process and the extracted important keywords and patterns;
A semantic similar question analysis step of learning a model using the classified question domain subcategory speech act classification feature as a semantic feature of a query sentence, and extracting sentence semantic vectors to measure similarity between vectors;
A representational similarity question analysis step of learning a language model of the character and extracting a sentence expression vector based on the character to analyze the similarity between the expression and the structure; And
Based on the analyzed semantic similarity and expressional similarity, a vector containing semantic and expressive information about an input query sentence that can be input in various forms is generated, and a paraphrase recognition technology is used as a paraphrase recognition technology in FAQ knowledge. A sentence paraphrase recognition-based dialogue system answering method comprising the step of providing a similar question answer to find and provide an answer. The method of claim 1, wherein the technology application domain key word recognition and pattern extraction step includes a technology application domain specific entity name recognition step, a compound word extraction step, a synonym / parent word analysis step, and a question expression pattern extraction step. Sentence paraphrase recognition based dialogue system answering method. The method of claim 2, wherein in the technology application domain important keyword recognition and pattern extraction step, domain-specific entity names and compound word dictionaries are constructed based on morphological analysis results analyzed during preprocessing, and To build The method of claim 1, wherein the classifying step of the question domain / detail category / conversation includes a first learning step and a classifying step, wherein the first learning step includes learning data tagging and construction step, a questioning intention classification model learning step, and the like. And wherein the classification step uses a learned classification model, and comprises a question intent classification step, analytical result textualization, and semi-automatic tagging of additional learning data. The method of claim 1, wherein in the classification of the question domain / detailed category / conversation act, learning data is generated using a morphological analysis result analyzed in the preprocessing process and the extracted important keywords and patterns, and a classification model is trained to generate a new query sentence. Sentence paraphrase recognition based dialogue system answering method characterized in that for extracting the classifiers. The method of claim 1, wherein the semantic similarity question analysis step includes a second learning step and a first similarity analysis step, wherein the second learning step includes sentence morphological tagging, meaning feature (domain, speech act) classification step, and semantic feature. And a step of learning a sentence based embedding model, wherein the similarity analysis step uses a learned embedding model (encoder), and includes a question intention embedding step and an embedding vector based semantic similarity measurement step. Sentence paraphrase recognition based dialogue system answer method. The method of claim 1, wherein the semantic similar question analysis step uses the classified question domain subcategory speech act classification feature as a semantic feature of a query sentence to learn a model, extract a sentence semantic vector, and measure similarity between vectors. Sentence paraphrase recognition based dialogue system answer method, characterized in that the. The method of claim 1, wherein the expression similarity question analysis step also includes a third learning step and a second similarity analysis step, wherein the third learning step includes a tokenization step by letter unit and a character expression-based embedding model learning step. And the second similarity analysis step uses a learned character level embedding model, and includes a character level embedding performing step and an expression vector based expressive similarity measuring step. . The sentence paraphrase recognition-based dialogue system answering method according to claim 1, wherein in the expression-like question analysis step, a language model of a letter is learned and a sentence expression vector based on the letter is extracted to analyze the similarity between the expression and the structure. . The method of claim 1, wherein the providing of the similar question answer may include a semantic similarity question analyzing step and an expression similarity question analyzing step, and searching for an FAQ question similar to the input question using the analyzed similarity result. And a sentence paraphrase recognition based dialogue system answering method comprising the step of determining whether or not to provide an answer. The method of claim 1, wherein the providing of the similar question answer recognizes that it is a paraphrase sentence of a refined FAQ query sentence based on the analyzed semantic similarity and expression similarity, and provides an answer of the corresponding FAQ query sentence according to a similarity score. Sentence paraphrase recognition based dialogue system answer method, characterized in that the.

Images