TWI734085B - Dialogue system using intention detection ensemble learning and method thereof - Google Patents

Abstract

The present invention relates to a dialogue system using intention detection ensemble learning and a method thereof, the method comprising: receiving text content input by a user; performing pre-processing of the unmarked data and the marked data, and the unmarked data and the marked data are imported into the semi-supervised dialogue topic model to output the conversation topic distribution; integrating multiple intent decision results and strengthening the error sample learning to produce a decision of final intent classification; determining a system response content according to the final intent classification, sending the system response content to the user and receiving a feedback data of the user to determine the correctness of the system response content; and finally, correspondingly marking and updating the text content and the feedback data of the user to import the marked information into the semi-supervised dialogue topic model for updating and learning.

Description

Dialogue system and method using intention detection integrated learning

The present invention relates to the technology of intention recognition in a dialogue system, and in detail, it relates to a dialogue system and method using semi-supervised intent detection integrated learning.

In recent years, the concept of conversation as a platform (conversation as a platform) has emerged among the major technology giants. The way of human-computer interaction has changed from a graphical interface to a conversational interactive interface. Various human tasks can be described by dialogue to achieve interactive services. Therefore, natural language has become a key factor in the human-computer interaction interface. Use dialog robots to automatically reply to most simple questions, allowing customer service staff to focus on more difficult questions, which can greatly reduce the labor cost of text customer service and the dialog robot can reply users in real-time (Real-Time), and can serve more at the same time Of users and 24 hours a day service throughout the year, so in order to prevent robots from answering unsolicited questions and reduce users’ willingness to use, improving the robot’s semantic intent recognition rate for user questions is the most important thing.

Past research shows that task-based dialogue systems need to analyze the intent of the question and give the corresponding answer defined by the knowledge base, but from the perspective of the user, of course, the more multi-functional (intent) support, the better, and the fewer dialogue rounds. The better you find the answer. Conventional technologies mostly construct intent recognizers by methods such as establishing rules or machine learning classifiers. With more types of intentions, the task of intention recognition becomes more difficult. in The big reason is due to insufficient labeled data and insufficient model input features. Therefore, how to make extensive use of unlabeled data and increase the diversity of input features to provide accurate intent recognition has become an urgent problem for task-based dialogue systems. On the other hand, in the face of ever-changing task requirements, fast and accurate intention updating is also an indispensable function for the next stage of dialogue robots.

It can be seen from the above that if we can find a technology to improve the intent recognition rate in the dialogue system, especially how to use the existing data and existing data to improve the intent prediction, and also to further update the judgment criteria based on the feedback mechanism, and then improve the future As a result of the identification, this will become the goal of those skilled in the art who are eager to pursue a solution.

The purpose of the present invention is to establish an accurate intent recognition and rapid intent update mechanism. The semi-supervised dialogue subject intent integrated recognition method proposed in this patent can improve the performance of the dialogue system and reduce the number of dialogue rounds required for users to find answers.

In order to achieve the above and other objectives, the present invention proposes a dialogue system using intent detection integrated learning, which includes: a text input module for receiving text content; a text preprocessing module for receiving the text content Defined as unlabeled data, the unlabeled data and the existing labeled data are pre-processed to convert the vocabulary into vector representation; the semi-supervised dialogue topic module is used to produce the dialogue topic distribution A semi-supervised dialogue topic model, wherein the semi-supervised dialogue topic model is modeled based on the training data composed of the unlabeled data vocabulary and vector from the text pre-processing module and the labeled data vocabulary and vector; The intent recognition integrated learning module has a sample and feature selector, an intent recognizer, and an intent recognition integrator. The sample and feature selector perform sample and feature selection for the training data It is taken to form multiple sets of training data subsets, each of the training data subsets corresponds to an intent recognizer for training, and then the intent recognition integrator is used to integrate the intent decision results of multiple intent recognizers to produce the final Intent classification decision; knowledge base search module is to query through the final intent classification decision to obtain system response content from the database; system response module is to send the system response content to receive related system response content Whether the reply data is correct; and the system marking module. When the reply data is incorrect, the text content is marked as the correct intention category and imported into the text pre-processing module to generate a new vocabulary According to the vector, the semi-supervised dialogue topic model is updated to optimize the output topic probability distribution, and then the intention category predicted by the intention recognition integrated learning module is re-adjusted.

In one embodiment, the text pre-processing module further includes: a sentence normalization unit, which is used to filter out specific symbols or languages, and to perform encoding conversion; and the sentence segmentation unit is used to use words as the text The unit is separated; and the vocabulary vectorization unit is used to convert the vocabulary after the word segmentation into a vector representation.

In another embodiment, the intention recognition integrated learning module uses a guided aggregation algorithm (Bagging) or a step-by-step boosting algorithm (Boosting) or a combination thereof for integrated learning, thereby reducing model variation and reducing model errors.

In another embodiment, the intent recognition integrated learning module further includes measuring the final intent classification provided by the intent recognition integrator and the true intent category of the text content, so as to feed back misclassified training samples to the sample And a feature selector to increase the weight of the misclassified training sample being selected, and iteratively train until the correct rate reaches the threshold.

In addition, the system's replies include replying to user questions, ongoing conversations, or user confirmations.

In yet another embodiment, the system tagging module uses the semi-supervised dialog topic module to find similar sentences with the same topic meaning as the wrong data, and uses auxiliary tagging or the nearest neighbor algorithm in the dialog topic space. Find the k most similar marked data in the database, and exclude the wrong data originally predicted by the intention recognition integrated learning module intent category, so as to mark the wrong data as the intention category with the highest vote.

In addition, the system marking module marks the text content into the correct intention category and imports it into the text pre-processing module to become the existing marked data.

The present invention further proposes a dialogue method using semi-supervised intention detection integrated learning, including: receiving text content; defining the text content as unmarked data, and pre-processing the unmarked data and the existing marked data, To convert the vocabulary into a vector representation; import the unlabeled data and the labeled data into a semi-supervised dialogue topic model to output the dialogue topic distribution; combine the dialogue topic distribution and dialogue text content as input to integrate multiple intent decision results And the method of strengthening the learning of the wrong sample to produce the final intention classification decision; according to the final intention classification to determine the system response content; reply the system response content to the user, to receive the user's feedback data to determine the correctness of the response data性; and correspondingly annotate and update the text content and feedback data, and then import the annotated data into the semi-supervised dialogue topic model for update and learning.

In one embodiment, the text pre-processing includes sentence normalization, sentence segmentation, and vocabulary vectorization.

In another embodiment, the output of the final intent classification decision includes selecting samples and features for the unlabeled data vocabulary and vector and the labeled data vocabulary and vector to form multiple sets of training data subsets, each of which The training data subset is trained to integrate the multiple intention decision results.

In another embodiment, the method for enhancing the learning of error samples includes measuring the final intent classification and the true intention category of the text content, and feeding back misclassified training samples to increase the weight of the misclassified training samples being selected , And through repeated iterative training until the correct rate reaches the threshold.

In addition, the system's replies include replying to user questions, ongoing conversations, or user confirmations.

In still another embodiment, correspondingly annotating and updating the text content and feedback data refers to using the semi-supervised dialogue topic model to find similar sentences with the same topic meaning as the wrong data, through auxiliary annotation or using nearest neighbor calculation The method is to find the k closest marked data in the dialogue topic space, and eliminate the intention category predicted by the wrong data, so as to mark the wrong data as the intention category with the highest vote.

In addition, correspondingly marking and updating the text content and the feedback data includes marking the text content as the correct intention category to become the existing marked data.

In summary, through the dialog system and method for integrated learning using intention detection proposed in the present invention, the intention recognition and knowledge base update mechanism are enhanced by using a semi-supervised dialog topic model, where semi-supervised refers to the extensive use of unintended The marked dialogue text content establishes a semi-supervised dialogue topic model and uses the marked data to guide the unmarked data to analyze the implied topic meaning of the dialogue sentence, and then generate more meaningful grouping results, which can more accurately identify the user's intention and dialogue Process and quickly update the user’s feedback intention to reduce the number of conversations and improve the performance of the conversation system.

1‧‧‧The use of intent detection integrated learning dialogue system

11‧‧‧Receive text input module

12‧‧‧Text preprocessing module

1201‧‧‧Sentence Regularization Unit

1202‧‧‧Sentence Segmentation Unit

1203‧‧‧Vocabulary Vectorization Unit

121‧‧‧Unlabeled data

122‧‧‧Marked data

123‧‧‧Sentence regularization

124‧‧‧Sentence Segmentation and Elimination of Words

125‧‧‧Vocabulary Vectorization

126‧‧‧Unlabeled data vocabulary and vector

127‧‧‧Marked data vocabulary and vector

13‧‧‧Semi-supervised dialogue theme module

131‧‧‧Bag of words model

132‧‧‧TFIDF model

133‧‧‧Semi-supervised LDA dialogue topic model

134‧‧‧Distribution of Dialogue Topics

14‧‧‧Intention recognition integrated learning module

141‧‧‧Sample and feature selector

142‧‧‧Intent Recognizer

143‧‧‧Intent Recognition Integrator

15‧‧‧Knowledge base search module

16‧‧‧System Response Module

17‧‧‧System Marking Module

S61~S67‧‧‧Step

Figure 1 is the system architecture of the dialogue system for integrated learning using intention detection of the present invention Figure; Figure 2 is a structural diagram of the text pre-processing module in the system of the present invention; Figure 3 is a flowchart of the execution of the text pre-processing module in the system of the present invention; Figure 4 is the system of the present invention Figure 5 is the execution flow chart of the semi-supervised dialogue topic module in the system of the present invention; and Figure 6 is the dialogue using the intent detection integrated learning module of the present invention Diagram of the steps of the method.

The following describes the technical content of the present invention with specific specific embodiments. Those familiar with the art can easily understand the advantages and effects of the present invention from the content disclosed in this specification. However, the present invention can also be implemented or applied by other different specific embodiments.

Figure 1 is a system architecture diagram of the dialogue system using intention detection integrated learning of the present invention. As shown in the figure, the dialog system 1 for using intention detection integrated learning of the present invention includes a text input module 11, a text preprocessing module 12, a semi-supervised dialog topic module 13, and an intention recognition integrated learning module 14. , Knowledge Base Search Module 15, System Reply Module 16, and System Marking Module 17.

The receiving text input module 11 is used for receiving text content input by the user. The dialogue system 1 using intent detection integrated learning receives the text content input by the user through the text input module 11. If the user's actual input is voice, the front end of the application can be connected to a voice recognizer, and after the voice is converted into text, the text input module 11 can be connected.

In specific implementation, the receiving text input module 11 is a receiving device for the user to input text. Interface, you can use the client-server web page receiving method or the mobile phone application (APP) method to provide the input screen for the client to input the dialog question, and then transmit it to the server through the network Or the cloud, and then send the text to the text pre-processing module 12 on the server or the cloud for processing.

The word pre-processing module 12 receives the text content input by the user to define it as unmarked data, and performs word pre-processing on the unmarked data and the existing marked data to convert the vocabulary into a vector representation. In short, the text pre-processing module 12 mainly performs text pre-processing on the text content input by the user and the content marked by the marking system 17. Specifically, as shown in Figure 2, the text pre-processing module 12 may include but not It is limited to the sentence normalization unit 1201, the sentence segmentation unit 1202, and the vocabulary vectorization unit 1203. The sentence normalization unit 1201 filters out and encodes specific symbols or languages, and the sentence segmentation unit 1202 converts the text input by the user. The content is separated in units of words, and the vocabulary vectorization unit 1203 converts the segmented vocabulary into a vector representation. In specific implementation, the word pre-processing module 12 can be a server-side or cloud-based word pre-processing program.

The semi-supervised dialogue topic module 13 has a semi-supervised dialogue topic model for generating a dialogue topic distribution, wherein the semi-supervised dialogue topic model is based on the unlabeled data vocabulary and vector from the text pre-processing module And the training data composed of the labeled data vocabulary and vectors are modeled. Semi-supervised dialogue topic module 13 mainly uses marked data to guide unmarked data to analyze the implicit topic meaning of dialogue sentences (ie text content), so as to produce more meaningful grouping results. Specifically, semi-supervised The dialogue topic module 13 receives the unlabeled data vocabulary and vector and the marked data vocabulary and vector recorded by the text preprocessing module 12 to perform a semi-supervised dialogue topic model modeling. Thereafter, the result of the current dialogue data processed by the text pre-processing module 12 is then input into the semi-supervised dialogue topic module 13 to produce the dialogue topic distribution of the current dialogue data. In actual implementation, the semi-supervised dialogue topic model can be a programming model that predicts the distribution of dialogue topics on the server or the cloud. The input data source is the unmarked data vocabulary and vector and the marked data vocabulary and vector of the word preprocessing module 12.

The intent recognition integrated learning module 14 has a sample and feature selector, an intent recognizer, and an intent recognition integrator. The sample and feature selector perform sample and feature selection for the training data to form multiple sets of training data subsets, each of which is trained The data subsets respectively correspond to one intent recognizer for training, and the intent recognition integrator is used to integrate the intent decision results of multiple intent recognizers to produce a final intent classification decision. The intention recognition integrated learning module 14 mainly applies an integrated learning method, using a guided aggregation algorithm (Bagging) or a gradual boosting algorithm (Boosting) or a combination of the above two methods to reduce model variation and reduce model errors.

Specifically, the intent recognition integrated learning module 14 can select training data through sample and feature selectors to form multiple sets of training data subsets, and each batch of training data corresponds to a set of intent recognizers. Training, the intent recognizer includes but is not limited to use: pattern matching, statistical methods (regression analysis), machine learning models (SVM, neural network, decision tree...), deep learning models (RNN, LSTM, DNN, CNN …) and other individual or combined methods, and then integrate the intention decision results of multiple groups of intention recognizers through the intention recognition integrator to produce the final intention classification decision. In addition, the intention recognition integrator includes, but is not limited to, the use of weighted average method, voting method and other integration methods.

In summary, the intention recognition integrated learning module 14 receives the unlabeled data vocabulary and vector and the labeled data vocabulary and vector from the text preprocessing module 12, and the dialogue topic distribution of the semi-supervised dialogue topic module 13, and then outputs The intention result of the current dialogue text, and send the result to the knowledge base search module. In actual implementation, the intention recognition integrated learning module 14 can be a processing program that enhances the ability of dialog intention prediction through integrated learning on the server side or the cloud.

The knowledge base search module 15 makes inquiries based on the final intent classification decision to obtain the system reply content from the database. The knowledge base search module 15 searches the knowledge base with the dialogue intention received by the intent identification integrated learning module 14, so as to determine the system reply content, and then transmits the system reply content to the system reply module 16, wherein the system reply content type Contains, but is not limited to, responding to user questions, ongoing conversations, or user confirmations. In specific implementation, the knowledge base search module 15 can be a database module that stores the dialogue intent and response sentence on the server or the cloud, and can provide indexing, search, and fuzzy comparison functions.

The system reply module 16 transmits the system reply content to the user, and receives the reply data returned by the user regarding whether the system reply content is correct. The system reply module 16 replies the system reply content generated by the knowledge base search module 15 to the user, and then determines the correctness of the reply data to confirm whether the dialogue system reply is correct, and it will answer the question sentence (Q) and answer sentence (A) , Record the corresponding data of Intent and Reply, and pass the part of the reply error (that is, the bad reply) to the system marking module 17 for data marking.

In specific implementation, the system reply module 16 can be an interface for replying to a text message to the user, and it can provide a reply screen to the client in a client-server web page display mode or a mobile application (APP) mode. (client) Watching dialogue responses, clicking text hyperlinks, and playing voice or video multimedia, etc. The user response interface provided by the system response module 16, which can be designed as a "like" or "dislike" button or mark response, and then these question sentences (Q), answer sentences (A), intent (Intent) and reply The evaluation (Reply) records the corresponding data and feeds it back to the system marking module 17.

When the reply data is incorrect, the system marking module 17 marks the text input by the user as the correct intention category and imports it into the text preprocessing module to generate a new vocabulary vector. Update the semi-supervised dialogue topic model to optimize the output topic machine Rate distribution, and then readjust the intent category predicted by the intent recognition integrated learning module. The main purpose of the system labeling module 17 is to update the intention category to improve the accuracy of intent identification. It receives the wrong user text input data from the system reply module 16 and marks the wrong data as the correct intention category, so that it can be updated. Increase the number of labeled training data into the text pre-processing module 12 to generate new vocabulary vectors, and then update the semi-supervised dialogue topic module 13 to output a new topic probability distribution, by re-adjusting the intention recognition integrated learning module 14 It can more accurately predict the intent category, search for more suitable reply content in the knowledge base and return it to the user, so as to complete the improvement of the performance of the dialogue system.

The labeling method of the system labeling module 17 includes, but is not limited to, manual labeling, automatic system prediction labeling, or a combination thereof. The system labeling module 17 can also use the semi-supervised dialogue topic module 13 to find similar sentences with the same subject meaning as the wrong data, thereby assisting manual labeling or using the nearest neighbor algorithm to find k in the dialogue subject space. The most similar marked data, and exclude the wrong data from the intention category predicted by the intention recognition integrated learning module 14 so as to automatically mark the wrong data as the intention category with the highest vote.

In specific implementation, the system labeling module 17 may be an application system that performs data labeling on the server side or the cloud. The system labeling module 17 can provide manual labeling and system automatic labeling functions. The manual labeling provides an operation management interface for labeling personnel to operate and use. It can be a web page database presentation method, including but not limited to dialogue sentence related data presentation, dialogue sentence Theme distribution display interface, mark input interface, graphical operation interface, chart trend display, authority management control, etc., and the system is automatically marked as an algorithm program, which can be found and marked in the semi-supervised dialogue theme module 13 Top m topics that are similar to the dialogue sentence, and select n marked dialogue data (that is, the question sentence (Q) corresponds to the intention (Intent)) among the m topics, and exclude this dialogue first The intention of the previous misclassification of the sentence, and then according to the natural language semantic similarity algorithm to find the most similar k dialogue sentences, and perform the majority list It's up to you to decide what the intention of automatic labeling is. After manual labeling or system automatic labeling, these labeled data can be imported into the semi-supervised dialogue topic model of the semi-supervised dialogue topic module 13 and the model update with the intent recognition integrated learning module 14, so as to refine the overall dialogue intention Recognition ability.

Figure 3 is the execution flow chart of the text pre-processing module in the system of the present invention. Please also refer to Figure 1. As shown in the figure, the sentence received from the received text input module 11 will be stored in the database of unmarked data 121 in the text preprocessing module 12. More specifically, the text preprocessing The main input source of the module 12 is divided into two parts, one is the text question sentence input by the user received by the text input module 11 (that is, the data source of the database of the unmarked data 121), and the other is the module marked by the system. The group 17 generates a dialogue text question based on the previous system feedback and user feedback and the intent category marked corresponding to the sentence (that is, the data source of the database of the marked data 122). The content of the dialogue text question received from the aforementioned two sources is imported into the text pre-processing module 12, and the vocabulary vector representation after the text pre-processing can be output.

The text pre-processing module 12 will execute programs including but not limited to sentence normalization 123, sentence segmentation and de-duplication 124, and vocabulary vectorization 125. Among them, the sentence normalization 123 will filter and encode specific symbols or languages. Conversion, sentence segmentation and deduplication 124 will separate the text input by the user in word units and remove redundant words based on stopwords, and vocabulary vectorization 125 will convert the segmented vocabulary into a vector representation Finally, the vocabulary and vector are stored in the database of unlabeled data vocabulary and vector 126 and the database of labeled data vocabulary and vector 127.

Figure 4 is an execution flow chart of the integrated learning module for intention recognition in the system of the present invention. Please also refer to Figure 1. Intent identification integrated learning module 14 uses integrated learning methods to reduce model variation and reduce model errors by using guided aggregation algorithm (Bagging) or step-by-step boosting algorithm (Boosting) or a combination of the above two methods. The training method is to concatenate the vocabulary and vector processed by the text pre-processing module 12 and the topic probability distribution result generated after importing into the semi-supervised dialogue topic module 13 as the input feature group. Training data is generated and imported into the intention recognition integrated learning module 14 to generate the dialogue intention of the sentence.

As shown in the figure, the intent recognition integrated learning module 14 may include a sample and feature selector 141, an intent recognizer 142, and an intent recognition integrator 143. The input features are composed of a dialogue text vector and a dialogue topic distribution. The feature selector 141 selects multiple sets of training subsets and transmits them to multiple intent recognizers 142 for training. Finally, the intent recognition integrator 143 integrates the final intent recognition results, and returns misclassified samples to the sample and feature selector 141 to adjust the weights. Iterative training is repeated until the correct rate converges. Specifically, the intention recognition integrated learning module 14 has a sample and feature selector 141 that can perform sample and feature selection for training data to form multiple sets of training data subsets, and each batch of training data corresponds to a set of intent recognizers 142. Training, each intent recognizer 142 includes but is not limited to the use of pattern matching, statistical methods (regression analysis), machine learning models (SVM, neural network, decision tree...), deep learning models (RNN, LSTM, DNN, CNN...) and other individual or combination methods. Then, an intention recognition integrator 143 integrates the intention decision results of multiple groups of intention recognizers 142 to produce a final intention classification decision. In addition, the intention recognition integrator 143 includes, but is not limited to, the use of weighted average method, voting method and other integration methods. Finally, the intention classification predicted by the intention recognition integrator 143 and the real intention category are evaluated, and then the wrongly classified samples are selected. The sample and feature selector 141 is fed back to update the training sample selection weight, and then iterative training is repeated until the accuracy reaches a satisfactory threshold. After the training is completed, the intention recognition integrated learning module 14 can recognize the intention of the dialogue sentence.

Figure 5 is the execution flow chart of the semi-supervised dialogue topic module in the system of the present invention. Please also refer to Figure 1. The semi-supervised dialogue topic model 13 mainly uses labeled data to guide unlabeled data to analyze the implicit theme meaning of dialogue sentences. The reason for using semi-supervised is that the marked data is limited and the acquisition cost is high. , While unlabeled data is relatively easy to obtain, but the modeling effect of using unlabeled data alone is less significant and difficult to explain. Therefore, combining the two data for modeling can use known types of labeled data to quote Import unlabeled data of unknown categories to generate more meaningful clustering results.

As shown in the figure, the components of the semi-supervised dialogue topic model 13 include a bag-of-words model 131, a term frequency-inverse document frequency (TFIDF) model 132, and a semi-supervised linear judgment analysis (LDA, Linear Discriminant Analysis) Dialogue topic model 133, and finally a dialogue topic distribution 134 will be produced. Accordingly, the semi-supervised dialogue topic model 13 is based on the unlabeled data (the unlabeled data vocabulary and vector 126 in Figure 3) and the labeled data (the labeled data vocabulary in Figure 3) output by the previous text preprocessing module 12 And vector 127) for semi-supervised dialogue topic model training modeling, semi-supervised dialogue topic model modeling methods include but not limited to the use of pattern matching, machine learning models (LDA, LSI...), depth Learning model (TopicRNN, LSTM+LDA...) and other individual or combination methods, when the semi-supervised dialogue topic model training is completed, input the sentence of the subject to be analyzed into the semi-supervised dialogue topic model to generate the segment The probability distribution of text topics.

Figure 6 is a step diagram of the dialogue method using intent detection integrated learning of the present invention.

In step S61, the text content input by the user is received. In this step, the text content entered by the user is received. In one embodiment, if the user's actual input is voice, the application front end can be connected to a voice recognizer, and after the voice is converted to text, the subsequent processing can be performed.

In step S62, the text content input by the user is defined as unmarked data, and the unmarked data and the existing marked data are subjected to text pre-processing to convert the vocabulary into a vector representation. In this step, the text content input by the user is pre-processed and the text pre-processing result is output. The source of the text content can be the text content input by the user received in the previous step S61 (that is, the unmarked data), or the marked data ( The following step S67), the text pre-processing includes but is not limited to sentence normalization, sentence segmentation, and vocabulary vectorization. The sentence regularization will filter out specific symbols or language and convert the code. The sentence segmentation will input the user The text content is separated by words, and vocabulary vectorization will After the word segmentation, the vocabulary is converted into a vector representation, and finally the results of the pre-processing of the text can be divided into marked and unmarked contents and stored in different databases.

In step S63, the unmarked data and the marked data are imported into the semi-supervised dialogue topic model to output the dialogue topic distribution. This step is to import the marked and unmarked data into the semi-supervised dialogue topic model, and then obtain the dialogue topic distribution. As mentioned above, a large number of unmarked dialogue text content is used to establish the topic model, and the marked data is used to guide the unmarked The data is analyzed for the implicit theme meaning of the dialogue sentence to produce a more meaningful grouping result, that is, the final distribution of the dialogue theme can be the theme feature of the later training data (that is, through the intention recognizer).

In step S64, the dialogue topic distribution and dialogue text content are combined as input, multiple intention decision results and methods of strengthening error sample learning are integrated to produce a final intention classification decision. In this step, combining the dialogue topic distribution and dialogue text content as input, integrating multiple intent recognizers and methods of strengthening error sample learning, and selecting samples and features for training data to form multiple sets of training data subsets. Each batch of training data is trained for a set of intent recognizers, and then multiple intent decision results are integrated to produce the final intent classification decision. In addition, in the end, the final intention and the true intention are measured, and the misclassified training samples are fed back to increase their weight for the next selection, and then iterative training is repeated until the accuracy reaches a satisfactory threshold. According to the above, the training After that, input the current dialogue text content and the dialogue topic distribution situation produced in step S64 to identify the dialogue intention.

In step S65, the system's reply content is determined according to the final intention classification. In this step, you can query through the database. The system reply content type includes, but is not limited to, replying to user questions, continuous dialogue, and user confirmation.

In step S66, reply the system reply content to the user, and receive the user’s Feedback data to determine the correctness of the response data. In this step, the system reply content generated by the system will be replies to the user, and then the correctness of the reply data will be judged to confirm whether the dialogue system reply is correct.

In step S67, the text content and feedback data of the user are correspondingly annotated and updated, and then the annotated data is imported into the semi-supervised dialogue topic model for updating and learning. In this step, the user’s question and user feedback are correspondingly labeled and updated, and the labeled data is imported into the semi-supervised dialogue topic model for model updating and learning to refine the system.

Hereinafter, referring to FIGS. 1-4, a specific example will be used to illustrate the operation of each component in an embodiment of the dialog system using semi-supervised intention detection integrated learning of the present invention.

First, the receiving text input module 11 receives the text content entered by the user in the dialog system through the text input interface. For example, in this embodiment, " I want to check the international roaming plan in Japan@#$@ ". The above will be passed to the text pre-processing module 12. Then, the text pre-processing module 12 will firstly normalize the user text input 123, retain only Chinese and English and a small part of punctuation marks, and convert the Chinese into traditional characters. At this time, the text of the example is changed to " I want to check Japan's international roaming plan ".

Next, the sentence segmentation and deduplication 124 are performed, the sentence is divided into a vocabulary according to the word breaker and the words listed in the stopwords list (stopwords) are removed. In this embodiment, the sentence segmentation is first "I want to check the Japan international roaming Solution", and assuming that words "I" with the "program" appears to stop vocabulary, so the final result after going off the word is superfluous "to investigate the Japan international roaming." Next, vocabulary vectorization 125 is performed to convert the segmented vocabulary into its representative vector. In this embodiment, the One-Hot vector representation method is adopted. The vector length is the dictionary size, and each dimension represents a word in the dictionary, and each vocabulary The One-Hot vector only has a representative dimension of 1, and the other dimensions are all 0. For example , a One-Hot vector of "Japan " is represented as [0,1,0,0,0,0,0]. Finally, the text normalization and word segmentation results and the vocabulary vectorization results are integrated as the text pre-processing results and passed to the semi-supervised dialogue topic model in the semi-supervised dialogue topic module 13. Therefore, in this embodiment, input " I want to check Japan’s international roaming plan@#$@ " into the text preprocessing module 12, and the text preprocessing result will include text normalization and word segmentation to remove redundant results: " To check Japan International Roaming " and the result of word vectorization " To check =[1,0,0,0,0,0,0]; Japan =[0,1,0,0,0,0,0]; International =[0 ,0,1,0,0,0,0]; Roaming =[0,0,0,1,0,0,0]".

Next, the semi-supervised dialogue topic model trains a topic model through pre-collected training data, that is, unlabeled data vocabulary and vector 126 and labeled data vocabulary and vector 127. In this embodiment, taking the semi-supervised Latent Dirichlet allocation (LDA) as an example, since the LDA model must be performed in a frequency-based representation, it is necessary to perform bag-of-words on the training data. Model (bag of words, BOW) 131 conversion, and in order to avoid the topic is often occupied by high-frequency words, it can also be converted by TFIDF model 132.

；(4)從詞彙的多項式分布

中採樣最終生成詞彙W _m,n 。 The implementation steps of the traditional LDA model generation are as follows: (1) Sampling from the Dirichlet distribution α to generate the topic distribution θ _{m of the} article m; (2) Sampling from the topic polynomial distribution θ _m to generate the topic Z of the nth word of the article _m,n ; (3) Sampling from Dirichlet distribution β to generate the vocabulary distribution corresponding to topic Z _m,n

; (4) From the polynomial distribution of the vocabulary

Medium sampling finally generates vocabulary W _m,n .

The above method is an unsupervised learning algorithm (that is, no labeled data is used). A simple way to add labeled data to this unsupervised model is to increase the probability of certain important words in a specific topic. . In this embodiment , it is assumed that a topic Z _i is " Japan International Roaming ", and we think that the vocabulary group {"日本","Go Abroad"} is highly related to this topic, so the topic Z _{i of} "Japan International Roaming " is added when the data is marked. The keyword of is marked as W _key ={ Japan,Going Abroad }, and the probability of the keyword appearing in the topic is P( Z _i | W _key )=1. In this way, in the LDA training process, input these labeled data, and whenever the labeled topic matches the keyword, the probability that the keyword belongs to the topic can be enhanced. In addition, a probability threshold can also be set to control the degree of influence of the marked data, that is, a random number between 0 and 1 is randomly generated, and this method is executed when the threshold is less than this threshold.

After training the semi-supervised LDA dialogue topic model 133, you can get which keywords each topic is composed of according to its weight. For example, the topic Z _{i is} composed of {0.5*Japan+0.3*Going Abroad+0.1*International+0.1*Roaming} Composition, so we can say that this topic should be named " Japan International Roaming ". Suppose we pre-set a total of 100 topics, and enter the sentence " I want to go abroad to Japan " when predicting. After pre-processing of the text, the sentence is broken into "Going abroad to Japan ", and the scores of this sentence belonging to the topic are calculated for all the topics Z ₀ ~ Z _99. Then, we can get the 100-dimensional prediction of the topic distribution 134 of this sentence as Topics ¹⁰⁰ = [0.12,0.3,0.8,...].

Then, the intention recognition integrated learning module 14 will concatenate the pre-built training data, that is, the vocabulary vector processed by the text preprocessing module 12 and the topic probability distribution result generated after importing the semi-supervised dialogue topic model. The training data is formed as input features, and the training data is imported into the intention recognition integrated learning module 14 to produce the dialogue intention of the sentence. In this embodiment, as shown in Figure 4, the training data is selected through the sample and feature selector 141 for sample and feature selection. The initial selection method can be a random selection method of extraction and replacement, and n sets of training data are selected as a subset T of size m. ₁ ~ T _n . Each batch of training data corresponds to a set of intent recognizer 142 for training. In this embodiment, SVM is used as the classification method of intent recognizer 142. Therefore, n groups of SVM models SVM ₁ ~ SVM _n need to be trained separately, and the same input sentence is input after training For each group of SVM _i models, the intent Intent _{i to} which the group belongs can be identified individually.

Then, an intent recognition integrator 143 integrates the intent Intent _i results of each group of SVM _i to produce a final intent classification decision. Here, the algorithm of the intention recognition integrator 143 can use a majority vote to determine the final intention. That is, to collect all intent decisions Intent _{1~ n to} find the category with the most votes is the final intent. Finally, after measuring the final intention and the true intention of the sentence, the misclassified training samples are fed back to the sample and feature selector 141 to increase the weight of their selection next time. In this embodiment, the weight update method can be implemented in the following Adaboost weight update method:

Then, iteratively train this model until the accuracy of the model reaches a satisfactory threshold, which can be, for example, 0.95. After the training is completed, the intention recognition integrated learning module 14 can recognize the intention of the dialogue sentence.

Then, the knowledge base search module 15 searches the knowledge base based on the dialogue intention generated by the intention recognition integrated learning module 14, and then determines the content of the system reply. The type of system reply includes but is not limited to replying to user questions, continuous dialogue, and user confirmation. For example, if the intent is identified as " Japan International Roaming " after searching the knowledge base, it can return to the user " Please use the APP to log in and select the international roaming option -> Japan International Roaming, select the desired use date, and send it out ."

The main function of the system reply module 16 is to receive the system reply content generated by the knowledge base search module 15 and reply to the user. Then, the correctness of the reply data is judged to confirm whether the dialogue system reply is correct, and it will reply the wrong user text The input data is recorded and sent to the system labeling module 17 for data labeling. In this embodiment, the answer sentence "Please log in with the APP, select the international roaming option, select the desired use date, and send it" will be sent to the user and receive user reviews. The system can design a rating interface on the reply message. The user can click the evaluation result " like " or " dislike " on the receiving screen, and the system reply module 16 can assign these question sentences (Q), answer sentences (A), Record the corresponding data of the Intent and Reply, for example: {Q: " I want to go abroad to Japan ", A: " Please use the APP to log in and select the international roaming option -> Japan international roaming, select the desired Use date, just send it out ", Intent: " Japan International Roaming ", Reply: Like}.

The system labeling module 17 is an advanced module that updates the intent category to improve the accuracy of intent recognition. It receives the system reply module 16 feedback as wrong user text input data and marks the wrong data as the correct intent category. In this way, Able to add multiple labeled training data to the text pre-processing module 12 to generate new vocabulary vectors, update the semi-supervised dialogue topic model to output new topic distributions, readjust the intention recognition integrated learning module 14 to make predictions more accurate It can search for more suitable reply content in the knowledge base and return it to the user to complete the improvement of the performance of the dialogue system.

The labeling methods of the system labeling module 17 include, but are not limited to, manual labeling, system automatic prediction labeling, or a combination thereof. A semi-supervised dialogue topic model can also be used to find similar sentences with the same theme meaning as the wrong data for assistance Manually label or use the nearest neighbor algorithm to find the k closest labeled data in the dialogue topic space, and exclude the wrong data from the intention category predicted by the intention recognition integrated learning module 14, thereby automatically making this error The information is marked as the category of intent with the highest vote. In this embodiment, suppose that the system response module 16 returns a data pair evaluated as wrong (the user dislikes) {Q: " I want to call Japan ", A: " Log in with the app and select the international roaming option- >Japan International Roaming, select the desired use date and send it out ", Intent: " Japan International Roaming ", Reply: I don't like it }. The system tagging module uses manual tagging to update {Q: " I want to call Japan ", Intent: " make an international call "}, and use the semi-supervised dialogue topic model to find the word segmentation result of Q " Call Japan". The two closest themes are Z _i {0.5*Japan+0.3*Go abroad+0.1*International+0.1*Roaming} (with a score of 0.5) and Z _j {0.4*Call+0.4*International+0.1*Call}( The score is 0.4), and the sentences of the two topics are listed separately ( Z _i : I want to apply for international roaming in Japan, Z _j : I want to make international calls ) for manual reference. When the system labeling module 17 uses automatic labeling, it will find out the labeled samples in Z _i and Z _j . Suppose there are 5 pens as S ₁ {Q: I want to apply for Japan International Roaming, Intent: " Japan International Roaming "}, S ₂ {Q: I want to go to Japan and need roaming, Intent: " Japan International Roaming "}, S ₃ {Q: I want to make an international call, Intent: " Make an international call "}, S ₄ {Q: I want to make a call to Korea, Intent: "international calls"}, S ₅ {Q: I want to make a call to Taipei, Intent: "local calls"}, exclude the original error mark "Japan international roaming" _{1, 2,} left after the S S _{Three strokes of 3, 4, and 5} , after using the nearest neighbor algorithm, the intention of automatic labeling is " make an international call ".

In summary, the dialogue system and method using intent detection integrated learning proposed in the present invention apply semi-supervised dialogue topic models and intent detection integrated learning to improve the accuracy of the dialogue system, compared with the previous known techniques It has the following characteristics and functions: First, a large number of unlabeled dialogue text content is used to build a theme model to generate the theme characteristics of the training data, which can increase the diversity of the input characteristics of the intent recognizer to provide accurate intent recognition and reduce the number of conversations. ; Second, it has a rapid feedback and refinement mechanism, which can assist AI trainers in tagging data in the labeling system through a semi-supervised dialogue topic model based on user feedback and system feedback, and can strengthen the training of the model based on the difference between user answers and robot recognition results. Make the system performance more improved; and third, combine the update mode of integrated learning to reduce the error and variation of the dialogue intention topic model, so as to train an effective and real-time update model.

The above-mentioned embodiments only exemplarily illustrate the principles and effects of the present invention, and are not intended to limit the present invention. Anyone who is familiar with this technique can modify and change the above-mentioned embodiments without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the rights of the present invention should be listed in the scope of patent application described later.

1‧‧‧The use of intent detection integrated learning dialogue system

11‧‧‧Receive text input module

12‧‧‧Text preprocessing module

13‧‧‧Semi-supervised dialogue theme module

14‧‧‧Intention recognition integrated learning module

15‧‧‧Knowledge base search module

16‧‧‧System Response Module

17‧‧‧System Marking Module

Claims (12)

A dialogue system using intent detection integrated learning includes: a text input module for receiving text content; a text preprocessing module for receiving the text content to define it as unmarked data, the unmarked data and Existing labeled data is pre-processed to convert vocabulary into vector representation; semi-supervised dialogue topic module is a semi-supervised dialogue topic model used to produce dialogue topic distribution, where the semi-supervised dialogue topic model The dialogue topic model is based on the unlabeled data vocabulary and vector from the text preprocessing module and the training data composed of the labeled data vocabulary and vector; the intention recognition integrated learning module has samples and features A selector, an intent recognizer, and an intent recognition integrator. The sample and feature selector select samples and features for the training data to form multiple sets of training data subsets, and each of the training data subsets corresponds to an intent recognizer to Perform training, and then integrate multiple intent decision results of the intent recognizer through the intent recognition integrator to produce the final intent classification decision; the knowledge base search module searches through the final intent classification decision to make The database obtains the system response content; the system response module sends the system response content to receive the response data about whether the system response content is correct; and the system mark module because the response data is that the system response content is incorrect When, mark the text content as the correct intention category and import it into the text pre-processing module to become the existing marked data and further generate a new vocabulary vector, and update the semi-supervised dialogue theme accordingly The model optimizes the output topic probability distribution, and then readjusts the intention category predicted by the intention recognition integrated learning module. For example, the use intention detection integrated learning dialogue system described in the scope of the patent application, wherein the text pre-processing module further includes: a sentence normalization unit, which is used to filter out specific symbols or languages and perform code conversion ; The sentence segmentation unit is used to separate the text content in units of words; and the vocabulary vectorization unit is used to convert the segmented vocabulary into a vector representation. For example, the dialogue system using intent detection integrated learning described in the scope of patent application, wherein the intent recognition integrated learning module adopts a guided aggregation algorithm or a gradual improvement algorithm or a combination thereof for integrated learning, thereby reducing model variation and Reduce model errors. As described in the first item of the scope of patent application, the dialogue system using intent detection integrated learning, wherein the intent recognition integrated learning module includes the final intent classification provided by the intent recognition integrator and the true intent of the text content The category is measured to feed the misclassified training sample to the sample and feature selector to increase the weight of the misclassified training sample being selected, and iterative training is performed until the correct rate reaches the threshold. As described in the first item of the scope of patent application, the dialogue system for using intent detection integrated learning, wherein the response content of the system includes replying to user questions, continuous dialogue or user confirmation. For example, the dialogue system using intent detection integrated learning described in the first item of the scope of patent application, wherein the system labeling module uses the semi-supervised dialogue topic module to find similar sentences with the same theme meaning as the wrong data. Use auxiliary annotation or use nearest neighbor algorithm to find the k closest marked data in the dialogue topic space, and eliminate the wrong data The intention category originally predicted by the intention recognition integrated learning module is used to mark the wrong data as the intention category with the highest vote. A dialogue method using semi-supervised intent detection integrated learning, including: receiving text content; defining the text content as unmarked data, and pre-processing the unmarked data and the existing marked data to convert the vocabulary It is a vector representation; the unlabeled data and the labeled data are imported into a semi-supervised dialogue topic model to output the dialogue topic distribution; combined with the dialogue topic distribution and dialogue text content as input, multiple intentional decision results are integrated and error samples are strengthened The method of learning is to produce the final intention classification decision; determine the system response content according to the final intention classification; reply the system response content to the user to receive the user's feedback data to determine the correctness of the response data; and The text content and feedback data are correspondingly annotated and updated, and the annotated data is imported into the semi-supervised dialogue topic model for updating and learning. Wherein, correspondingly annotating and updating the text content and the feedback data includes the The text content is marked into the correct intention category to become the existing marked data. As described in item 7 of the scope of patent application, the dialogue method using semi-supervised intent detection integrated learning, wherein the text pre-processing includes sentence normalization, sentence segmentation and vocabulary vectorization. As described in item 7 of the scope of patent application, the semi-supervised intent detection ensemble learning dialogue method, wherein the output of the decision of the final intent classification includes the unmarked The data vocabulary and vector and the labeled data vocabulary and vector are sampled and feature selected to form multiple sets of training data subsets, and each of the training data subsets is trained to integrate the multiple intention decision results. For example, the dialog method using semi-supervised intent detection ensemble learning as described in item 7 of the scope of patent application, wherein the method of enhancing error sample learning includes measuring the final intent classification and the true intent category of the text content, The wrongly classified training samples are fed back to increase the weight of the wrongly classified training samples being selected, and the training is repeated iteratively until the correct rate reaches the threshold. As described in item 7 of the scope of patent application, the dialogue method using semi-supervised intent detection integrated learning, wherein the system reply content includes reply to user questions, continuous dialogue or user confirmation. For example, the dialog method using semi-supervised intent detection integrated learning described in item 7 of the scope of patent application, wherein, correspondingly annotating and updating the text content and feedback data refers to using the semi-supervised dialog topic model to find out and For similar sentences with the same subject meaning of the wrong data, use auxiliary annotation or use the nearest neighbor algorithm to find the k closest marked data in the dialogue topic space, and eliminate the intention category originally predicted by the wrong data. Mark the erroneous material as the category of intent with the highest vote.

Images