TWM645216U - Reply Generation System Based on Intent Classification and Keyword Extraction - Google Patents

Abstract

This new model provides a reply generation system based on intention classification and keyword extraction, which includes a dialogue robot module, an intention classification module, a keyword extraction module and a reply generation module. After the dialogue robot module reads the dialogue information, , connect the conversation robot module with the intent classification module signal to read and analyze the intent contained in the conversation information, and then connect the intent classification module with the keyword extraction module signal to respond to the intent classification result from the conversation information. Extract keyword groups as the basis for reply generation. Finally, the reply generation module signal connects the keyword extraction module and the conversation robot module to generate real-time replies according to the reply instructions generated by the keyword extraction module, and through the conversation robot The module outputs this instant reply.

Description

Reply generation system based on intent classification and keyword extraction

The present invention relates to a dialogue reply generation system, in particular, an intelligent reply generation system that can identify the user's intention and can be combined with a feedback mechanism to optimize reply content.

Chatbot is a computer program that communicates with users through voice or text dialogue. It has the ability to simulate human dialogue. Today, chatbots are widely used in the field of e-commerce to solve general customer service problems or extract information by talking to customers. Information; some conventional chat robots are equipped with natural language processing systems, but most systems can only judge the input keywords and find appropriate response information in the database; there are other question and answer situations that chat robots in conventional technologies can handle. limitations, and make its responses less accurate and less humane.

For example, the "Method and System for Providing User Self-Service" disclosed in the Republic of China Patent Announcement No. I751567 uses an artificial intelligence semantic analysis module to analyze the conversation process uploaded by the chatbot module, and decides to provide corresponding services or subsequently transfer to a real person. service, the system cannot generate accurate reply content through its own feedback mechanism; another example is that the "intelligent voice customer service system" disclosed in the Republic of China Patent Announcement No. M615272 uses a semantic content analysis module to read text record files. After analyzing the semantic results, the appropriate response is searched for in the dialogue response database. There is also a lack of feedback mechanism to optimize the accuracy of the response; another example is the "dialogue interactive system" disclosed in the Republic of China Patent Announcement No. M609010. ”, although this patented technology also uses a sentence intent analysis module to identify the input sentence and generate keywords, it still needs to select the appropriate reply template from multiple reply templates, and it is still unable to generate a response that is both accurate and human-like. Reply content.

In order to provide accurate information and humanized responses, the chatbot must analyze the user's intention of sending a message, and then complete different responses according to the intention. To achieve the above purpose, this new model provides an intention-based A reply generation system for classification and keyword extraction, which can perform intent classification and keyword extraction on user messages, and then capture the corresponding information according to the intent in the message and the extracted keyword groups, and expand the keyword groups into complete sentences. to formulate a reply.

The invention provides a reply generation system based on intention classification and keyword extraction, which includes: a dialogue robot module, which is used to read a user information dialogue information; an intention classification module, which signals connect the dialogue robot module A group that reads and analyzes the user information dialogue information to generate an intent classification result, which includes a clear intent result or an unclear intent result; a keyword extraction module, a signal connected to the intent classification module, which responds to The intent clear result is to extract a keyword group from the user information dialogue information, wherein the keyword extraction module accesses a business information, and is used to compare the keyword group to generate a reply instruction, wherein the reply instruction includes A merchant reply instruction or a rhetorical question reply instruction; and a reply generation module, the signal is connected to the conversation robot module and the keyword extraction module, which generates an instant reply according to the reply instruction and outputs it through the conversation robot module The instant reply includes a merchant reply or a rhetorical reply, wherein the reply generation module includes: a merchant reply unit that responds to the merchant reply instruction to generate the merchant reply; and a rhetorical reply unit, The rhetorical question reply is generated in response to the rhetorical question reply instruction.

The system provided by this new model performs intent classification by reading user information, and then classifies the user information into merchant Q&A or non-merchant Q&A according to the classification results, and further extracts representative keyword groups from the merchant Q&A information. Then compare it with the pre-established business information to determine whether the keyword group is complete to obtain the business information corresponding to the keyword group; finally, import it into a natural language processing model (Natural language processing model) to generate a reply sentence.

The specific effect of this new model is to use a language generation model to perform intent classification and keyword extraction, and then combine it with a text-to-text natural language processing model to improve the accuracy of reply sentences and increase the number of reply sentences. The diversity of sentence patterns makes the chat robot more humane in replying; this new model not only avoids the too free generation method of the language generation model in the previous technology, resulting in the generated reply content being too divergent and causing the text to be incorrect, but also It requires a large amount of data set training to provide correct information responses.

100: Reply generation system based on intent classification and keyword extraction

1: Conversation robot module

11: Dialogue interface

2: Intent classification module

3: Keyword extraction module

4: Reply generation module

40:T5 model

41: Merchant reply unit

42: Rhetorical question reply unit

43: Default reply unit

5:Database

51: Business information template

52: Rhetorical question reply template

53: Default reply template

Figures 1A to 1B are a series of block diagrams illustrating the system architecture of the first embodiment of the present invention; Figure 2 is a block diagram illustrating the system architecture of the second embodiment of the present invention; Figure 3 is a block diagram illustrating the system architecture of the second embodiment of the present invention. To illustrate the system architecture diagram of the third embodiment of the present invention; Figure 4 is a flow chart to illustrate the working principle of the present invention during execution.

In order to explain the present invention more clearly, several embodiments are listed below and described in detail with drawings; please refer to Figures 1A to 1B, which are a reply generation system (100) based on intent classification and keyword extraction provided by the present invention. ), which can be equipped with electronic devices that require intensive human-computer interaction, such as personal computers, smart phones, tablets, smart audio and video systems, car navigation systems, e-commerce customer service systems, etc. Specifically, the system ( 100) includes a conversation robot module (1), an intent classification module (2), a keyword extraction module (3) and a reply generation module (4).

The conversation robot module (1) is used to read a conversation information, and has a conversation interface (11) for interacting with the user; the conversation robot module (1) can conduct conversations with the user through the conversation interface (11). Obtain the dialogue information, where the dialogue interface (11) can be, for example, a voice or text input platform, specifically a voice call channel or a text chat room in an e-commerce customer service system.

The intention classification module (2) is signal-connected to the conversation robot module (1), which reads and analyzes the conversation information to generate an intent classification result, which includes a clear-intention result or an unclear-intention result; specifically, , the intention classification module (2) analyzes the dialogue information through the language generation model, and compares it with the preset intention type to generate the intention classification result. If the sentences contained in the dialogue information are consistent with the preset dialogue The types are the same, for example, comment-type statements such as "The black tea in your store is full of fragrance" or business question-and-answer statements such as "How much does a dress cost in your store?" will produce a result with clear intent; if the dialogue information contains The statement cannot conform to any preset dialogue type, and the result of unclear intention is generated; in some implementation cases of the present invention, the result of clear intention corresponds to merchant Q&A, and the result of unclear intention corresponds to non-merchant Q&A. .

The keyword extraction module (3) is signal-connected to the intent classification module (2), which responds to the intent clarification result to extract a keyword group from the conversation information, wherein the keyword extraction module (3) accesses A merchant information is used to compare the keyword group to generate a reply instruction, wherein the reply instruction includes a merchant reply instruction or a rhetorical reply instruction; specifically, the merchant information may be stored in the system (100) The data can be obtained from a database or from the website information of a specific merchant through a link network; for example, the merchant information can include merchant name, address, phone number, service items, product information such as product specifications, product prices, or Inventory quantity, payment methods, etc., any merchant information required to use the system (100) for customer service response are all items on which this new model can implement the aforementioned functions.

The reply generation module (4) is connected by signal to the conversation robot module (1) and the keyword extraction module (3). It generates an instant reply according to the reply instruction and outputs it through the conversation robot module (1). The instant reply includes a merchant reply or a rhetorical question reply, wherein the reply generation module (4) includes a merchant reply unit (41) and a rhetorical question reply unit (42); the merchant reply unit (41) In response to the merchant reply instruction to generate the merchant reply, the rhetorical question reply unit (42) responds to the reply instruction to generate the rhetorical question reply; specifically, the keyword extraction module (3) is in response to the keyword group. The same as the merchant information to output the merchant reply instruction, which is used to instruct the merchant reply unit (41) to generate a merchant reply, or the keyword extraction module (3) responds to the keyword group being different from the merchant information. Output the rhetorical question reply instruction, which is used to instruct the rhetorical question reply unit (42) to generate a rhetorical question reply; for example, when there is a keyword "address" in the keyword group, through the keyword extraction module (3 ) compares the merchant information and it is the same as the "merchant address", then the merchant reply instruction is issued to call the merchant reply unit (41) to output the merchant reply, and the user can obtain the "merchant address" related information through the dialogue interface (11); in addition, On the one hand, if only keywords such as "live" and "where" appear in the keyword group, and it is impossible to compare any keywords that are the same as those in the business information, a rhetorical question reply instruction is issued to call the output of the rhetorical question reply unit (42) Reply to the question, for example, reply with "Do you want to ask for the address of our store?" in the hope that the user can obtain more accurate conversation information.

In this embodiment, as shown in FIG. 1B , the reply generation module (4) further includes a preset reply unit (43) that responds to a preset reply instruction to generate a preset reply, wherein the preset reply is Assume that the reply instruction is generated by the keyword extraction module (3) in response to the unclear intent result; specifically, if the intent classification module (2) generates an unclear intent result, then the keyword extraction module The group (3) generates a preset reply instruction in response to the unclear intention result, the preset reply unit (43) generates a preset reply according to the preset reply instruction, and outputs it through the conversation robot module (1) The default reply; for example, when the sentence contained in the conversation information does not match any default conversation type, for example, "Hello, I am so-and-so" is a greeting type rather than a business Q&A, then the reply The generation module (4) can generate default responses such as "Hello, how can I help you?" or "Hello, this is the XXX store, how can I serve you?" and there are no special restrictions.

Please refer to Figure 2, which is a second embodiment of a reply generation system (100) based on intent classification and keyword extraction provided by the present invention. The basic components and operating mechanism are the same as those in the first embodiment. To reiterate, the merchant reply unit (41) is based on a natural language processing model to respond to the merchant reply instructions to generate the merchant reply. More specifically, the natural language processing model (Natural Language Processing model, NLP) is a T5 model (40) to generate the merchant reply in response to the merchant reply instruction, wherein the T5 model (40) is based on the Transformer encoder-decoder architecture; in the T5 model (40) architecture , all NLP tasks are restructured into a unified text-to-text format. All inputs and outputs are text representations, allowing the same model and loss function to be used on any NLP task. and hyperparameters.

Please refer to Figure 3, which is the third embodiment of a reply generation system (100) based on intent classification and keyword extraction provided by the present invention. The basic components and operating mechanism are the same as those in the first embodiment. To reiterate, the merchant reply unit (41) is based on a T5 model (40) to respond to the merchant reply instruction to generate the merchant reply, and the system (100) further includes a database (5), and the signal is connected to the merchant reply unit (41). The keyword extraction module (3) and the reply generation module (4) pre-store multiple business information templates (51), multiple rhetorical reply templates (52) and multiple default reply templates (53); required It is noted that the business information templates (51), multiple rhetorical reply templates (52) and multiple default reply templates (53) may have been pre-stored in the database (5) before the system (100) is initialized. ), or crawl from the website information of a specific merchant through the database (5) to connect to the Internet, without special restrictions.

In the third embodiment, the keyword extraction module (3) accesses the merchant information from the merchant information templates (51).

In the third embodiment, the rhetorical question reply unit (42) responds to the rhetorical question reply instruction to access the rhetorical question reply from the rhetorical question reply templates (52).

In the third embodiment, the default reply unit (43) responds to the default reply instruction to access the default reply from the default reply templates (53).

In the above embodiment, the intent classification module (2) is based on a language generation model to analyze the sentences used in the dialogue information to generate the intent classification result, specifically a bidirectional encoding conversion. Bidirectional Encoder Representations from Transformers model (BERT model) to achieve the above intention classification of dialogue information.

In the above embodiment, the keyword extraction module (3) is based on a language generation model to collect keywords from the sentences used in the dialogue information to generate the keyword group, specifically a bidirectional encoding converter model (Bidirectional Encoder Representations from Transformers model, BERT model) to achieve the extraction of the above keyword phrases; the BERT model is a method that can convert text content into specific vectors to capture its meaning, and analyze words and text content based on these specific vectors. contextual relationships between them, and finally uses the cosine similarity method to compare the words most relevant to the text content; for example, the BERT model can perform bidirectional operations on a large number of unlabeled text content databases, including the entire Wikipedia and Book Corpus. Pre-training, through training on tasks such as sentence prediction and cloze, enables the model to more accurately understand the meaning of language, and thus be more effectively used for downstream tasks such as emotion classification or intention detection.

Please refer to Figure 4, which illustrates the reply generation system (100) based on intent classification and keyword extraction provided by the present invention and performs a counter-questioning mechanism when there are insufficient keywords to obtain more required keywords to generate appropriate replies. , wherein the rhetorical question mechanism includes: the conversation robot module (1) reads a user reply information in response to the rhetorical question reply; the intent classification module (2) reads and analyzes the user information reply information to generate another intention The classification result includes another clear-intent result or another unclear-intent result; the keyword extraction module (3) responds to the other clear-intention result to extract another keyword group from the dialogue information, wherein the key The word extraction module (3) accesses another merchant information to compare with another keyword group to generate another reply instruction, wherein the other reply instruction includes another merchant reply instruction or another rhetorical reply instruction; and the The reply generation module (4) generates another real-time reply according to the another reply instruction, and outputs the other real-time reply through the conversation robot module (1). The other real-time reply includes another merchant reply or another rhetorical question. Reply, wherein the merchant reply unit (41) responds to the other merchant reply instruction Indicates to generate the merchant reply, or the rhetorical question reply unit (42) responds to the other rhetorical reply instruction to generate the rhetorical reply.

In different situations, the reply generation module (4) can generate the rhetorical question reply based on the missing keywords; in some embodiments, the reply generation module (4) can generate the rhetorical question reply from the rhetorical question reply templates (52) Access a first rhetorical question reply template or a second rhetorical question template, and the rhetorical question reply unit (42) directly outputs the first rhetorical question reply template as a rhetorical question reply, or generates the second rhetorical question reply template with the missing keywords Reply to the question and output.

Please refer to Figure 4, which is a block flow chart to illustrate the working principle of the present invention; as shown in Figure 4, after the system (100) is initialized, after the dialogue robot module (1) reads the user dialogue information, The intention classification module (2) reads the dialogue information and analyzes the specific intention of the dialogue information based on the BERT model. The generated intention classification result is read by the keyword extraction module (3) and determines its intention. Whether it is a merchant Q&A; if the intention is judged to be a merchant Q&A, then the keyword extraction module (3) responds to the intent classification result and extracts keyword groups from the conversation information based on the BERT model, and the keyword extraction module (3) 3) Access the corresponding business information from outside the system (100), such as the Internet, or from the internal database (5) of the system (100), to compare the keyword group to generate a reply instruction; if the intention is determined to be non- If the merchant asks a question, the keyword extraction module (3) responds to the intent classification result to generate a preset reply instruction.

Then, if the reply generation module (4) reads the default reply instruction, it generates a default reply to try to obtain more information from the client; if the keyword phrase is compared and judged that its content can completely correspond to the merchant information, the reply generation module (4) generates a merchant reply according to the reply instruction; if the content of the keyword phrase is judged not to fully correspond to the merchant information after comparison, in order to obtain more information from the client, the reply is generated The module (4) generates a rhetorical reply according to the reply instruction, wherein the reply generation module (4) is based on the T5 dialogue generation model to realize the generation of the above reply content.

Please continue to refer to Figure 4. If the system (100) generates a rhetorical question reply, the rhetorical question mechanism is further executed; after the user reads the rhetorical question reply generated by the system (100), the user further enters more users on the dialogue interface (11). Reply information; after the conversation robot module (1) reads the user reply information, the intent classification module (2) reads and analyzes the user information reply information to generate another intent classification result; this other intent classification As a result, since the content is different from the previous intention classification result, it still needs to be read by the keyword extraction module (3) to determine whether the intention is still a merchant Q&A; if the intention is still judged to be a merchant Q&A, the keyword extraction The module (3) responds to the other intent classification result to extract another keyword group from the user's reply information; similarly, the keyword extraction module (3) accesses the corresponding merchant information to compare the other keyword group. A keyword group to generate another reply instruction; then, if the content of the other keyword group is compared and determined to completely correspond to the merchant information, the reply generation module (4) generates a merchant reply based on the other reply instruction. ; If the content of the keyword phrase is judged after comparison to still be insufficient to completely correspond to the merchant information, in order to further obtain more information from the user end, the reply generation module (4) generates another rhetorical reply according to the reply instruction, and Start a new round of rhetorical questions.

This new model combines two language models to generate more accurate and humane reply content; this new model can detect the intention in the conversation information and extract keywords as the basis for subsequent reply generation; this new model can also use Continuous interaction with users generates reply content that is closer to the user's intention; by combining intention analysis and keyword extraction, this new model can generate corresponding replies from keyword expansion on the premise of being close to the user's intention, which not only improves the The accuracy of the reply content and the closeness to the user's original intention make the reply content more humane, making the chatbot no longer a cold information-providing machine and adding human warmth while providing information.

100: Reply generation system based on intent classification and keyword extraction

1: Conversation robot module

2: Intent classification module

3: Keyword extraction module

4: Reply generation module

41: Merchant reply unit

42: Rhetorical question reply unit

Claims (10)

A reply generation system based on intent classification and keyword extraction, which includes: a conversation robot module, which is used to read a conversation information; an intent classification module, which is connected to the conversation robot module with signals, and which reads and analyzes The dialogue information is used to generate an intention classification result, which includes an intention clear result or an intention unclear result; a keyword extraction module is signal-connected to the intention classification module, which responds to the intention clear result to extract from the dialogue information Extracting a keyword group, wherein the keyword extraction module accesses a merchant information, which is used to compare the keyword group to generate a reply instruction, wherein the reply instruction includes a merchant reply instruction or a rhetorical reply instruction; and A reply generation module, the signal is connected to the conversation robot module and the keyword extraction module, it generates an instant reply according to the reply instruction, and outputs the instant reply through the dialog robot module, the instant reply includes a merchant reply Or a rhetorical question reply, wherein the reply generation module includes: a merchant reply unit that responds to the merchant reply instruction to generate the merchant reply; and a rhetorical reply unit that responds to the rhetorical reply instruction to generate the rhetorical reply . The system of claim 1, wherein the keyword extraction module outputs the merchant reply instruction in response to the keyword group being the same as a template information, which is used to instruct the merchant reply unit to generate the merchant reply, or The keyword extraction module responds to the keyword group being different from the template information to output the rhetorical question reply instruction, which is used to instruct the rhetorical question reply unit to generate the rhetorical question reply. The system of claim 2, wherein the reply generation module further includes a preset reply unit that responds to a preset reply instruction to generate a preset reply, wherein the preset reply instruction is generated by the key The word extraction module generates results in response to this ambiguous intent. The system according to any one of claims 1 to 3, further executes a rhetorical question mechanism, including: the conversation robot module reads a user reply information in response to the rhetorical question; the intent classification module reads and analyzes The user information replies to generate another intent classification result, which includes another intent-clear result or another intent-unclear result; the keyword extraction module responds to the other intent-clear result to extract another intent classification result from the conversation information. Keyword group, wherein the keyword extraction module accesses another merchant information to compare with the other keyword group to generate another reply instruction, wherein the other reply instruction includes another merchant reply instruction or another rhetorical question reply instruction; and the reply generation module generates another real-time reply according to the another reply instruction, and outputs the other real-time reply through the conversation bot module, the other real-time reply includes another merchant reply or another rhetorical question reply, Wherein, the merchant reply unit responds to the other merchant reply instruction to generate the merchant reply, or the rhetorical question reply unit responds to the other rhetorical question reply instruction to generate the rhetorical question reply. The system of claim 4, wherein the intent classification module analyzes the sentences used by the user according to the dialogue information to generate the intent classification results, wherein the intent classification results respectively correspond to merchant Q&A or non-merchant Q&A. The system as described in claim 4, further comprising a database connected by signals to the keyword extraction module and the reply generation module, which pre-stores a plurality of business information templates, a plurality of rhetorical question reply templates and a plurality of presets Reply template, wherein the keyword extraction module accesses the business information from the business information templates. The system of claim 6, wherein the rhetorical question reply unit responds to the rhetorical question reply instruction to access the rhetorical question reply from the rhetorical question reply templates. The system of claim 6, wherein the default reply unit responds to the default reply instruction to access the default reply from the default reply templates. The system as described in claim 4, wherein the intent classification module is based on a Bidirectional Encoder Representations from Transformers (BERT) model to analyze the sentences used in the dialogue information to generate the intent classification result; wherein the keyword extraction The module is based on a Bidirectional Encoder Representations from Transformers (BERT) model to collect keywords from the sentences used in the dialogue information to generate the keyword group. The system of claim 4, wherein the merchant reply unit is based on a T5 model to generate the merchant reply in response to the merchant reply instruction, wherein the T5 model is based on a Transformer encoder-decoder architecture.