CN111553157A - Entity replacement-based dialog intention identification method - Google Patents

Abstract

The invention discloses a dialogue intention recognition method based on entity replacement, comprising the following steps: step 1, text segmentation; step 2, text filtering; step 3, text named entity recognition; step 4, text named entity replacement; step 5, text feature Extraction: Step 6, text intention recognition; to realize the dialogue intention recognition of entity replacement, this method uses the named entity recognition result to replace the entity name in the text information with the entity type, which reduces the magnitude and imbalance of the dialogue system corpus data , so as to comprehensively improve the accuracy of intention recognition in the dialogue process.

Description

A Dialogue Intent Recognition Method Based on Entity Replacement

technical field

The present invention relates to a method for identifying dialogue intentions, in particular to a method for identifying dialogue intentions based on entity replacement.

Background technique

In recent years, under the influence of the rapid development of artificial intelligence and semiconductor chip technology and the increasing demand for voice interaction, various application products based on dialogue systems, such as smart speakers, smart furniture, and smart voice customer service, are gradually blooming in the market.

Such dialogue systems generally consist of five modules: Speech Recognition (ASR), Natural Language Understanding (NLU), Dialogue Management (DM), Natural Language Generation (NLG) and Speech Synthesis (TTS). At present, the speech recognition module has a better solution using deep learning technology. The natural language generation and speech synthesis modules are relatively easy to control, and the difficulty in the design of the dialogue system mainly lies in the natural language understanding and dialogue management modules. The goal of the natural language understanding module is to convert the text information obtained by the speech recognition module into a semantic representation, so that the machine has the same language understanding ability as a human being. Therefore, the accuracy of the language understanding module is the premise and guarantee for maintaining the normal operation of the dialogue system.

With the continuous optimization and upgrading of deep learning algorithms, machine computing power and big data technology, the accuracy of intent recognition of simple dialogue systems such as voice ordering systems and voice song ordering systems has basically reached the commercial level. However, since the magnitude of corpus data and the complexity of intentions in complex dialogue systems are significantly higher than the former, problems such as imbalanced corpus data and a wide variety of intentions aggravate the difficulty of dialogue intention recognition. For example, the invention patent "Method and Device for Understanding Natural Language Intent in Human-Computer Interaction" (CN201710219326) takes the word vector of text information as input, and uses the intent recognition model to obtain the intent type of the text information. Once the distribution of training sample categories is unbalanced, the intent recognition model is prone to serious overfitting and underfitting, which has certain limitations. For example, in the invention patent "An Intent Recognition Method and Device" (CN201811368503), text information is input into at least one intention recognition model, a prediction result corresponding to each intention recognition model is generated, and the text intention is finally determined. With the increase of intent types, the cost and difficulty of model training will increase greatly, and it is not suitable for intent recognition of complex dialogue systems.

SUMMARY OF THE INVENTION

Purpose of the invention: The present invention aims to make up for the lack and inadequacy of the existing technical means, and provides a method for identifying dialogue intentions based on entity replacement; the method utilizes the named entity identification result to replace the entity name in the text information with the entity type, reducing the need for The magnitude and imbalance of the corpus data of the dialogue system are improved, so as to comprehensively improve the accuracy of intention recognition in the dialogue process.

Technical scheme: In order to realize the above-mentioned purpose of the invention, the technical scheme adopted in the present invention is:

A dialogue intent recognition method based on entity replacement, comprising the following steps:

Step 1. Text segmentation:

Use the word segmentation tool to segment the text information obtained by the speech recognition module, and obtain a word segmentation result set Token; wherein the word segmentation result is represented as a set {W}, and W represents the segmented word;

Step 2. Text filtering:

Establish the required stop word thesaurus according to the dialogue system, use the stop word thesaurus to filter the text information of the word segmentation result set Token obtained in step 1, and obtain the result Token ^* after the text cleaning;

Step 3. Text Named Entity Recognition:

Through named entity recognition, the named entity recognition result is {E:T}, where E represents the entity name and T represents the entity type;

Step 4. Text named entity replacement:

Use specific characters to map the named entity types involved in the dialogue system one-to-one, denoted as {T:C}, and recombine to obtain a new corpus, where T represents the entity type and C represents a specific character; the selected specific character needs to be Make sure that it does not exist in the corpus of the dialogue system;

Step 5. Text feature extraction:

Based on different types of pre-training models, using the new corpus obtained in step 4, fine-tune the above-mentioned pre-training model to obtain a fine-tuned feature extraction model; use the fine-tuned feature extraction model to obtain the word vector Vec of the dialogue system corpus;

Step 6. Text Intent Recognition:

The network structure of bi-directional long short-term memory Bi-LSTM + attention mechanism Attention is used to realize text intent recognition.

Further, step 3, the specific work steps of text named entity recognition, the specific work is as follows:

1) Based on rule matching,

Design corresponding regular expressions according to the requirements of the dialogue system, extract named entities based on regular expressions, and match fields that meet the requirements;

2) Based on entity dictionary

Build a corresponding named entity dictionary according to the dialogue system, and match the word segmentation results obtained in step 1 based on the named entity dictionary;

3) Model based

The original corpus Sentence is obtained by collecting the historical corpus or corpus generation of the dialogue system, and manually or automatically labeling each position in the ^Sentence to complete the sequence labeling task; It consists of ET and ST, and then realizes the model-based named entity recognition by training the named entity recognition model.

Further, in the model-based method in step 3, the sequence annotation can be in the BIO annotation mode or the BIOES annotation mode; wherein in the BIOES annotation mode, B is Begin, representing the beginning of the entity, I is Intermediate, representing the middle of the entity, and O is the Other, unrelated characters representing non-entity, E is End, representing the end of the entity, S is Single, representing that the entity consists of a single character.

Further, described step 4, the concrete working steps of Chinese text named entity replacement: replace the entity name T in the named entity recognition result {E:T} obtained in step 3 with the specific character C, obtain after the named entity replacement. The result set {E:C} is substituted into the word segmentation result Token ^* obtained in step 2, after the word W contained in the entity name E is replaced with a specific character C, the new corpus Sentence' is obtained by recombining;

Further, the network structure in the text intention recognition in step 6, the network structure is mainly composed of 4 parts, specifically:

1) Input layer: take the word vector Vec of the dialogue system corpus obtained in step 5 as the input V;

2) Bidirectional LSTM layer: use the bidirectional long short-term memory network to perform forward calculation on the word vector of the input layer to obtain a vector _VL , and backward calculation to obtain a vector _VR ; splicing the front and rear vectors to obtain the spliced LSTM layer output vector _VC , where V _C =[ _VL , _VR ];

3) Attention layer: Perform Attention weighting on the output vector _VC of the LSTM layer, and further obtain the output result VA. _The calculation method is as follows:

V _m =tanh(V _c )

α=softmax(w ^T V _m )

V _A =V _c α ^T

where w is the weight matrix of the Attention layer.

4) Output layer: the output result VA of the Attention layer is used to predict the sentence intent using the _Softmax classifier, and the intent prediction result is obtained

where W _S and b _S are the weight matrix and bias value of the output layer, respectively.

Beneficial effect: Compared with the prior art, the method uses the named entity recognition result to replace the entity name in the text information with the entity type, which reduces the magnitude and imbalance of the corpus data of the dialogue system, thereby comprehensively improving the intention of the dialogue process recognition accuracy.

Description of drawings

Fig. 1 is a schematic flow chart of a method for recognizing dialog intent based on entity replacement of the present invention;

Fig. 2 is an example of a text named entity replacement process of the present invention;

Fig. 3 is a kind of corpus sequence labeling method of the present invention;

FIG. 4 is a network structure for realizing text intention recognition according to the present invention.

Detailed ways

The present invention is further described below in conjunction with specific embodiments, but these embodiments are not intended to limit the present invention.

A dialogue intent recognition method based on entity replacement, as shown in Figure 1, the method includes the following steps:

Step 1: Text segmentation

A word segmentation tool is used to segment the text information obtained by the speech recognition module, and a word segmentation result set Token is obtained, wherein the word segmentation result can be expressed as a set {W}, and W represents the segmented words.

Step 2: Text Filtering

A required stop word lexicon is established according to the dialogue system. Usually, stop words include but are not limited to auxiliary words, modal particles, conjunctions, and the like. Use the stop word thesaurus to filter the text information of the word segmentation result set Token obtained in step 1, and obtain the result Token ^* after text cleaning.

Step 3: Text Named Entity Recognition

Named entity recognition includes but is not limited to the following three methods, and multiple methods can be mixed to obtain a named entity recognition result as {E:T}, where E represents the entity name and T represents the entity type.

1) Based on rule matching,

Design corresponding regular expressions according to the requirements of the dialogue system, extract named entities such as phone numbers, email addresses, ID numbers and other types based on the regular expressions, and match the fields that meet the requirements.

2) Based on entity dictionary

A corresponding named entity dictionary is constructed according to the dialogue system, and the word segmentation result obtained in step 1 is matched based on the named entity dictionary. The matching methods include but are not limited to string multimodal matching, word segmentation matching, and so on.

3) Model based

The original corpus Sentence is obtained by collecting the historical corpus or corpus generation of the dialogue system, and manually or automatically labeling each position in the Sentence to complete the sequence labeling task. Usually sequence annotation can be in BIO annotation mode or BIOES annotation mode. Taking the BIOES annotation mode as an example, B is Begin, representing the beginning of an entity, I is Intermediate, representing the middle of an entity, O is Other, representing a non-entity irrelevant character, E is End, representing the end of an entity, S is Single, representing The entity consists of single characters. After the annotation is completed, the annotation sentence Sentence ^* is obtained, which is composed of BT, IT, O, ET, and ST, and then the model-based named entity recognition is realized by training the named entity recognition model. Specifically, Figure 3 shows the labeling result of a corpus data sequence of a meal ordering system. Named entity recognition can generally use HMM, CRF and other models. Preferably, the bidirectional long short-term memory (BiLSTM) + conditional random field (CRF) model is used in the patent of the present invention to achieve named entity recognition, which can achieve better results.

Step 4: Text Named Entity Replacement

The named entity types involved in the dialogue system are mapped one-to-one with specific characters, denoted as {T:C}, where T represents the entity type and C represents a specific character. The selected specific characters need to ensure that they do not exist in the corpus of the dialogue system, including but not limited to English characters, Roman numerals, Greek letters, etc.

Replace the entity name T in the named entity recognition result {E:T} obtained in step 3 with a specific character C to obtain the result set {E:C} after the named entity replacement, and substitute it into the word segmentation result Token obtained in step 2 ^* , after replacing the word W contained in the entity name E with a specific character C, recombine to obtain a new corpus Sentence'.

For example, the corpus contains 3 sentences which are S1, S2, S3 respectively. After word segmentation of text information, S1=abc ₁ d, S2=abc ₂ d, S3=abc ₃ d, where a, b, c ₁ , c ₂ , c ₃ , d represent different words in the corpus tokenization result Token, and c ₁ , c ₂ , and c ₃ represent different entity names under the same named entity type. After replacing c ₁ , c ₂ , and c ₃ with a specific character c ₀ , three corpora after the named entity replacement are obtained are S1', S2', S3', where S1'=abc _o d, S2'=abc _o d, S3′=abc _o d, thereby reducing the diversity of corpus in the intention recognition model and reducing the imbalance of text information. Specifically, Figure 2 shows an example of corpus data named entity replacement for a weather query system.

Step 5: Text Feature Extraction

Based on pre-training models such as BERT, GPT, XLNet, and XLM, and using the corpus Sentence' obtained in step 4, fine-tune the above-mentioned pre-training model to obtain a fine-tuned feature extraction model. The word vector Vec of the dialogue system corpus is obtained by using the fine-tuned feature extraction model.

Step 6: Text Intent Recognition

In the present invention, the network structure of bidirectional long short-term memory (Bi-LSTM) + attention mechanism (Attention) is used to realize text intention recognition. The network structure is mainly composed of 4 parts, as shown in Figure 4, which are as follows:

1) Input layer: take the word vector Vec of the dialogue system corpus obtained in step 5 as the input V;

2) Bidirectional LSTM layer: using a bidirectional long short-term memory network to perform forward calculation on the word vector of the input layer to obtain a vector _VL , and backward calculation to obtain a vector VR _. Splicing the front and rear vectors to obtain the spliced LSTM layer output vector V _C , where V _C =[ _VL , _VR ];

3) Attention layer: Perform Attention weighting on the output vector _VC of the LSTM layer, and further obtain the output result VA. _The calculation method is as follows:

V _m =tanh(V _c )

α=softmax(w ^T V _m )

V _A =V _c α ^T

where w is the weight matrix of the Attention layer.

4) Output layer: the output result VA of the Attention layer is used to predict the sentence intent using the _Softmax classifier, and the intent prediction result is obtained

where W _s and b _s are the weight matrix and bias value of the output layer, respectively.

This method uses the named entity recognition result to replace the entity name in the text information with the entity type, which reduces the magnitude and imbalance of the corpus data of the dialogue system, thereby comprehensively improving the accuracy of intention recognition in the dialogue process.

The above descriptions are only preferred embodiments of the present invention and are not intended to limit the present invention in any form. Although the present invention has been disclosed above with preferred embodiments, it is not intended to limit the present invention. Technical personnel, within the scope of the technical solution of the present invention, can make some changes or modifications to equivalent embodiments of equivalent changes by using the technical content disclosed above, but any content that does not depart from the technical solution of the present invention, according to the present invention Any simple modifications, equivalent changes and modifications made to the above embodiments still fall within the scope of the technical solutions of the present invention.

Claims (10)

1. a dialogue intention recognition method based on entity replacement, is characterized in that, comprises the following steps: Step 1. Text segmentation: Use the word segmentation tool to segment the text information obtained by the speech recognition module, and obtain the token result set of word segmentation; Step 2. Text filtering: Establish the required stop word thesaurus according to the dialogue system, use the stop word thesaurus to filter the text information of the word segmentation result set Token obtained in step 1, and obtain the result Token ^* after the text cleaning; Step 3. Text Named Entity Recognition: Perform named entity recognition on the text cleaning results obtained in step 2 through a deep learning model; Step 4. Text named entity replacement: Use specific characters to map the named entity types involved in the dialogue system one-to-one, denoted as {T:C}, and recombine to obtain a new corpus, where T represents the entity type, and C represents a specific character; Make sure that it does not exist in the corpus of the dialogue system; Step 5. Text feature extraction: Based on different types of pre-training models, using the new corpus obtained in step 4, fine-tune the above-mentioned pre-training model to obtain a fine-tuned feature extraction model; use the fine-tuned feature extraction model to obtain the word vector Vec of the dialogue system corpus; Step 6. Text Intent Recognition: The network structure of bi-directional long short-term memory Bi-LSTM + attention mechanism Attention is used to realize text intent recognition. 2. the method for recognizing dialog intention based on entity replacement according to claim 1, is characterized in that: the word segmentation result of described step one set Token is represented as set {W}, and W represents the word of segmentation; 3. The method for recognizing dialog intent based on entity replacement according to claim 1, characterized in that: in the step 3, after entity recognition is performed by naming, a named entity recognition result is obtained as {E:T}, wherein E represents Entity name, T for entity type. 4. The method for recognizing dialog intent based on entity replacement according to claim 1, characterized in that: step 3, the specific work steps of text named entity recognition, and the specific work is as follows: 1) Based on rule matching, Design corresponding regular expressions according to the requirements of the dialogue system, extract named entities based on regular expressions, and match fields that meet the requirements; 2) Based on entity dictionary Build a corresponding named entity dictionary according to the dialogue system, and match the word segmentation results obtained in step 1 based on the named entity dictionary; 3) Model based The original corpus Sentence is obtained by collecting the historical corpus or corpus generation of the dialogue system, and manually or automatically labeling each position in the Sentence to complete the sequence labeling task; after the labeling is completed, the labeling sentence Sentence ^* is obtained, and then the named entity recognition model is trained by training. Implements model-based named entity recognition. 5 . The method for recognizing dialog intent based on entity replacement according to claim 4 , wherein the labeled sentence Sentence ^* is composed of BT, IT, O, ET, and ST. 6 . 6 . The method for recognizing dialog intent based on entity replacement according to claim 4 , wherein: in the model-based, sequence annotation can be in a BIO annotation mode or a BIOES annotation mode. 7 . 7. the method for recognizing dialog intention based on entity replacement according to claim 6, is characterized in that: in BIOES labeling pattern, B is Begin, represents the beginning of entity, I is Intermediate, represents the middle of entity, and O is Other, represents Non-entity irrelevant characters, E is End, representing the end of the entity, S is Single, representing that the entity consists of a single character. 8. The method for recognizing dialog intention based on entity replacement according to claim 1, characterized in that: the step 4, the concrete working steps of Chinese text named entity replacement: the named entity recognition result {E:T obtained in step 3 } The entity name T in } is replaced with a specific character C, and the result set {E:C} after the replacement of the named entity is obtained, which is substituted into the word segmentation result Token ^* obtained in step 2, and the word W contained in the entity name E is replaced by a specific character. After the character C is replaced, a new corpus Sentence' is recombined. 9. The method for recognizing dialog intent based on entity replacement according to claim 1, wherein the network structure in the text intent recognizing in the step 6 is mainly composed of 4 parts, which are specifically: 1) Input layer: take the word vector Vec of the dialogue system corpus obtained in step 5 as the input V; 2) Bidirectional LSTM layer: use the bidirectional long short-term memory network to perform forward calculation on the word vector of the input layer to obtain a vector _VL , and backward calculation to obtain a vector _VR ; splicing the front and rear vectors to obtain the spliced LSTM layer output vector _VC , where V _C =[ _VL , _VR ]; 3) Attention layer: Perform Attention weighting on the output vector _VC of the LSTM layer, and further obtain the output result VA. _The calculation method is as follows: V _m =tanh(V _c ) α=softmax(w ^T V _m ) V _A =V _c α ^T where w is the weight matrix of the Attention layer. 4) Output layer: the output result VA of the Attention layer is used to predict the sentence intent using the _Softmax classifier, and the intent prediction result is obtained

where W _S and b _S are the weight matrix and bias value of the output layer, respectively. 10 . The method for recognizing dialog intent based on entity replacement according to claim 1 , wherein in the step 3, a bi-directional long short-term memory BiLSTM + conditional random field CRF model is used to realize named entity recognition, which can achieve better results. 11 .

Images