CN111462752B - Attention mechanism, feature embedding and BI-LSTM (business-to-business) based customer intention recognition method - Google Patents
Attention mechanism, feature embedding and BI-LSTM (business-to-business) based customer intention recognition method Download PDFInfo
- Publication number
- CN111462752B CN111462752B CN202010251934.8A CN202010251934A CN111462752B CN 111462752 B CN111462752 B CN 111462752B CN 202010251934 A CN202010251934 A CN 202010251934A CN 111462752 B CN111462752 B CN 111462752B
- Authority
- CN
- China
- Prior art keywords
- intention
- lstm
- attention mechanism
- user intention
- preset
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000007246 mechanism Effects 0.000 title claims abstract description 44
- 238000000034 method Methods 0.000 title claims abstract description 40
- 238000004140 cleaning Methods 0.000 claims abstract description 11
- 238000012216 screening Methods 0.000 claims abstract description 11
- 238000013528 artificial neural network Methods 0.000 claims description 18
- 238000012549 training Methods 0.000 claims description 18
- 239000013598 vector Substances 0.000 claims description 15
- 230000004927 fusion Effects 0.000 claims description 8
- 238000013136 deep learning model Methods 0.000 claims description 4
- 238000002372 labelling Methods 0.000 claims description 4
- 238000010606 normalization Methods 0.000 claims description 4
- 238000004458 analytical method Methods 0.000 abstract description 9
- 230000008569 process Effects 0.000 description 7
- 238000010586 diagram Methods 0.000 description 6
- 238000013527 convolutional neural network Methods 0.000 description 5
- 238000012545 processing Methods 0.000 description 5
- 238000012360 testing method Methods 0.000 description 4
- 238000006243 chemical reaction Methods 0.000 description 3
- 238000012217 deletion Methods 0.000 description 3
- 230000037430 deletion Effects 0.000 description 3
- 238000012795 verification Methods 0.000 description 3
- 230000002457 bidirectional effect Effects 0.000 description 2
- 125000004122 cyclic group Chemical group 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 239000013604 expression vector Substances 0.000 description 2
- 238000001914 filtration Methods 0.000 description 2
- 238000005065 mining Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000009193 crawling Effects 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000003058 natural language processing Methods 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 230000000306 recurrent effect Effects 0.000 description 1
- 230000006403 short-term memory Effects 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/049—Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/06—Creation of reference templates; Training of speech recognition systems, e.g. adaptation to the characteristics of the speaker's voice
- G10L15/063—Training
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/08—Speech classification or search
- G10L15/10—Speech classification or search using distance or distortion measures between unknown speech and reference templates
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/08—Speech classification or search
- G10L15/16—Speech classification or search using artificial neural networks
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/08—Speech classification or search
- G10L15/18—Speech classification or search using natural language modelling
- G10L15/1822—Parsing for meaning understanding
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/06—Creation of reference templates; Training of speech recognition systems, e.g. adaptation to the characteristics of the speaker's voice
- G10L15/063—Training
- G10L2015/0631—Creating reference templates; Clustering
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Health & Medical Sciences (AREA)
- Theoretical Computer Science (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Human Computer Interaction (AREA)
- Acoustics & Sound (AREA)
- Multimedia (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biophysics (AREA)
- Data Mining & Analysis (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Biomedical Technology (AREA)
- Machine Translation (AREA)
Abstract
The invention discloses a customer intention recognition method based on an attention mechanism, feature embedding and BI-LSTM, which comprises the following steps: acquiring customer service call content, and cleaning and denoising the customer service call content; screening out preset intention work order data by using a preset keyword library; and judging the user intention corresponding to the preset intention worksheet data by using a pre-established and trained user intention recognition model based on the BI-LSTM and the attention mechanism. Through the technical scheme of the invention, the semantic and intention analysis performance is improved, the accurate understanding of text meanings is realized, and the recognition capability of user intention is improved.
Description
Technical Field
The invention relates to the technical field of voice analysis, in particular to a customer intention recognition method based on an attention mechanism, feature embedding and BI-LSTM.
Background
In the existing algorithm for semantic recognition, the rule-based intent recognition model is complex to construct and has poor generalization capability. Specifically, the rule-based intent classification solution is to classify sentence intent by heuristic rules of predefined category information. The method generally constructs a rule-based classifier according to predefined rules. The keywords of the question are obtained in the classifier by utilizing the rules defined in advance, and the intention of the natural language question is understood according to the keywords, so that the purpose of classification is achieved. However, in order to obtain a better recognition effect in the use process, a large number of rules are often required to be defined, and all the rules are required to be manually marked for obtaining, so that when the corpus is large in number, a large amount of manpower is required to be consumed. Secondly, the ability to construct most rules without generalization, rules constructed by analyzing the corpus of one domain can only be used on data sets similar to that domain, and the use of the rules on other domains or other data sets will be poor. Therefore, it is difficult to construct a general rule framework with generalization.
The intent recognition model based on deep neural networks focuses on semantic relationships. Deep learning methods have been widely used in various sub-fields of natural language processing and text mining. The core of the deep learning method is a deep neural network. Currently, deep neural networks mainly comprise three types, namely convolutional neural networks, cyclic neural networks and mixed networks of the convolutional neural networks and the cyclic neural networks. The networks perform feature learning on texts such as sentences or paragraphs through the learning or unsupervised reconstruction mode of the supervised signals, and a representation vector capable of effectively representing the text semantics is generated. But whether CNN (Convolutional Neural Networks, convolutional neural network) or RNN (Recurrent Neural Network ), the input is typically a word vector or a word vector, so its representation vector is focused on semantics. The rule-based method is also an effective method in text mining, and the rule-based thought is mostly based on logic of combination or conditional combination.
The deep neural network-based intent recognition model shows good performance in capturing the whole semantics to realize more reasonable intent recognition. But when the content is too long, text information that depends on for a long time cannot be captured. That is, it is actually difficult to capture the dependency relationship between text words whose two moments are widely separated. This makes it impossible to handle some tasks that have a strong dependency on text context. Therefore, the semantics of the whole sentence needs to be fused in, but the conventional intent recognition model is difficult to meet the semantic fusion and processing requirements of the long sentence pattern and the complex sentence pattern whether input or output.
Existing intent recognition tasks all assume that a given text segment does not have too much extraneous or even disturbing information and therefore are typically analyzed directly on the given text segment. But with the original call text information, it is highly likely that useless or disturbing information is brought about. The reason is the multi-thematic nature of the user's expression and the randomness of the social text. Useless or noisy information is most likely to significantly degrade the performance of subsequent intent and semantic analysis. So that algorithms such as user intent trend analysis actually work, it is necessary to study how to accurately locate the target text segment.
Disclosure of Invention
Aiming at least one of the problems, the invention provides a customer intention recognition method based on an attention mechanism, feature embedding and BI-LSTM (BI-directional Long Short-Term Memory network), which is used for carrying out text conversion after cleaning and denoising customer service voice calls, filtering irrelevant interference information in voice call contents, improving the performance of semantic and intention analysis, screening preset intention work order data related to the user intention to be recognized by using a preset keyword library, judging the corresponding user intention by using a pre-established and trained user intention recognition model based on the BI-LSTM and the attention mechanism, improving the attention capability of the model on key feature information and the semantic fusion and processing capability on complex sentence patterns, realizing the accurate understanding of text meanings, and improving the recognition capability of the user intention.
To achieve the above object, the present invention provides a method for identifying customer intention based on attention mechanism, feature embedding and BI-LSTM, comprising: acquiring customer service call content, and cleaning and denoising the customer service call content; screening out preset intention work order data by using a preset keyword library; and judging the user intention corresponding to the preset intention worksheet data by using a pre-established and trained user intention recognition model based on the BI-LSTM and the attention mechanism.
In the above technical solution, preferably, the method for establishing and training the user intention recognition model based on BI-LSTM and attention mechanism includes: determining keywords and related paraphrasing words according to the user intention to be identified, and constructing a preset keyword library; screening keywords of the customer service call content by using the preset keyword library; manually labeling user intention labels by taking the work order data of the screened keywords as a training set; constructing a user intention recognition model by a network structure of BI-LSTM combined with an attention mechanism; inputting the work order data in the training set into the user intention recognition model, and training the user intention recognition model through the manually marked user intention labels.
In the above technical solution, preferably, the method for determining the user intention corresponding to the preset intention worksheet data by using the user intention recognition model includes: judging a user intention label implicit in each piece of preset intention work order data through the user intention recognition model; and determining the user intention corresponding to each preset intention worksheet data according to the type and the number of the user intention labels in each preset intention worksheet data through a voting principle and the original state of the user.
In the above technical solution, preferably, a paraphrasing model is used to determine the paraphrasing related to the keyword, the paraphrasing model is based on a deep learning model based on word2vec, and the semantic similarity, the pinyin similarity and the editing distance are fused to provide a semantic paraphrasing recognition result exceeding the keyword face, so that the keyword and the recognized paraphrasing are constructed together into the preset keyword library.
In the above technical solution, preferably, in the user intention recognition model, word vector information and feature information of the work order data are used as inputs of a BI-LSTM neural network, hidden state outputs of the BI-LSTM neural network are spliced with the feature information and then used as inputs of an attention mechanism, weight of each hidden state is calculated through normalization, and feature expression vectors of the work order data are obtained through weighted summation.
In the foregoing technical solution, preferably, the attention mechanism is an Encoder model, and the weighted summation is used for calculating the hidden state output vector set of the BI-LSTM neural network according to the feature information vector.
In the above technical solution, preferably, cleaning the customer service call content specifically includes file merging, duplication removal and empty data deletion.
In the foregoing technical solution, preferably, the method for identifying customer intention based on attention mechanism, feature embedding and BI-LSTM further includes: and carrying out label fusion on the user intention corresponding to the preset intention worksheet data and the worksheet information corresponding to the preset intention worksheet data.
Compared with the prior art, the invention has the beneficial effects that: through carrying out text conversion after cleaning and denoising the customer service voice call, irrelevant interference information in voice call content is filtered, semantic and intention analysis performance is improved, preset intention work order data related to user intention to be identified is screened out by utilizing a preset keyword library, corresponding user intention is judged by utilizing a pre-established and trained user intention recognition model based on BI-LSTM and an attention mechanism, attention capability of the model on key feature information and semantic fusion and processing capability on complex sentence patterns are improved, accurate understanding of text meaning is realized, and recognition capability of the user intention is improved.
Drawings
FIG. 1 is a flow chart of a method for identifying customer intent based on attention mechanism, feature embedding and BI-LSTM according to one embodiment of the present invention;
FIG. 2 is a schematic diagram of a process flow of a customer intent recognition model for number portability disclosed in one embodiment of the present invention;
FIG. 3 is a schematic diagram of a process flow of a paraphrasing model disclosed in one embodiment of the invention;
FIG. 4 is a schematic diagram of a training process of a user intent recognition model according to an embodiment of the present invention;
FIG. 5 is a schematic diagram illustrating a process for integrating prediction results of a user intent recognition model according to an embodiment of the present invention;
FIG. 6 is a schematic diagram of a network structure of a user intent recognition model according to an embodiment of the present invention;
fig. 7 is a schematic diagram of an algorithm of an attention mechanism disclosed in an embodiment of the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments of the present invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The invention is described in further detail below with reference to the attached drawing figures:
as shown in fig. 1, the method for identifying customer intention based on attention mechanism, feature embedding and BI-LSTM according to the present invention includes: acquiring customer service call content, and cleaning and denoising the customer service call content; screening out preset intention work order data by using a preset keyword library; and judging the user intention corresponding to the preset intention worksheet data by using a pre-established and trained user intention recognition model based on the BI-LSTM and the attention mechanism.
In the embodiment, text conversion is performed after noise is removed in customer service voice call cleaning, the cleaning specifically comprises file merging, duplicate removal and empty data deletion, irrelevant interference information in voice call content is filtered, semantic and intention analysis performance is improved, preset intention work order data related to user intention to be recognized is screened out by utilizing a preset keyword library, corresponding user intention is judged by utilizing a pre-established and trained user intention recognition model based on BI-LSTM and an attention mechanism, attention capability of the model to key feature information and semantic fusion and processing capability to complex sentence patterns are improved, accurate understanding of text meaning is achieved, and recognition capability of the user intention is improved.
Specifically, in the following embodiments, taking the company of the Unicom as an example to determine the customer number-carrying and network-transferring intention according to the customer service call content, the above-mentioned customer intention recognition method based on the attention mechanism, feature embedding and BI-LSTM will be described in detail.
As shown in fig. 2, according to the above-mentioned customer intention recognition method based on the attention mechanism, feature embedding and BI-LSTM, a company can recognize the intention of a user through a user intention recognition model according to the historical call data between the user and customer service, and interface with the market department to provide relatively scientific data support for decision making, promote the accurate marketing of the number-carrying and network-transferring customers, manage and control by means of big data means, and reduce the scale of the number-carrying and network-transferring customers.
Specifically, the original data of the voice-to-text of the customer service call content and the complaint work order are a plurality of small files, the small files are combined, the data is de-duplicated according to the unique identification (the unique identification of the customer service voice-to-text is a contact number, the complaint work order is a work order serial number), and the work order data in which the unique identification is not in a uniform format, the telephone number is null and characters except numbers exist is deleted.
As shown in fig. 3, in the above embodiment, it is preferable to determine the keyword related paraphrasing by using a paraphrasing model, where the paraphrasing model is based on a word2 vec-based deep learning model, and the semantic similarity, pinyin similarity and edit distance are fused to provide a semantic paraphrasing recognition result exceeding the keyword face, so as to construct a preset keyword library together with the keyword and the recognized paraphrasing.
Specifically, first setting seed keywords, namely 'carrying number and turning net', crawling content and work order data on the Internet, obtaining 'carrying number and turning net' similar meaning words, such as 'oblique number turning net', 'carrying number turning net', 'turning business hall', 'turning net without changing number', 'turning operator' and the like, by utilizing a similar meaning word model, and screening the work order data of consultation carrying number turning business by utilizing a preset keyword library. The near-meaning word model is based on a deep learning model based on word2vec, multi-dimensional information such as semantic similarity, pinyin similarity, editing distance and the like is fused, a semantic near-meaning word recognition result exceeding the literal is provided, and a deep near-meaning word model is built.
As shown in FIG. 4, in the above embodiment, the method for building and training the user intention recognition model based on the BI-LSTM and the attention mechanism preferably comprises: determining keywords and related paraphrasing words according to the user intention to be identified, and constructing a preset keyword library; screening keywords of customer service call contents by using a preset keyword library; manually labeling user intention labels by taking the work order data of the screened keywords as a training set; constructing a user intention recognition model by a network structure of BI-LSTM combined with an attention mechanism; inputting the work order data in the training set into a user intention recognition model, and training the user intention recognition model through the manually marked user intention labels.
Specifically, keywords related to the number-carrying and network-transferring to be identified are obtained based on a near-meaning word model, and a preset keyword library is constructed, so that the keyword information in customer service call content is extracted, and the aim of filtering irrelevant information in the call content is fulfilled. And extracting a plurality of call contents related to the number-carrying network-transferring service from each customer service work order, judging the user intention implicit in the key information in each call content according to the user intention recognition model, and manually marking the important information extracted according to the key words, wherein the marking result is used as training data of the user intention recognition model. 10000 labeling data are obtained, a training set, a verification set and a test set are divided according to a ratio of 7:2:1, a user intention recognition model adopts a network structure of BI-LSTM+attribute, and training is carried out by using the training set.
As shown in fig. 5, on the basis that training of the user intention recognition model is completed in the above embodiment, preferably, the method for judging the user intention corresponding to the preset intention worksheet data by using the trained user intention recognition model includes: judging a user intention label implicit in each piece of preset intention work order data through a user intention recognition model; according to the types and the quantity of the user intention labels in each piece of preset intention worksheet data, determining the user intention corresponding to the piece of preset intention worksheet data through a voting principle and the original state of the user.
Specifically, for a plurality of pieces of key information extracted from the preset intention worksheet data, the user intention labels (in-coming, out-coming and other) implicit in each piece of key information are judged through a user intention recognition model, and finally the final in-coming and out-coming intention of the user is judged through a voting principle, namely a minority obeys a majority and combining network access information of the user corresponding to the preset intention worksheet data. When all the labels are other, the final label of the work order data is other, when the quantity of the converted and converted labels is unequal, the quantity of the converted label and the converted label is equal according to the voting principle, the minority is obeyed to the majority, the different network user is the converted according to the network information of the user, and the local network user is the converted.
In the above embodiment, as shown in fig. 6, preferably, in the user intention recognition model, word vector information and feature information of the work order data are used as input of the BI-LSTM neural network, hidden state output of the BI-LSTM neural network is spliced with the feature information and then used as input of an attention mechanism, weight of each hidden state is calculated through normalization, and feature expression vector of the work order data is obtained through weighted summation.
Specifically, the BI-LSTM neural network is of a bidirectional structure, dependency relations among the semantics are captured from two directions, hidden state output of the bidirectional LSTM network is fused, characteristic information in input is spliced again to serve as input of an attention mechanism to perform weight calculation, important information and important characteristics in the semantics are helped to be learned by a model, and finally, characteristic information representation of the text is obtained through weighted summation of the hidden state information.
In the above embodiment, the attention mechanism is preferably a weighted sum for boosting an RNN-based Encoder model for computing a set of hidden state output vectors of a BI-LSTM neural network from feature information vectors. Attention mechanisms in machine translation, speech recognition applications, assign different weights to each word in a sentence, making the use of neural network models more flexible (soft) and accurate.
As shown in fig. 7, the algorithmic process of the attention mechanism is:
1) The encode encodes the input sequence to obtain a state c of the last time step and an output h of each time step, wherein c is used as an initial state z0 of the decode;
2) The outputs h and z for each time step 0 Performing matching, i.e. match operation, to obtain a matching vector for each time step
3) Outputs h and z for all time steps 0 Is a degree of matching alpha of (a) 0 Normalization with softmax to obtain the time steps for z 0 Matching scores of (2);
4) And (3) carrying out weighted summation on the output h of each time step and the matching score to obtain c0.
In the above embodiment, preferably, the method for identifying customer intention based on the attention mechanism, feature embedding and BI-LSTM further comprises: and carrying out label fusion on the user intention corresponding to the preset intention worksheet data and the worksheet information corresponding to the preset intention worksheet data.
According to the customer intention recognition method based on the attention mechanism, the feature embedding and the BI-LSTM provided by the embodiment, through data analysis of the number-carrying and network-transferring business of the Unicom company, the work order of the consultation number-carrying and network-transferring business is selected in the Beijing area, intention judgment is carried out, verification results after important information extracted from the work order data is marked are shown in a table 1, and verification results of final turning-in and turning-out intention judgment of a user are shown in a table 2.
TABLE 1 extraction of critical portion test results
| Number of test sets | Correct number of | Accuracy rate of | |
| Transfer-in | 268 | 231 | 86.19％ |
| Roll out | 452 | 433 | 95.80％ |
| Others | 281 | 246 | 87.54％ |
| Totals to | 1001 | 910 | 91％ |
TABLE 2 post-integration test results
It can be known that the core ideas of the rule-based method are fused into the semantic analysis deep neural network, the target text segment is precisely positioned, irrelevant interference information in the call content is filtered, and the performance of semantic and intention analysis is improved. By introducing feature information embedding, relevant information such as parts of speech, feature words, user levels and the like is added into the user intention recognition model, and simultaneously, a attention mechanism is introduced, so that the attention capability of the user intention recognition model to key feature information and the semantic fusion and processing capability of complex sentence patterns are improved, and the recognition capability of the user intention recognition model to user intention is improved.
In the implementation process of the customer intention recognition method based on the attention mechanism, feature embedding and BI-LSTM provided by the embodiment, the method specifically comprises the following steps:
1. data cleaning: carrying out file merging, duplicate removal, empty data deletion and the like;
2. data screening: expanding business keywords through a near-meaning word model, constructing a preset keyword library, and screening work order data related to the business according to the preset keyword library;
3. intent recognition model prediction: extracting key information of a work order to be predicted according to a business keyword lexicon constructed by the near-meaning word model, and carrying out intention prediction by adopting a trained model;
4. integrating the model prediction result: judging the implicit user intention in each piece of information by using a user intention recognition model for a plurality of pieces of key information extracted from one work order, and finally judging the final intention of the user by using a voting principle, namely a minority obeys a majority and combining the current state information of the user.
The above is only a preferred embodiment of the present invention, and is not intended to limit the present invention, but various modifications and variations can be made to the present invention by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (6)
1. A method for identifying customer intent based on attention mechanism, feature embedding and BI-LSTM, comprising:
acquiring customer service call content, and cleaning and denoising the customer service call content;
screening out preset intention work order data by using a preset keyword library;
judging the user intention corresponding to the preset intention worksheet data by using a pre-established and trained user intention recognition model based on a BI-LSTM and an attention mechanism, wherein the method specifically comprises the following steps of:
judging a user intention label implicit in each piece of preset intention work order data through the user intention recognition model;
determining the user intention corresponding to each preset intention worksheet data according to the type and the number of the user intention labels in each preset intention worksheet data through a voting principle and the original state of the user;
in the user intention recognition model, word vector information and characteristic information of the work order data are used as input of a BI-LSTM neural network, hidden state output of the BI-LSTM neural network is spliced with the characteristic information and then used as input of an attention mechanism, weight of each hidden state is calculated through normalization, and a characteristic representation vector of the work order data is obtained through weighted summation.
2. The attention mechanism, feature embedding and BI-LSTM based customer intent recognition method as claimed in claim 1, wherein the BI-LSTM and attention mechanism based user intent recognition model building and training method includes:
determining keywords and related paraphrasing words according to the user intention to be identified, and constructing a preset keyword library;
screening keywords of the customer service call content by using the preset keyword library;
manually labeling user intention labels by taking the work order data of the screened keywords as a training set;
constructing a user intention recognition model by a network structure of BI-LSTM combined with an attention mechanism;
inputting the work order data in the training set into the user intention recognition model, and training the user intention recognition model through the manually marked user intention labels.
3. The method for recognizing the client intention based on the attention mechanism, the feature embedding and the BI-LSTM according to claim 2, wherein a paraphrasing model is adopted to determine the paraphrasing related to the keyword, the paraphrasing model is based on a deep learning model based on word2vec, semantic similarity, pinyin similarity and editing distance are fused, a semantic paraphrasing recognition result exceeding the keyword face is provided, and therefore the keyword and the recognized paraphrasing are constructed together into the preset keyword library.
4. The attention mechanism, feature embedding and BI-LSTM based customer intent recognition method of claim 1, wherein the attention mechanism is an Encoder model for calculating a weighted sum of a set of hidden state output vectors of the BI-LSTM neural network from the feature information vectors.
5. The method for identifying customer intention based on attention mechanism, feature embedding and BI-LSTM according to claim 1, wherein cleaning said customer service session content comprises merging files, deduplicating and deleting null data.
6. The attention mechanism, feature embedding and BI-LSTM based customer intent recognition method as recited in claim 1, further comprising:
and carrying out label fusion on the user intention corresponding to the preset intention worksheet data and the worksheet information corresponding to the preset intention worksheet data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010251934.8A CN111462752B (en) | 2020-04-01 | 2020-04-01 | Attention mechanism, feature embedding and BI-LSTM (business-to-business) based customer intention recognition method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010251934.8A CN111462752B (en) | 2020-04-01 | 2020-04-01 | Attention mechanism, feature embedding and BI-LSTM (business-to-business) based customer intention recognition method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111462752A CN111462752A (en) | 2020-07-28 |
| CN111462752B true CN111462752B (en) | 2023-10-13 |
Family
ID=71681173
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010251934.8A Active CN111462752B (en) | 2020-04-01 | 2020-04-01 | Attention mechanism, feature embedding and BI-LSTM (business-to-business) based customer intention recognition method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111462752B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112507696B (en) * | 2021-02-04 | 2021-04-20 | 湖南大学 | Human-computer interaction diagnosis guiding method and system based on global attention intention recognition |
| CN112949317B (en) * | 2021-02-26 | 2022-10-11 | 平安普惠企业管理有限公司 | Text semantic recognition method and device, computer equipment and storage medium |
| CN113190656B (en) * | 2021-05-11 | 2023-07-14 | 南京大学 | Chinese named entity extraction method based on multi-annotation frame and fusion features |
| CN115114407B (en) * | 2022-07-12 | 2024-04-19 | 平安科技（深圳）有限公司 | Intention recognition method, device, computer equipment and storage medium |
Citations (20)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108681562A (en) * | 2018-04-26 | 2018-10-19 | 第四范式（北京）技术有限公司 | Category classification method and system and Classification Neural training method and device |
| CN108805088A (en) * | 2018-06-14 | 2018-11-13 | 南京云思创智信息科技有限公司 | Physiological signal analyzing subsystem based on multi-modal Emotion identification system |
| CN108805087A (en) * | 2018-06-14 | 2018-11-13 | 南京云思创智信息科技有限公司 | Semantic temporal fusion association based on multi-modal Emotion identification system judges subsystem |
| CN108829667A (en) * | 2018-05-28 | 2018-11-16 | 南京柯基数据科技有限公司 | It is a kind of based on memory network more wheels dialogue under intension recognizing method |
| CN108874782A (en) * | 2018-06-29 | 2018-11-23 | 北京寻领科技有限公司 | A kind of more wheel dialogue management methods of level attention LSTM and knowledge mapping |
| CN109101537A (en) * | 2018-06-27 | 2018-12-28 | 北京慧闻科技发展有限公司 | More wheel dialogue data classification methods, device and electronic equipment based on deep learning |
| CN109101493A (en) * | 2018-08-01 | 2018-12-28 | 东北大学 | A kind of intelligence house-purchase assistant based on dialogue robot |
| CN109241255A (en) * | 2018-08-20 | 2019-01-18 | 华中师范大学 | A kind of intension recognizing method based on deep learning |
| CN109741751A (en) * | 2018-12-11 | 2019-05-10 | 上海交通大学 | Intension recognizing method and device towards intelligent sound control |
| CN110096570A (en) * | 2019-04-09 | 2019-08-06 | 苏宁易购集团股份有限公司 | A kind of intension recognizing method and device applied to intelligent customer service robot |
| CN110110059A (en) * | 2019-05-20 | 2019-08-09 | 挂号网（杭州）科技有限公司 | A kind of medical conversational system intention assessment classification method based on deep learning |
| CN110162610A (en) * | 2019-04-16 | 2019-08-23 | 平安科技（深圳）有限公司 | Intelligent robot answer method, device, computer equipment and storage medium |
| CN110209791A (en) * | 2019-06-12 | 2019-09-06 | 百融云创科技股份有限公司 | It is a kind of to take turns dialogue intelligent speech interactive system and device more |
| CN110287283A (en) * | 2019-05-22 | 2019-09-27 | 中国平安财产保险股份有限公司 | Intent model training method, intension recognizing method, device, equipment and medium |
| CN110532355A (en) * | 2019-08-27 | 2019-12-03 | 华侨大学 | A kind of intention based on multi-task learning combines recognition methods with slot position |
| CN110570853A (en) * | 2019-08-12 | 2019-12-13 | 阿里巴巴集团控股有限公司 | Intention recognition method and device based on voice data |
| CN110827831A (en) * | 2019-11-15 | 2020-02-21 | 广州洪荒智能科技有限公司 | Voice information processing method, device, equipment and medium based on man-machine interaction |
| CN110866099A (en) * | 2019-10-30 | 2020-03-06 | 南昌众荟智盈信息技术有限公司 | Intelligent steward service method and system based on intelligent sound box voice interaction |
| CN110909543A (en) * | 2019-11-15 | 2020-03-24 | 广州洪荒智能科技有限公司 | Intention recognition method, device, equipment and medium |
| CN110928997A (en) * | 2019-12-04 | 2020-03-27 | 北京文思海辉金信软件有限公司 | Intention recognition method and device, electronic equipment and readable storage medium |
-
2020
- 2020-04-01 CN CN202010251934.8A patent/CN111462752B/en active Active
Patent Citations (20)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108681562A (en) * | 2018-04-26 | 2018-10-19 | 第四范式（北京）技术有限公司 | Category classification method and system and Classification Neural training method and device |
| CN108829667A (en) * | 2018-05-28 | 2018-11-16 | 南京柯基数据科技有限公司 | It is a kind of based on memory network more wheels dialogue under intension recognizing method |
| CN108805088A (en) * | 2018-06-14 | 2018-11-13 | 南京云思创智信息科技有限公司 | Physiological signal analyzing subsystem based on multi-modal Emotion identification system |
| CN108805087A (en) * | 2018-06-14 | 2018-11-13 | 南京云思创智信息科技有限公司 | Semantic temporal fusion association based on multi-modal Emotion identification system judges subsystem |
| CN109101537A (en) * | 2018-06-27 | 2018-12-28 | 北京慧闻科技发展有限公司 | More wheel dialogue data classification methods, device and electronic equipment based on deep learning |
| CN108874782A (en) * | 2018-06-29 | 2018-11-23 | 北京寻领科技有限公司 | A kind of more wheel dialogue management methods of level attention LSTM and knowledge mapping |
| CN109101493A (en) * | 2018-08-01 | 2018-12-28 | 东北大学 | A kind of intelligence house-purchase assistant based on dialogue robot |
| CN109241255A (en) * | 2018-08-20 | 2019-01-18 | 华中师范大学 | A kind of intension recognizing method based on deep learning |
| CN109741751A (en) * | 2018-12-11 | 2019-05-10 | 上海交通大学 | Intension recognizing method and device towards intelligent sound control |
| CN110096570A (en) * | 2019-04-09 | 2019-08-06 | 苏宁易购集团股份有限公司 | A kind of intension recognizing method and device applied to intelligent customer service robot |
| CN110162610A (en) * | 2019-04-16 | 2019-08-23 | 平安科技（深圳）有限公司 | Intelligent robot answer method, device, computer equipment and storage medium |
| CN110110059A (en) * | 2019-05-20 | 2019-08-09 | 挂号网（杭州）科技有限公司 | A kind of medical conversational system intention assessment classification method based on deep learning |
| CN110287283A (en) * | 2019-05-22 | 2019-09-27 | 中国平安财产保险股份有限公司 | Intent model training method, intension recognizing method, device, equipment and medium |
| CN110209791A (en) * | 2019-06-12 | 2019-09-06 | 百融云创科技股份有限公司 | It is a kind of to take turns dialogue intelligent speech interactive system and device more |
| CN110570853A (en) * | 2019-08-12 | 2019-12-13 | 阿里巴巴集团控股有限公司 | Intention recognition method and device based on voice data |
| CN110532355A (en) * | 2019-08-27 | 2019-12-03 | 华侨大学 | A kind of intention based on multi-task learning combines recognition methods with slot position |
| CN110866099A (en) * | 2019-10-30 | 2020-03-06 | 南昌众荟智盈信息技术有限公司 | Intelligent steward service method and system based on intelligent sound box voice interaction |
| CN110827831A (en) * | 2019-11-15 | 2020-02-21 | 广州洪荒智能科技有限公司 | Voice information processing method, device, equipment and medium based on man-machine interaction |
| CN110909543A (en) * | 2019-11-15 | 2020-03-24 | 广州洪荒智能科技有限公司 | Intention recognition method, device, equipment and medium |
| CN110928997A (en) * | 2019-12-04 | 2020-03-27 | 北京文思海辉金信软件有限公司 | Intention recognition method and device, electronic equipment and readable storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111462752A (en) | 2020-07-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110096570B (en) | Intention identification method and device applied to intelligent customer service robot | |
| CN111462752B (en) | Attention mechanism, feature embedding and BI-LSTM (business-to-business) based customer intention recognition method | |
| CN111209401A (en) | System and method for classifying and processing sentiment polarity of online public opinion text information | |
| CN109886270B (en) | Case element identification method for electronic file record text | |
| CN110414004B (en) | Method and system for extracting core information | |
| CN112231472B (en) | Judicial public opinion sensitive information identification method integrated with domain term dictionary | |
| CN112818093B (en) | Evidence document retrieval method, system and storage medium based on semantic matching | |
| CN113254574A (en) | Method, device and system for auxiliary generation of customs official documents | |
| Alotaibi et al. | Optical character recognition for quranic image similarity matching | |
| CN113033183B (en) | Network new word discovery method and system based on statistics and similarity | |
| CN113204967B (en) | Resume named entity identification method and system | |
| CN111753082A (en) | Text classification method and device based on comment data, equipment and medium | |
| CN112800184B (en) | Short text comment emotion analysis method based on Target-Aspect-Opinion joint extraction | |
| CN113761377B (en) | False information detection method and device based on attention mechanism multi-feature fusion, electronic equipment and storage medium | |
| CN112287090A (en) | Financial question asking back method and system based on knowledge graph | |
| CN115759119B (en) | Financial text emotion analysis method, system, medium and equipment | |
| CN112069312A (en) | Text classification method based on entity recognition and electronic device | |
| CN116070632A (en) | Informal text entity tag identification method and device | |
| CN113239690A (en) | Chinese text intention identification method based on integration of Bert and fully-connected neural network | |
| CN113486178A (en) | Text recognition model training method, text recognition device and medium | |
| CN116341519A (en) | Event causal relation extraction method, device and storage medium based on background knowledge | |
| CN115408488A (en) | Segmentation method and system for novel scene text | |
| CN115146062A (en) | Intelligent event analysis method and system fusing expert recommendation and text clustering | |
| CN114239579A (en) | Electric power searchable document extraction method and device based on regular expression and CRF model | |
| TW202034207A (en) | Dialogue system using intention detection ensemble learning and method thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |