WO2021143018A1 - 意图识别方法、装置、设备及计算机可读存储介质 - Google Patents

意图识别方法、装置、设备及计算机可读存储介质 Download PDF

Info

Publication number
WO2021143018A1
WO2021143018A1 PCT/CN2020/093227 CN2020093227W WO2021143018A1 WO 2021143018 A1 WO2021143018 A1 WO 2021143018A1 CN 2020093227 W CN2020093227 W CN 2020093227W WO 2021143018 A1 WO2021143018 A1 WO 2021143018A1
Authority
WO
WIPO (PCT)
Prior art keywords
text
recognized
neural network
attention
network model
Prior art date
Application number
PCT/CN2020/093227
Other languages
English (en)
French (fr)
Inventor
曾博
许开河
王少军
Original Assignee
平安科技（深圳）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 平安科技（深圳）有限公司 filed Critical 平安科技（深圳）有限公司
Publication of WO2021143018A1 publication Critical patent/WO2021143018A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Definitions

  • This application relates to the field of artificial intelligence technology, and in particular to an intention recognition method, device, equipment, and computer-readable storage medium.
  • Natural Language Processing is a computer that accepts user input in the form of natural language, and internally performs a series of operations such as processing and calculation through algorithms defined by humans to simulate human understanding of natural language and return to the user The process of the desired result.
  • intent recognition is a common task in NLP.
  • the common intent recognition scenario is that the terminal compares and analyzes two (or more) texts to determine whether the two texts express the same meaning; in the intent recognition process, it is necessary to Align sentence elements and analyze the similarity of text intentions based on the alignment.
  • the traditional sentence element alignment is to segment the sentence through the word segmentation tool, and then match the segment after the segmentation with the segment after the segmentation of another sentence, and determine whether the two segments can correspond according to the matching result.
  • the main purpose of this application is to provide an intention recognition method, device, equipment, and computer-readable storage medium, aiming to solve the technical problem of poor text alignment accuracy, which further affects the intention recognition result.
  • an embodiment of the present application provides an intent identification method, and the intent identification method includes:
  • an embodiment of the present application further provides an intention recognition device, the intention recognition device including:
  • the text labeling module is configured to perform word vector feature extraction on unlabeled text through the first language model to obtain unlabeled features, and label the unlabeled text according to the unlabeled features to obtain labeled training text;
  • a model construction module for constructing an attention neural network model based on the second language model and the labeled training text
  • the feature extraction module is used to obtain the text to be recognized, and perform feature extraction on the text to be recognized through the attention neural network model to obtain a candidate feature set;
  • the intention recognition module is configured to calculate the similarity of the text to be recognized according to the candidate feature set, and determine whether the text to be recognized corresponds to the same expression intention according to the similarity, and obtain an intention recognition result.
  • embodiments of the present application also provide an intention recognition device, which includes a memory, a processor, and computer-readable instructions that are stored on the memory and can run on the processor.
  • an intention recognition device which includes a memory, a processor, and computer-readable instructions that are stored on the memory and can run on the processor.
  • embodiments of the present application also provide a computer-readable storage medium, the computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions can be executed by at least one processor , So that the at least one processor executes the following steps:
  • the embodiment of the present application performs feature extraction by means of neural networks with stronger feature extraction capabilities, so that text characteristics can be considered more comprehensively, errors caused by human experience can be avoided, the accuracy of intention recognition is improved, and it is also beneficial Improve the efficiency of intent recognition.
  • FIG. 1 is a schematic diagram of the hardware structure of the intention recognition device involved in the solution of the embodiment of the application;
  • FIG. 2 is a schematic flowchart of the first embodiment of the intention identification method of this application.
  • FIG. 3 is a schematic flowchart of a second embodiment of the intention identification method of this application.
  • FIG. 4 is a schematic diagram showing the recognition result involved in the third embodiment of the intention recognition method of this application.
  • FIG. 5 is a schematic diagram of functional modules of the first embodiment of the intention identification device of this application.
  • the intent identification method involved in the embodiments of the present application is mainly applied to an intent identification device, and the intent identification device may be a device with a data processing function, such as a server, a personal computer (PC), or a notebook computer.
  • a data processing function such as a server, a personal computer (PC), or a notebook computer.
  • FIG. 1 is a schematic diagram of the hardware structure of the intention recognition device involved in the solution of the embodiment of this application.
  • the intention recognition device may include a processor 1001 (for example, a central processing unit, a CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005.
  • the communication bus 1002 is used to realize the connection and communication between these components;
  • the user interface 1003 may include a display (Display), an input unit such as a keyboard (Keyboard);
  • the network interface 1004 may optionally include a standard wired interface, a wireless interface (Such as wireless fidelity WIreless-FIdelity, WI-FI interface);
  • the memory 1005 can be a high-speed random access memory (random access memory, RAM), or a stable memory (non-volatile memory), such as a disk memory, a memory
  • 1005 may also be a storage device independent of the foregoing processor 1001.
  • the hardware structure shown in FIG. 1 does not constitute a limitation to the present application, and may include more or less components than those shown in the figure, or a combination of certain components, or different component arrangements.
  • the memory 1005 as a computer-readable storage medium in FIG. 1 may include an operating system, a network communication module, and computer-readable instructions.
  • the network communication module can be used to connect to a preset database and perform data communication with the database; and the processor 1001 can call computer-readable instructions stored in the memory 1005 and execute the intention recognition method provided by the embodiment of the present application.
  • the embodiment of the present application provides an intention recognition method.
  • FIG. 2 is a schematic flowchart of the first embodiment of the intention identification method of this application.
  • the intention recognition method includes the following steps:
  • Step S10 Perform word vector feature extraction on the unmarked text through the first language model to obtain unmarked features, and label the unmarked text according to the unmarked features to obtain the marked training text;
  • Intent recognition is a common task in natural language processing (Natural Language Processing, abbreviated as NLP).
  • NLP Natural Language Processing
  • a common intent recognition scenario is that the terminal compares and analyzes two (or more) texts to determine whether the two texts express the same Meaning:
  • sentence elements need to be aligned, and the similarity of text intents should be analyzed according to the alignment.
  • the traditional sentence element alignment is to segment the sentence through the word segmentation tool, and then match the segmented segment with the segment after the segmentation of another sentence, and determine whether the two segments can correspond according to the matching result; these Traditional methods rely heavily on word segmentation tools, and these word segmentation tools are often constructed through artificial feature engineering.
  • this embodiment proposes an intention recognition method, which first obtains annotated training text through model annotation, and uses it to construct an attention neural network model, and then uses an attention neural network model to characterize the text to be recognized (aligned) Extract and align elements of the text to be recognized according to the extracted features, analyze the matching between the text to be recognized, and determine whether the text to be recognized corresponds to the same expression intention; compared with the existing method, this embodiment adopts a neural network method Feature extraction has stronger feature extraction capabilities, so that text characteristics can be considered more comprehensively, errors caused by human experience can be avoided, the accuracy of intention recognition is improved, and the efficiency of intention recognition is also improved.
  • the intent recognition method in this embodiment is implemented by an intent recognition device.
  • the intent recognition device may be a server, a personal computer, a notebook computer, or other devices.
  • a server is used as an example for description.
  • Traditional training text is obtained by manually labeling a large number of unlabeled texts.
  • the labeling training text will be obtained by language model labeling.
  • the language model is to predict what the next word will be based on the context, and can learn a wealth of semantic knowledge from an unrestricted large-scale monolingual corpus.
  • the first language model used in this embodiment for text annotation may be implemented based on the Bert language model (Bidirectional Encoder Representations from Transformers); the Bert language model includes the Transformer encoder. Due to the self-attention mechanism, the model The upper and lower layers are directly connected to each other. It can be considered that all the layers of the model are bidirectional.
  • the input of the model includes token embedding, segmentation embedding, and position embedding.
  • Bert When Bert is pre-trained, it includes two Masked LM and Next Sentence Prediction task, and the samples used for pre-training can be unlabeled corpus, such as corpus text crawled from the Internet.
  • the first language model can also be constructed in other ways.
  • the server may obtain the unmarked text to be annotated, and then perform feature extraction on the unmarked text through the first language model.
  • the unlabeled text can be converted into the corresponding original text vector in the form of word bag or mapping, and the original text vector can be processed through the convolutional layer and the pooling layer of the first language model to obtain the text feature vector.
  • the text feature vector can be regarded as the unmarked feature of the unmarked text; of course, the specific form of the text feature vector (such as the vector dimension, the numerical range of each dimension) can be defined according to the actual situation. It is worth noting that in this embodiment, when the text feature vector is extracted through the first language model (Bert language model), it is based on the word vector.
  • the storage space of the word vector is much smaller than the word vector, so the feature can be improved.
  • the extraction efficiency and storage efficiency are also conducive to saving storage space.
  • the fine-grained character vector is higher than that of the word vector, the accuracy of subsequent text labeling (classification) and intent recognition can be improved.
  • the specific labeling process is to calculate the spatial distance between the unlabeled feature and the sample feature (known corpus), and then use the sample feature with the smallest spatial distance as the target sample feature, and then label the target sample feature corresponding to the target sample feature. Mark the text for labeling.
  • the language model is used for feature extraction processing, and the language model is a type of network model to a certain extent. Due to the powerful data analysis capabilities of the network model, the amount of feature information extracted usually requires It is higher than the feature information extracted by artificial feature engineering, and can ensure the quality and accuracy of text annotation to a certain extent.
  • Step S20 construct an attention neural network model based on the second language model and the labeled training text
  • the server When the server obtains the labeled training text, it can construct an attention neural network model for element alignment through the labeled training text and the second language model.
  • an unsupervised language model and task fine-tuning (Finetune) transfer learning method are used to ensure that better results can be obtained even with limited data sets.
  • the unsupervised language model used to construct the attention neural network model can be called the second language model, and the second language model is also The Bert language model can be used, and the task fine-tuning (Finetune) is based on the existing parameters of the Bert language model (second language model), and transfer learning (training) to it by labeling the training text, so as to perform some parameters Fine-tuning to obtain a model that meets actual usage requirements; model construction through task fine-tuning is beneficial to ensure the accuracy of model processing results while reducing the cost of model construction, and at the same time improving the efficiency of model construction.
  • the Bert language model can be used, and the task fine-tuning (Finetune) is based on the existing parameters of the Bert language model (second language model), and transfer learning (training) to it by labeling the training text, so as to perform some parameters Fine-tuning to obtain a model that meets actual usage requirements; model construction through task fine-tuning is beneficial to ensure the accuracy of model processing results while reducing the cost of model construction, and at the same time improving
  • the upper and lower layers of the second language model are all directly connected to each other. All layers of the two-language model are bidirectional, so when learning the labeled training text, you can learn the word and word combination information in the sentence, and the word and word combination information.
  • the cross-attention processing between two sentences can be added on the basis of self-attention, and the interactive features of the two sentences can be obtained through the cross-attention cross-attention, and then used to carry out the relationship between the two sentences.
  • the accuracy of the sentence relationship judgment is characterized by sentence classification loss (sentence), and the classification loss can be represented by a general cross entropy loss function cross entropy loss.
  • the cross-attention processing process can be:
  • sentence A [h1,h2,...,hm] and sentence B:[H1,H2,...,Hn], in which sentence A is represented as Embedding(A) and sentence B is represented as Embedding(B);
  • the total loss of the attention neural network model in this embodiment can be set in the form of multi-task loss, for example, including main alignment loss, sentence classification loss, and attention regularization is added to guide the output of sparse attention Attention value operation.
  • the loss function of the attention neural network model is:
  • loss(t) loss(attention)+ ⁇ *loss(sentence)+ ⁇ *L1_norm(attention_p) In the above formula, loss(t) is the total loss of the first language model
  • the alignment loss (attention) is the alignment loss, which is mainly used to characterize the alignment accuracy of sentence elements.
  • the alignment loss can use the mean square error MSE rule, that is
  • attention is the prediction result of the attention neural network model on the labeled training samples
  • attention (true) is the labeled result of the labeled training samples
  • sentence is the sentence classification loss (that is, the above-mentioned cross entropy loss);
  • attention_p is the predicted value of attention in the attention neural network model
  • ⁇ and ⁇ are preset parameters, and both are greater than zero and less than 1. Among them, ⁇ can be considered as a hyperparameter between the attention regularization and the main loss function.
  • the initial second language model is also used to extract features of several labeled training texts, and then determine whether these standard training texts belong to the same category based on the extracted features, and then calculate the corresponding total loss of the model according to the judgment; If the total loss is greater than a certain threshold, you can adjust the model parameters and continue training until the total loss is lower than or equal to the threshold, and the training is considered complete, and the attention neural network model is obtained.
  • Step S30 Obtain the text to be recognized, and perform feature extraction on the text to be recognized through the attention neural network model to obtain a candidate feature set;
  • the server when the attention neural network model is obtained, processing such as text feature extraction and intention recognition is performed through the attention neural network model.
  • the server first obtains the to-be-recognized text that needs to be recognized; the to-be-aligned text may be input by the user or obtained from a database, and the to-be-recognized text may be two or more sentences.
  • the server can perform feature extraction on the to-be-recognized text through the attention neural network model to obtain a number of candidate features, which form a candidate feature set.
  • the text to be recognized includes sentence A and sentence B, sentence A corresponds to candidate feature set X, sentence B corresponds to candidate feature set Set Y.
  • the attention neural network model is constructed based on Bert, the feature extraction can be based on word vectors, which is more granular than word vectors, which can improve the accuracy of subsequent intention recognition.
  • Step S40 Calculate the similarity of the text to be recognized according to the candidate feature set, and determine whether the text to be recognized corresponds to the same expression intention according to the similarity, and obtain an intention recognition result.
  • the text to be recognized when the candidate feature set is obtained, can be aligned according to the candidate feature set.
  • the alignment can be characterized by text similarity, that is, the closer the two text (sentence) elements are, the closer the two text (sentence) elements are. The better the alignment is, the higher the text similarity between the two is.
  • text similarity it can be characterized by the spatial distance based on the characteristics of the two (that is, various distance formulas).
  • the similarity of the text to be recognized is obtained, the similarity can be compared with a preset threshold. If the similarity is less than the preset threshold, it can be considered that the two elements are similar, and the two correspond to the same expression intention; otherwise, The two correspond to different expression intentions.
  • the text to be recognized includes sentence A and sentence B, feature extraction of sentence A through the attention neural network model to obtain candidate feature set X, feature extraction of sentence B through attention neural network model to obtain candidate feature set Y, and then calculate The spatial distance between the candidate feature set X and the candidate feature set Y.
  • the spatial distance can be considered to represent the similarity between sentence A and sentence B.
  • the similarity spatial distance
  • the similarity is less than a certain threshold, it can be considered that sentence A and The two elements of sentence B are relatively similar, and the two correspond to the same expression intention; when the similarity (spatial distance) is greater than or equal to the threshold, it is considered that the two correspond to different expression intentions.
  • the word vector feature extraction is performed on the unmarked text through the first language model to obtain the unmarked feature, and the unmarked text is annotated according to the unmarked feature to obtain the marked training text; based on the second language
  • the model and the labeled training text construct an attention neural network model; obtain the text to be recognized, and perform feature extraction on the text to be recognized through the attention neural network model to obtain a candidate feature set; calculate according to the candidate feature set According to the similarity of the text to be recognized, it is determined whether the text to be recognized corresponds to the same expression intention according to the similarity, and the intention recognition result is obtained.
  • this embodiment first obtains the labeled training text through language model labeling, and uses it to build the attention neural network model; and the process of obtaining the labeled training text is based on the word vector, which is beneficial to improve feature extraction and The efficiency of feature storage is also conducive to saving storage space.
  • the fine-grained word vector is higher than that of the word vector, it can improve the accuracy of subsequent text labeling (classification) and intent recognition; then through the attention nerve
  • the method of the network model performs feature extraction on the text to be recognized (aligned), and aligns the elements of the text to be recognized based on the extracted features, analyzes the matching between the text to be recognized, and judges whether the text to be recognized corresponds to the same expression intent;
  • this embodiment has stronger feature extraction capabilities for feature extraction through neural networks, so that text characteristics can be considered more comprehensively, errors caused by human experience can be avoided, and the accuracy of intention recognition is improved.
  • FIG. 3 is a schematic flowchart of a second embodiment of the intention identification method of this application.
  • the method further includes:
  • Step S50 voting on the labeled training text based on the compound decision rule to determine whether the labeled training text is valid;
  • the model labeling result in order to improve the quality of text labeling, after labeling through the first language model, the model labeling result can also be judged by related rules to determine whether the standard training text is valid; The model labeling result is used as the text labeling result of the unlabeled text, and subsequent model training is performed; if it is invalid, the unlabeled text needs to be manually relabeled manually. It is worth mentioning that in this embodiment, when making a judgment, the model labeling results are judged separately through multiple rule compound decision-making, and then the decision is made through voting.
  • a composite decision rule is used to vote on the labeled training text obtained in step S10 to determine whether the labeled training text is valid; wherein, the composite decision rule can select two or more types according to actual conditions (here " The above "includes this number, the same below) rules.
  • the compound decision rule in this embodiment may include the maximum entropy rule, the minimum confidence rule, the Bayesian uncertainty rule based on sampling, and the normalized log. Logarithmic value (MNLP) rules; for different rules, they can be used separately to evaluate the effectiveness of annotated training text.
  • MNLP Logarithmic value
  • the evaluation distribution is characterized by a score, that is, according to different rules, the effectiveness of annotated training text can be obtained separately
  • a score that is, according to different rules, the effectiveness of annotated training text can be obtained separately
  • P is the comprehensive score of the voting decision
  • p1 is the recognition score of the labeled training text according to the maximum entropy rule
  • p2 is the recognized score of the labeled training text according to the minimum confidence rule
  • p3 is the Bayesian based on the sampling Approval scores for annotated training text based on the uncertainty rule
  • p4 is the approved scores for annotated training text according to MNLP rules
  • w1, w2, w3, w4 are parameters greater than zero
  • w1+w2+w3+w4 1.
  • the comprehensive score can be compared with the preset score threshold. If the comprehensive score is higher than the score threshold, the model labeling result can be considered effective, that is, the training text is effective; if the comprehensive score is low If it is less than or equal to the score threshold, it is considered invalid to label the training text. It is worth noting that, in addition to the above-mentioned weighting method for synthesizing the approved scores of each rule, other methods can also be used to integrate multiple rules, such as calculating an average score.
  • the step S20 includes:
  • Step S21 If the labeled training text is valid, construct an attention neural network model based on the second language model and the labeled training text.
  • an attention neural network model can be constructed based on the second language model and the annotated training text. The specific construction process will not be repeated here.
  • the standard training text needs to be corrected accordingly to avoid the inaccuracy of the model labeling from causing subsequent construction of the attention neural network model Adverse effects; at this time, the corresponding manual labeling prompt will be output to prompt relevant personnel to review the model labeling results.
  • the manual labeling prompt can be voice, text, etc.; relevant personnel can mark the training text input according to the manual labeling prompt Corresponding manual annotation; when the server receives the input manual annotation, it can correct the standard training text according to the manual annotation; after the correction, it can construct the attention nerve based on the second language model and the corrected annotation training text Network model.
  • the first language model can be transferred and trained according to the corrected labeled training text, thereby updating the first language model, thereby improving the applicability of the first language model ,
  • the updated first language model can be used for other text annotation tasks in the future.
  • a multi-criteria judgment method can be used to determine whether the labeled training text is valid, and the model can be constructed only when it is valid. If it is invalid, the labeled training text can be manually labeled.
  • the training text is corrected to ensure the labeling quality of the training text used to construct the attention neural network model, and to avoid the inaccuracy of the model labeling from adversely affecting the subsequent construction of the attention neural network model.
  • the method further includes:
  • the feature text corresponding to each candidate feature in the text to be recognized will also be determined.
  • the text to be recognized includes sentence A.
  • Sentence A can be expressed as "h1, h2, ..., hm", where h1, h2, ..., hm are individual characters, and the candidate features of sentence A include x1, x2, x3 , X4, where the feature text corresponding to x1 and x2 is h1, the feature text corresponding to x3 is h2 and hm, and the feature text corresponding to x4 is hm.
  • one candidate feature may correspond to multiple feature texts.
  • step S40 it further includes:
  • the intent recognition result may be displayed together with the relevant candidate features, so that the user can learn the processing process of the intent recognition and realize that the result can be interpreted.
  • the to-be-recognized text and the intent-recognition result can be displayed, and the candidate features and feature text can be displayed according to preset display rules; for example, the text to be recognized includes sentence A and sentence B, and sentence A can be expressed as "h1, h2,..., hm", sentence B can be expressed as "H1, H2,..., Hn", where h1, h2,..., hm and H1, H2,..., Hn are individual characters; the candidate features of sentence A include x1, x2, x3, x4, where the feature text corresponding to x1 and x2 is h1, the feature text corresponding to x3 is h2 and hm, and the feature text corresponding to x4 is hm; the candidate features of
  • a and sentence B are displayed as two lines respectively, and the candidate features of each sentence are displayed near the corresponding feature text (shown below in Figure 4) and connected by lines, while texts with different features are marked by boxes (Ie hm and Hn), the intention recognition result is displayed at the bottom (sentence A and sentence B correspond to different expression intentions).
  • other display rules can also be set. For example, texts with different characteristics can be displayed in different colors, or the same characteristics can be displayed in different colors.
  • the intent recognition result can be displayed together with the relevant candidate features, so that the user can learn the process of intent recognition and realize that the result can be interpreted.
  • step S30 includes:
  • the intent identification method may be applied in a self-service process, for example, a self-service terminal of a certain self-service terminal, or a self-service of a certain mobile phone software.
  • a self-service process for example, a self-service terminal of a certain self-service terminal, or a self-service of a certain mobile phone software.
  • users can use the voice collection device of the terminal or mobile phone to perform voice input to obtain the corresponding service.
  • the server collects the user voice of the user, and then converts the user voice into a corresponding voice text.
  • the standard text when the voice text is obtained, the standard text can be obtained from the preset text library.
  • the standard text stored in the preset text library can be considered as a condition for triggering a certain service function.
  • the acquired voice text and the standard text correspond to the same expression intent, it can be considered that the corresponding service function is triggered.
  • the speech text and the standard text are used as the text to be recognized, and feature extraction is performed on the text to be recognized through the attention neural network model to obtain a candidate feature set.
  • the speech text and the standard text can be used as the text to be recognized (it can be considered as the sentence A and the sentence B in the first embodiment), and the attention neural network model is used to perform the recognition on the text to be recognized.
  • Feature extraction, candidate feature sets are obtained, and subsequent intent recognition processing is performed to determine whether the voice text and the standard text correspond to the same expression intent.
  • the method further includes:
  • the processing strategy corresponding to the standard text is acquired, and the intention feedback processing is performed based on the processing strategy.
  • the server can query the processing strategy corresponding to the standard text, and perform intent feedback processing based on the processing strategy.
  • the intent recognition method in this embodiment can be applied to the self-service process.
  • the user can trigger the corresponding service function by voice, and the server recognizes the user’s voice and executes the corresponding intent feedback processing based on the intent recognition result. , So as to provide users with self-service, which is conducive to the realization of service intelligence and improve the user's service experience.
  • an embodiment of the present application also provides an intention recognition device.
  • FIG. 5 is a schematic diagram of the functional modules of the first embodiment of the intention identification device of the present application.
  • the intention recognition device includes:
  • the text labeling module 10 is configured to perform word vector feature extraction on unlabeled text through the first language model to obtain unlabeled features, and label the unlabeled text according to the unlabeled features to obtain labeled training text;
  • the model construction module 20 is configured to construct an attention neural network model based on the second language model and the labeled training text;
  • the feature extraction module 30 is configured to obtain the text to be recognized, and perform feature extraction on the text to be recognized through the attention neural network model to obtain a candidate feature set;
  • the intention recognition module 40 is configured to calculate the similarity of the text to be recognized according to the candidate feature set, and determine whether the text to be recognized corresponds to the same expression intention according to the similarity, and obtain an intention recognition result.
  • each virtual function module of the above-mentioned intention recognition apparatus is stored in the memory 1005 of the recognition device shown in the schematic diagram of FIG. 1, and is used to realize all the functions of computer-readable instructions; when each module is executed by the processor 1001, the function of the intention recognition can be realized .
  • the intention recognition device further includes:
  • the voting decision module is used to make a voting decision on the labeled training text based on a compound decision rule to determine whether the labeled training text is valid;
  • the model construction module 20 is further configured to construct an attention neural network model based on the second language model and the annotated training text if the annotated training text is valid.
  • the intention recognition device further includes:
  • the prompt output module is configured to output a corresponding manual annotation prompt if the annotation training text is invalid;
  • a text correction module configured to correct the marked training text according to the manual annotation when the manual annotation input based on the manual annotation prompt is received;
  • the model construction module 20 is also used to construct an attention neural network model based on the second language model and the corrected labeled training text.
  • the loss function of the attention neural network model is:
  • loss(t) is the total loss of the first language model
  • sentence loss is the sentence classification loss
  • attention_p is the predicted value of attention in the attention neural network model
  • ⁇ and ⁇ are preset parameters, and both are greater than zero and less than 1.
  • the intention recognition device further includes:
  • a text determination module configured to determine the feature text corresponding to each candidate feature in the candidate feature set in the text to be recognized
  • the result display module is used to display the to-be-recognized text and the intent-recognition result, and display the candidate feature and the feature text according to a preset display rule.
  • model construction module 20 includes:
  • the voice collection unit is used to collect the user voice of the user and convert the user voice into corresponding voice text
  • the text acquisition unit is used to acquire standard text from the preset text library
  • the feature extraction unit is configured to use the speech text and the standard text as the text to be recognized, and perform feature extraction on the text to be recognized through the attention neural network model to obtain a candidate feature set.
  • the intention recognition device further includes:
  • the feedback processing module is configured to, if the intent recognition result is that the text to be recognized corresponds to the same expression intent, obtain the processing strategy corresponding to the standard text, and perform intent feedback processing based on the processing strategy.
  • each module in the above-mentioned intention recognition apparatus corresponds to each step in the above-mentioned embodiment of the intention recognition method, and the functions and realization processes thereof will not be repeated here.
  • the embodiments of the present application also provide a computer-readable storage medium.
  • the computer-readable storage medium may be non-volatile or volatile.
  • the computer-readable storage medium of this application stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the following steps are implemented:
  • the technical solution of this application essentially or the part that contributes to the existing technology can be embodied in the form of a software product, and the computer software product is stored in a storage medium (such as ROM/RAM) as described above. , Magnetic disks, optical disks), including several instructions to make a terminal device (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) execute the method described in each embodiment of the present application.
  • a terminal device which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Machine Translation (AREA)

Abstract

一种意图识别方法、装置、设备及计算机可读存储介质，涉及人工智能技术领域，该方法包括：通过第一语言模型对无标记文本进行字向量特征提取，得到无标记特征，并根据所述无标记特征对所述无标记文本进行标注，得到标注训练文本(S10)；基于第二语言模型和所述标注训练文本构造注意力神经网络模型(S20)；获取待识别文本，并通过所述注意力神经网络模型对所述待识别文本进行特征提取，得到候选特征集(S30)；根据所述候选特征集计算所述待识别文本的相似度，并根据所述相似度判断所述待识别文本是否对应同一表达意图，得到意图识别结果(S40)。通过神经网络的方式进行特征提取，可以更全面的考虑文本特点，提高了意图识别的准确性。

Description

意图识别方法、装置、设备及计算机可读存储介质
本申请申明2020年01月16日递交的申请号为202010049994.1、名称为“意图识别方法、装置、设备及计算机可读存储介质”的中国专利申请的优先权，该中国专利申请的整体内容以参考的方式结合在本申请中。
技术领域
本申请涉及人工智能技术领域，尤其涉及一种意图识别方法、装置、设备及计算机可读存储介质。
背景技术
自然语言处理(Natural Language Processing，简写NLP)是计算机接受用户自然语言形式的输入，并在内部通过人类所定义的算法进行加工、计算等系列操作，以模拟人类对自然语言的理解，并返回用户所期望的结果的过程。其中意图识别为NLP中常见的任务，常见的意图识别场景是由终端对两个(或多个)文本进行比对分析，判断这两个文本是否表达相同的意思；在意图识别过程中，需要进行句子要素对齐，并根据对齐情况分析文本意图的相似性。传统的句子要素对齐是通过分词工具对句子进行分词，然后再将分词后的片段与另一个句子分词之后的片段进行匹配，并根据匹配结果来确定是否这两个片段能否对应得上。发明人意识到，这些传统方法对分词工具较大的依赖性，且这些分词工具往往是通过人工特征工程的方式构建，因此传统方法容易受到已有经验的限制，导致对齐处理的准确性较差，进而影响后续意图识别的准确性。
发明内容
本申请的主要目的在于提供一种意图识别方法、装置、设备及计算机可读存储介质，旨在解决文本对齐准确性差、进而影响意图识别结果的技术问题。
为实现上述目的，本申请实施例提供一种意图识别方法，所述意图识别方法包括：
通过第一语言模型对无标记文本进行字向量特征提取，得到无标记特征，并根据所述无标记特征对所述无标记文本进行标注，得到标注训练文本；
基于第二语言模型和所述标注训练文本构造注意力神经网络模型；
获取待识别文本，并通过所述注意力神经网络模型对所述待识别文本进行特征提取，得到候选特征集；
根据所述候选特征集计算所述待识别文本的相似度，并根据所述相似度判断所述待识 别文本是否对应同一表达意图，得到意图识别结果。
此外，为实现上述目的，本申请实施例还提供一种意图识别装置，所述意图识别装置包括：
文本标注模块，用于通过第一语言模型对无标记文本进行字向量特征提取，得到无标记特征，并根据所述无标记特征对所述无标记文本进行标注，得到标注训练文本；
模型构造模块，用于基于第二语言模型和所述标注训练文本构造注意力神经网络模型；
特征提取模块，用于获取待识别文本，并通过所述注意力神经网络模型对所述待识别文本进行特征提取，得到候选特征集；
意图识别模块，用于根据所述候选特征集计算所述待识别文本的相似度，并根据所述相似度判断所述待识别文本是否对应同一表达意图，得到意图识别结果。
此外，为实现上述目的，本申请实施例还提供一种意图识别设备，所述意图识别设备包括存储器、处理器及存储在所述存储器上并可在所述处理器上运行的计算机可读指令，所述计算机可读指令被处理器执行时实现以下步骤：
通过第一语言模型对无标记文本进行字向量特征提取，得到无标记特征，并根据所述无标记特征对所述无标记文本进行标注，得到标注训练文本；
基于第二语言模型和所述标注训练文本构造注意力神经网络模型；
获取待识别文本，并通过所述注意力神经网络模型对所述待识别文本进行特征提取，得到候选特征集；
根据所述候选特征集计算所述待识别文本的相似度，并根据所述相似度判断所述待识别文本是否对应同一表达意图，得到意图识别结果。
此外，为实现上述目的，本申请实施例还提供一种计算机可读存储介质，所述计算机可读存储介质内存储有计算机可读指令，所述计算机可读指令可被至少一个处理器所执行，以使所述至少一个处理器执行如下步骤：
通过第一语言模型对无标记文本进行字向量特征提取，得到无标记特征，并根据所述无标记特征对所述无标记文本进行标注，得到标注训练文本；
基于第二语言模型和所述标注训练文本构造注意力神经网络模型；
获取待识别文本，并通过所述注意力神经网络模型对所述待识别文本进行特征提取，得到候选特征集；
根据所述候选特征集计算所述待识别文本的相似度，并根据所述相似度判断所述待识别文本是否对应同一表达意图，得到意图识别结果。
本申请实施例通过神经网络的方式进行特征提取具有更强的特征提取能力，从而可以更全面的考虑文本特点，避免了人为经验带来的误差，提高了意图识别的准确性，此外还有利于提高意图识别的效率。
附图说明
图1为本申请实施例方案中涉及的意图识别设备的硬件结构示意图；
图2为本申请意图识别方法第一实施例的流程示意图；
图3为本申请意图识别方法第二实施例的流程示意图；
图4为本申请意图识别方法第三实施例涉及的识别结果显示示意图；
图5为本申请意图识别装置第一实施例的功能模块示意图。
本申请目的的实现、功能特点及优点将结合实施例，参照附图做进一步说明。
具体实施方式
应当理解，此处所描述的具体实施例仅仅用以解释本申请，并不用于限定本申请。
本申请实施例涉及的意图识别方法主要应用于意图识别设备，该意图识别设备可以是服务器、个人计算机(personal computer，PC)、笔记本电脑等具有数据处理功能的设备。
参照图1，图1为本申请实施例方案中涉及的意图识别设备的硬件结构示意图。本申请实施例中，该意图识别设备可以包括处理器1001(例如中央处理器Central Processing Unit，CPU)，通信总线1002，用户接口1003，网络接口1004，存储器1005。其中，通信总线1002用于实现这些组件之间的连接通信；用户接口1003可以包括显示屏(Display)、输入单元比如键盘(Keyboard)；网络接口1004可选的可以包括标准的有线接口、无线接口(如无线保真WIreless-FIdelity，WI-FI接口)；存储器1005可以是高速随机存取存储器(random access memory，RAM)，也可以是稳定的存储器(non-volatile memory)，例如磁盘存储器，存储器1005可选的还可以是独立于前述处理器1001的存储装置。本领域技术人员可以理解，图1中示出的硬件结构并不构成对本申请的限定，可以包括比图示更多或更少的部件，或者组合某些部件，或者不同的部件布置。
继续参照图1，图1中作为一种计算机可读存储介质的存储器1005可以包括操作系统、 网络通信模块以及计算机可读指令。在图1中，网络通信模块可用于连接预设数据库，与数据库进行数据通信；而处理器1001可以调用存储器1005中存储的计算机可读指令，并执行本申请实施例提供的意图识别方法。
基于上述的硬件架构，提出本申请意图识别方法的各实施例。
本申请实施例提供了一种意图识别方法。
参照图2，图2为本申请意图识别方法第一实施例的流程示意图。
本实施例中，所述意图识别方法包括以下步骤：
步骤S10，通过第一语言模型对无标记文本进行字向量特征提取，得到无标记特征，并根据所述无标记特征对所述无标记文本进行标注，得到标注训练文本；
意图识别为自然语言处理(Natural Language Processing，简写NLP)中常见的任务，常见的意图识别场景是由终端对两个(或多个)文本进行比对分析，判断这两个文本是否表达相同的意思；在意图识别过程中，需要进行句子要素对齐，并根据对齐情况分析文本意图的相似性。传统的句子要素对齐是通过分词工具对句子进行分词，然后再将分词后的片段与另一个句子分词之后的片段进行匹配，并根据匹配结果来确定是否这两个片段能否对应得上；这些传统方法对分词工具较大的依赖性，且这些分词工具往往是通过人工特征工程的方式构建，因此传统方法容易受到已有经验的限制，导致对齐处理的准确性较差，进而影响后续意图识别的准确性。对此，本实施例提出一种意图识别方法，先通过模型标注的方式获得标注训练文本，并用以构建注意力神经网络模型，然后通过注意力神经网络模型的方式对待识别(对齐)文本进行特征提取，并根据提取的特征对待识别文本进行要素对齐，分析待识别文本之间的匹配性，从而判断待识别文本是否对应同一表达意图；与现有方法相比，本实施例通过神经网络的方式进行特征提取具有更强的特征提取能力，从而可以更全面的考虑文本特点，避免了人为经验带来的误差，提高了意图识别的准确性，此外还有利于提高意图识别的效率。
本实施例中的意图识别方法是由意图识别设备实现的，该意图识别设备可以是服务器、个人计算机、笔记本电脑等设备，本实施例中以服务器为例进行说明。本实施例在进行意图前，首先需要获取(构建)一个用以进行要素对齐的注意力神经网络模型；而训练该注意力神经网络模型需要使用一定的训练文本。传统的训练文本是通过人工标注的方式对大量无标记文本进行标注得到，本实施例中为了减少人力成本、提高标注效率，将采用语言模型标注的方式获得标注训练文本。其中，语言模型的就是根据上下文去预测下一个词是什么，能够从无限制的大规模单语语料中，学习到丰富的语义知识。本实施例中所采用的 用以进行文本标注的第一语言模型，可以是基于Bert语言模型(Bidirectional Encoder Representations from Transformers)实现的；Bert语言模型包括Transformer编码器，由于self-attention机制，所以模型上下层直接全部互相连接的，可认为模型的所有层中是双向的，模型的输入包括token embedding、segmentation embedding、和position embedding共同构成；而Bert在进行预训练时，包括两个Masked LM和Next Sentence Prediction任务，而其预训练所用的样本，则可以是使用无标记语料，如从网络爬取的语料文本等内容。当然，在实际中，也可以是通过其它的方式构建第一语言模型。
在得到第一语言模型时，服务器可获取将要进行标注的无标记文本，然后通过第一语言模型对无标记文本进行特征提取。例如，首先可将无标记文本以词袋或者映射的方式转换为对应的原始文本向量，并通过第一语言模型的卷积层和池化层对原始文本向量进行处理，得到文本特征向量，该文本特征向量即可认为是无标记文本的无标记特征；当然对于文本特征向量的具体形式(如向量维度、各维度的数值范围)可以根据实际情况进行定义。值得说明的是，本实施例中在通过第一语言模型(Bert语言模型)提取文本特征向量时，是基于字向量的方式进行，字向量的存储空间要远远小于词向量，因此可提高特征提取的效率和存储效率，同时还有利于节约存储空间，此外，由于字向量的细粒度较词向量而言更高，因而可提高后续文本标注(分类)及意图识别的准确性。在得到无标记文本的无标记特征时，即可根据该无标记特征对无标记文本进行标注，得到模型标注结果，该模型标注结果与无标记文本共同形成标注训练文本。具体标注过程，也即计算无标记特征与样本特征(已知语料)之间的空间距离，然后根据将空间距离最小的样本特征作为目标样本特征，再根据目标样本特征所对应的目标标注对无标记文本进行标注。值得说明的是，本实施例中是利用语言模型进行特征提取处理，而语言模型在一定程度属于网络模型的一种，由于网络模型强大的数据分析能力，因此所提取得到的特征信息量通常要高于人工特征工程所提取的特征信息，进而能够在一定程度上保证文本标注质量和准确性。
步骤S20，基于第二语言模型和所述标注训练文本构造注意力神经网络模型；
服务器在得到标注训练文本时，即可通过标注训练文本和第二语言模型构造用以进行要素对齐的注意力神经网络模型。对于该注意力神经网络模型模型的构造过程，本实施例中是采用了无监督语言模型和任务微调(Finetune)的迁移学习方式进行，从而保证在有限数据集的情况下也能获得较好的效果，有利于减小训练样本不足所带来的负面影响；其中，为了说明方便，该用以构造注意力神经网络模型的无监督语言模型可称为第二语言模型，该第二语言模型也可以是采用Bert语言模型，而任务微调(Finetune)则是在Bert语言模型(第二语言模型)已有的参数基础上，通过标注训练文本对其进行迁移学习(训练)，从 而对部分参数进行微调，得到符合实际使用需求的模型；通过任务微调的方式进行模型构造，有利于在保证模型处理结果准确性的同时、降低模型构造成本，同时还可提高模型构造的效率。
由于第二语言模型(Bert语言模型)中的自注意力self-attention机制，第二语言模型上下层(以及基于第二语言模型构造的注意力神经网络模型)直接全部互相连接的，可认为第二语言模型的所有层中是双向的，因此在对标注训练文本进行学习时，可以学习到句子中字与字组合信息，以及词与词的组合信息。此外，还可在self-attention基础上添加了两个句子之间的交叉注意力cross-attention处理，并且通过交叉注意力cross-attention来获得两个句子交互特征，进而用以进行两个句子关系判断，该句子关系判断准确程度使用句子分类损失loss(sentence)进行表征，而对于该分类损失可采用一般的交叉熵损失函数cross entropy loss表示。具体的，该cross-attention处理的过程可以为:
假设句子A:[h1,h2,…,hm]和句子B:[H1,H2,…,Hn],其中句子A的表示为Embedding(A)，句子B表示为Embedding(B)；
对于句子A和B，可计算EAB：
Figure PCTCN2020093227-appb-000001
其中，
Figure PCTCN2020093227-appb-000002
b为大于零的常数；
同理可计算得到EBA；
在得到EAB、EBA之后执行数组连接concat操作，即可得到cross-attention处理之后的结果。
进一步的，本实施例中的注意力神经网络模型的总损失可设置为多任务损失的形式，例如包括主要的对齐损失、句子分类损失，并加入了注意力attention正则化，引导输出稀疏注意力attention值操作。具体的，注意力神经网络模型的损失函数为：
loss(t)＝loss(attention)+λ*loss(sentence)+γ*L1_norm(attention_p)在上式中，loss(t)为所述第一语言模型的总损失；
loss(attention)为对齐损失，该对齐损失主要用于表征句子要素对齐准确性，该对齐损失可采用均方误差MSE规则，也即
Figure PCTCN2020093227-appb-000003
其中power为平方函数，attention(prediction)为注意力神经网络模型对标注训练样本的预测结果，attention(true)为标注训练样本的标注结果；
loss(sentence)为句子分类损失(即上述的交叉熵损失)；
attention_p为注意力神经网络模型中的注意力attention的预测值；
λ、γ为预设参数，且均大于零且小于1；其中γ可认为是调注意力值attention正则化与主损失函数之间的超参数。
在训练过程中，也通过初始的第二语言模型对若干的标注训练文本进行特征提取，然后根据提取的特征对判断这些标准训练文本是否属于同一类，然后根据判断情况计算对应的模型总损失；若总损失大于一定阈值，则可调整模型参数后继续训练，直至总损失低于或等于该阈值时即可认为训练完成，得到注意力神经网络模型。
通过上述设置多任务损失函数的方式进行模型训练，综合考虑了文本(句子)片段之间对齐的监督信息以及文本分类(标签)信息，从而使得模型能够同时学习到句子中不同片段对应关系和两个句子关系，提高了模型的处理能力，并且能够达到模型可解释的效果。
步骤S30，获取待识别文本，并通过所述注意力神经网络模型对所述待识别文本进行特征提取，得到候选特征集；
本实施例中，在得到注意力神经网络模型时，即通过注意力神经网络模型进行文本特征提取和意图识别等处理。服务器首先获取需要进行识别的待识别文本；该待对齐文本可以由用户进行输入，也可以是从数据库中获取得到，该待识别文本可以是两个或两个以上的句子。得到待对识别文本时，服务器可通过注意力神经网络模型对待识别文本进行特征提取，得到若干的候选特征，这些候选特征形成了候选特征集。值得说明的是，对于不同的句子(文本)是分别进行特征提取，得到对应的候选特征集；例如，待识别文本包括句子A和句子B，句子A对应候选特征集X，句子B对应候选特征集Y。当然，由于注意力神经网络模型是基于Bert构建，因此在进行特征提取时，可以是基于字向量的方式，较词向量而言其细粒度更高，因而可提高后续意图识别的准确性。
步骤S40，根据所述候选特征集计算所述待识别文本的相似度，并根据所述相似度判断所述待识别文本是否对应同一表达意图，得到意图识别结果。
本实施例中，在得到候选特征集时，即可根据该候选特征集对待识别文本进行对齐处理，该对齐情况可以通过文本相似度来表征，也即两个文本(句子)要素越相近，两者对齐情况越好，表现为两者的文本相似度越高；而在计算文本相似度时，可以通过基于两者特征的空间距离(即各种距离公式)进行表征。当得到待识别文本相似度时，可将该相似度与一预设阈值进行比较，若该相似度小于该预设阈值，即可认为两者要素较相近，两者对应同一表达意图；反之即两者对应不同的表达意图。例如，待识别文本包括句子A和句子B，通过注意力神经网络模型对句子A进行特征提取得到候选特征集X，通过注意力神经网络模型对句子B进行特征提取得到候选特征集Y，然后计算候选特征集X和候选特征 集Y的空间距离，该空间距离即可认为是表征句子A和句子B的相似度，当该相似度(空间距离)小于某一阈值时，即可认为句子A和句子B两者要素较相近，两者对应同一表达意图；当该相似度(空间距离)大于或等于该阈值时，即认为两者对应不同的表达意图。
本实施例中，通过第一语言模型对无标记文本进行字向量特征提取，得到无标记特征，并根据所述无标记特征对所述无标记文本进行标注，得到标注训练文本；基于第二语言模型和所述标注训练文本构造注意力神经网络模型；获取待识别文本，并通过所述注意力神经网络模型对所述待识别文本进行特征提取，得到候选特征集；根据所述候选特征集计算所述待识别文本的相似度，并根据所述相似度判断所述待识别文本是否对应同一表达意图，得到意图识别结果。通过以上方式，本实施例先通过语言模型标注的方式获得标注训练文本，并用以构建注意力神经网络模型；而在获得标注训练文本的过程是基于字向量的方式进行，有利于提高特征提取和特征存储的效率，同时还有利于节约存储空间，此外，由于字向量的细粒度较词向量而言更高，因而可提高后续文本标注(分类)及意图识别的准确性；然后通过注意力神经网络模型的方式对待识别(对齐)文本进行特征提取，并根据提取的特征对待识别文本进行要素对齐，分析待识别文本之间的匹配性，从而判断待识别文本是否对应同一表达意图；与现有方法相比，本实施例通过神经网络的方式进行特征提取具有更强的特征提取能力，从而可以更全面的考虑文本特点，避免了人为经验带来的误差，提高了意图识别的准确性，此外还有利于提高意图识别的效率。
基于上述图2所示实施例，提出本申请意图识别方法第二实施例。
参照图3，图3为本申请意图识别方法第二实施例的流程示意图。
本实施例中，所述步骤S10之后，还包括：
步骤S50，基于复合决策规则对所述标注训练文本进行投票决策，以判断所述标注训练文本是否有效；
本实施例中，为了提高文本标注的质量，在通过第一语言模型进行标注后，还可通过相关规则来对模型标注结果进行判断，以确定标准训练文本是否有效；若有效，则可将该模型标注结果作为无标记文本的文本标注结果，并进行后续的模型训练；若无效，则需要通过人工的方式对无标记文本重新进行人工标注。值得说的是，本实施例在进行判断时，是通过多种规则复合决策分别对模型标注结果进行判断，然后通过投票决策的方式确定。具体的，本实施例中是通过复合决策规则对步骤S10所得的标注训练文本进行投票决策，以判断标注训练文本是否有效；其中，该复合决策规则可根据实际情况选用两种以上(此处“以上”包括本数，下同)的规则，例如，本实施例中的复合决策规则可以包括熵entropy 最大规则、最小置信度规则、基于采样的贝叶斯不确定性bayesian uncertainty规则、归一化log对数值(MNLP)规则；对于不同规则，均可单独用以评价标注训练文本的有效性，该评价分布用一分值进行表征，也即根据不同的规则可分别得出针对标注训练文本有效性的认可分值，分值越高，则认为该规则下对标注训练文本有效的认可度越高；然后可通过加权的方式综合这些规则的分值，得到综合评分，具体可表示为：
P＝w1*p1+w2*p2+w3*p3+w4*p4
上式中，P为投票决策的综合评分，p1为根据entropy最大规则对标注训练文本的认可分值，p2为根据最小置信度规则对标注训练文本的认可分值，p3为根据采样的贝叶斯不确定性规则对标注训练文本的认可分值，p4为根据MNLP规则对标注训练文本的认可分值；w1、w2、w3、w4为大于零的参数，且w1+w2+w3+w4＝1。
在得到综合评分时，可将该综合评分与预设分值阈值比较，若该综合评分高于该分值阈值，则可认为模型标注结果有效，也即标注训练文本有效；若该综合评分低于或等于该分值阈值，则认为标注训练文本无效。值得说明的是，除了上述加权的方式对各规则的认可分进行综合外，还可以采用其它的方式对多规则进行集成，例如计算平均分等。
所述步骤S20包括：
步骤S21，若所述标注训练文本有效，则基于第二语言模型和所述标注训练文本构造注意力神经网络模型。
本实施例中，若判断标注训练文本有效，则可根据基于第二语言模型和标注训练文本构造注意力神经网络模型。具体构造过程此处不再赘述。
进一步的，若判断标注训练文本无效(综合分值低于或等于该分值阈值)，则需要对标准训练文本进行相应的校正处理，避免模型标注的不准确对后续构造注意力神经网络模型造成不利影响；此时将会输出对应的人工标注提示，以提示相关人员对模型标注结果进行检视，该人工标注提示可以是语音、文字等方式；相关人员可根据该人工标注提示就标注训练文本输入对应的人工标注；服务器在接收到输入的人工标注时，即可根据该人工标注对标准训练文本进行校正；在校正后，即可基于第二语言模型和校正后的标注训练文本构造注意力神经网络模型。
再进一步的，在对标注训练文本进行校正后时，还可以根据校正后的标注训练文本对第一语言模型进行迁移训练，从而对第一语言模型进行更新，从而提高第一语言模型的适用性，后续可用更新的第一语言模型进行其它的文本标注任务。
本实施例在得到模型标注的标注训练文本时，可采用多准则判断的方法来判断标注训练文本是否有效，并在有效时才用以进行模型构造，若无效则可通过人工标注的方式对标 注训练文本进行校正，从而保证了构造注意力神经网络模型所用训练文本的标注质量，避免模型标注的不准确对后续构造注意力神经网络模型造成不利影响。
基于上述图2所示实施例，提出本申请意图识别方法第三实施例。
本实施例中，所述步骤S30之后，还包括：
在所述待识别文本中确定所述候选特征集中各候选特征对应的特征文本；
本实施例中，在通过注意力神经网络模型进行特征提取，得到候选特征集后，还将会确定各候选特征在待识别文本中对于的特征文本。例如，待识别文本包括句子A，句子A具体可表示为“h1、h2、…、hm”，其中h1、h2、...、hm为各个字，句子A的候选特征包括x1、x2、x3、x4，其中x1和x2对应的特征文本为h1，x3对应的特征文本为h2和hm，x4对应的特征文本为hm。值得说明的是，在实际中，一个候选特征可能对应多个特征文本。
所述步骤S40之后，还包括：
显示所述待识别文本和意图识别结果，并根据预设显示规则显示所述候选特征和特征文本。
本实施例中，在得到对待识别文本的意图识别完成，得到意图识别结果后，可以是将意图识别结果与相关候选特征一同显示，从而方便用户获知意图识别的处理过程，实现结果可解释。具体的，可显示待识别文本和意图识别结果，同时根据预设显示规则显示候选特征和特征文本；例如，待识别文本包括句子A和句子B，句子A可表示为“h1、h2、…、hm”，句子B可表示为“H1、H2、…、Hn”，其中h1、h2、...、hm和H1、H2、...、Hn为各个字；句子A的候选特征包括x1、x2、x3、x4，其中x1和x2对应的特征文本为h1，x3对应的特征文本为h2和hm，x4对应的特征文本为hm；句子B的候选特征包括x1、x2、x3、x5，其中x1和x2对应的特征文本为H1，x3对应的特征文本为H2和Hm，x5对应的特征文本为Hm；在显示时，参照图4，图4为本实施例中的识别结果显示示意图，句子A和句子B分别显示为两行，而各句子的候选特征则显示在对应特征文本的附近(图4是显示在下方)，并用线连接，而对于不同特征的文本则用方框的方式标注(即hm和Hn)，意图识别结果则显示在最下方(句子A和句子B对应不同的表达意图)。当然，除了上述举例外，还可以是设置其它的显示规则，例如可以是以不同的颜色显示具有不同特征的文本，又或者是以不同的颜色显示相同的特征等。
通过以上方式，在得到对待识别文本的意图识别完成，得到意图识别结果后，可以是将意图识别结果与相关候选特征一同显示，从而方便用户获知意图识别的处理过程，实现 结果可解释。
基于上述图2所示实施例，提出本申请意图识别方法第四实施例。
本实施例中，所述步骤S30包括：
采集用户的用户语音，并将所述用户语音转换为对应的语音文本；
本实施例中，该意图识别方法可以是应用在自助服务流程中，例如某一自助服务终端的自助服务，某一手机软件的自助服务。当用户需要使用自助服务时，可以通过终端或手机的语音采集装置进行语音输入，以获取相应的服务。服务器则采集用户的用户语音，然后将该用户语音转换为对应的语音文本。
从预设文本库中获取标准文本；
本实施例中，在得到语音文本时，可从预设文本库中获取标准文本。其中，预设文本库所存储的标准文本可认为是触发某一服务功能的条件，当获取的语音文本与标准文本对应同一表达意图时，即可认为触发了对应服务功能。
所述语音文本和所述标准文本作为待识别文本，并通过所述注意力神经网络模型对所述待识别文本进行特征提取，得到候选特征集。
在得到标准文本时，可将该语音文本和标准文本作为待识别文本(可认为是上述第一实施例中的句子A和句子B)，并通过注意力神经网络模型对所述待识别文本进行特征提取，得到候选特征集，并进行后续的意图识别处理，以判断语音文本和标准文本是否对应同一表达意图。
进一步的，所述步骤S40之后还包括：
若所述意图识别结果为所述待识别文本对应同一表达意图，则获取所述标准文本对应的处理策略，并基于所述处理策略进行意图反馈处理。
本实施例中，若意图识别结果为语音文本和标准文本对应同一表达意图，则可认为当前用户正在请求标准文本所对应的服务功能。此时服务器可查询该标准文本对应的处理策略，并基于该处理策略进行意图反馈处理。
本实施例中的意图识别方法可以应用在自助服务流程中，用户可通过语音的方式触发对应的服务功能，服务器则通过对用户的语音进行意图识别，并根据意图识别结果执行对应的意图反馈处理，从而为用户提供自助服务，有利于实现服务的智能化，提高用户的服务体验。
此外，本申请实施例还提供一种意图识别装置。
参照图5，图5为本申请意图识别装置第一实施例的功能模块示意图。
本实施例中，所述意图识别装置包括：
文本标注模块10，用于通过第一语言模型对无标记文本进行字向量特征提取，得到无标记特征，并根据所述无标记特征对所述无标记文本进行标注，得到标注训练文本；
模型构造模块20，用于基于第二语言模型和所述标注训练文本构造注意力神经网络模型；
特征提取模块30，用于获取待识别文本，并通过所述注意力神经网络模型对所述待识别文本进行特征提取，得到候选特征集；
意图识别模块40，用于根据所述候选特征集计算所述待识别文本的相似度，并根据所述相似度判断所述待识别文本是否对应同一表达意图，得到意图识别结果。
其中，上述意图识别装置的各虚拟功能模块存储于图1所示意图识别设备的存储器1005中，用于实现计算机可读指令的所有功能；各模块被处理器1001执行时，可实现意图识别的功能。
进一步的，所述意图识别装置还包括：
投票决策模块，用于基于复合决策规则对所述标注训练文本进行投票决策，以判断所述标注训练文本是否有效；
所述模型构造模块20，还用于若所述标注训练文本有效，则基于第二语言模型和所述标注训练文本构造注意力神经网络模型。
进一步的，所述意图识别装置还包括：
提示输出模块，用于若所述标注训练文本无效，则输出对应的人工标注提示；
文本校正模块，用于在接收到基于所述人工标注提示输入的人工标注时，根据所述人工标注对所述标注训练文本进行校正；
所述模型构造模块20，还用于基于第二语言模型和校正后的标注训练文本构造注意力神经网络模型。
进一步的，所述注意力神经网络模型的损失函数为:
loss(t)＝loss(attention)+λ*loss(sentence)+γ*L1_norm(attention_p)
其中，loss(t)为所述第一语言模型的总损失；
loss(attention)为对齐损失；
loss(sentence)为句子分类损失；
attention_p为所述注意力神经网络模型中的注意力attention的预测值；
λ、γ为预设参数，且均大于零并小于1。
进一步的，所述意图识别装置还包括：
文本确定模块，用于在所述待识别文本中确定所述候选特征集中各候选特征对应的特征文本；
结果显示模块，用于显示所述待识别文本和意图识别结果，并根据预设显示规则显示所述候选特征和特征文本。
进一步的，所述模型构造模块20包括：
语音采集单元，用于采集用户的用户语音，并将所述用户语音转换为对应的语音文本；
文本获取单元，用于从预设文本库中获取标准文本；
特征提取单元，用于将所述语音文本和所述标准文本作为待识别文本，并通过所述注意力神经网络模型对所述待识别文本进行特征提取，得到候选特征集。
进一步的，所述意图识别装置还包括：
反馈处理模块，用于若所述意图识别结果为所述待识别文本对应同一表达意图，则获取所述标准文本对应的处理策略，并基于所述处理策略进行意图反馈处理。
其中，上述意图识别装置中各个模块的功能实现与上述意图识别方法实施例中各步骤相对应，其功能和实现过程在此处不再一一赘述。
此外，本申请实施例还提供一种计算机可读存储介质，所述计算机可读存储介质可以是非易失性，也可以是易失性。
本申请计算机可读存储介质上存储有计算机可读指令，其中所述计算机可读指令被处理器执行时，实现以下步骤：
通过第一语言模型对无标记文本进行字向量特征提取，得到无标记特征，并根据所述无标记特征对所述无标记文本进行标注，得到标注训练文本；
基于第二语言模型和所述标注训练文本构造注意力神经网络模型；
获取待识别文本，并通过所述注意力神经网络模型对所述待识别文本进行特征提取，得到候选特征集；
根据所述候选特征集计算所述待识别文本的相似度，并根据所述相似度判断所述待识别文本是否对应同一表达意图，得到意图识别结果
其中，计算机可读指令被执行时所实现的方法可参照本申请意图识别方法的各个实施例，此处不再赘述。
需要说明的是，在本文中，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他 性的包含，从而使得包括一系列要素的过程、方法、物品或者系统不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者系统所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括该要素的过程、方法、物品或者系统中还存在另外的相同要素。
上述本申请实施例序号仅仅为了描述，不代表实施例的优劣。
通过以上的实施方式的描述，本领域的技术人员可以清楚地了解到上述实施例方法可借助软件加必需的通用硬件平台的方式来实现，当然也可以通过硬件，但很多情况下前者是更佳的实施方式。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品存储在如上所述的一个存储介质(如ROM/RAM、磁碟、光盘)中，包括若干指令用以使得一台终端设备(可以是手机，计算机，服务器，空调器，或者网络设备等)执行本申请各个实施例所述的方法。
以上仅为本申请的优选实施例，并非因此限制本申请的专利范围，凡是利用本申请说明书及附图内容所作的等效结构或等效流程变换，或直接或间接运用在其他相关的技术领域，均同理包括在本申请的专利保护范围内。

Claims (20)

  1. 一种意图识别方法，所述意图识别方法包括：
    通过第一语言模型对无标记文本进行字向量特征提取，得到无标记特征，并根据所述无标记特征对所述无标记文本进行标注，得到标注训练文本；
    基于第二语言模型和所述标注训练文本构造注意力神经网络模型；
    获取待识别文本，并通过所述注意力神经网络模型对所述待识别文本进行特征提取，得到候选特征集；
    根据所述候选特征集计算所述待识别文本的相似度，并根据所述相似度判断所述待识别文本是否对应同一表达意图，得到意图识别结果。
  2. 如权利要求1所述的意图识别方法，所述通过第一语言模型对无标记文本进行字向量特征提取，得到无标记特征，并根据所述无标记特征对所述无标记文本进行标注，得到标注训练文本的步骤之后，还包括：
    基于复合决策规则对所述标注训练文本进行投票决策，以判断所述标注训练文本是否有效；
    所述基于第二语言模型和所述标注训练文本构造注意力神经网络模型的步骤包括：
    若所述标注训练文本有效，则基于第二语言模型和所述标注训练文本构造注意力神经网络模型。
  3. 如权利要求2所述的意图识别方法，所述基于复合决策规则对所述标注训练文本进行投票决策，以判断所述标注训练文本是否有效的步骤之后，还包括：
    若所述标注训练文本无效，则输出对应的人工标注提示；
    在接收到基于所述人工标注提示输入的人工标注时，根据所述人工标注对所述标注训练文本进行校正；
    所述基于第二语言模型和所述标注训练文本构造注意力神经网络模型的步骤还包括：
    基于第二语言模型和校正后的标注训练文本构造注意力神经网络模型。
  4. 如权利要求1所述的意图识别方法，所述注意力神经网络模型的损失函数为:
    loss(t)＝loss(attention)+λ*loss(sentence)+γ*L1_n orm(attention_p)
    其中，loss(t)为所述第一语言模型的总损失；
    loss(attention)为对齐损失；
    loss(sentence)为句子分类损失；
    attention_p为所述注意力神经网络模型中的注意力attention的预测值；
    λ、γ为预设参数，且均大于零并小于1。
  5. 如权利要求1所述的意图识别方法，所述获取待识别文本，并通过所述注意力神经网络模型对所述待识别文本进行特征提取，得到候选特征集的步骤之后，还包括：
    在所述待识别文本中确定所述候选特征集中各候选特征对应的特征文本；
    所述根据所述候选特征集计算所述待识别文本的相似度，并根据所述相似度判断所述待识别文本是否对应同一表达意图，得到意图识别结果的步骤之后，还包括：
    显示所述待识别文本和意图识别结果，并根据预设显示规则显示所述候选特征和特征文本。
  6. 如权利要求1所述的意图识别方法，所述获取待识别文本，并通过所述注意力神经网络模型对所述待识别文本进行特征提取，得到候选特征集的步骤包括：
    采集用户的用户语音，并将所述用户语音转换为对应的语音文本；
    从预设文本库中获取标准文本；
    将所述语音文本和所述标准文本作为待识别文本，并通过所述注意力神经网络模型对所述待识别文本进行特征提取，得到候选特征集。
  7. 如权利要求1所述的意图识别方法，所述根据所述候选特征集计算所述待识别文本的相似度，并根据所述相似度判断所述待识别文本是否对应同一表达意图，得到意图识别结果的步骤之后，还包括：
    若所述意图识别结果为所述待识别文本对应同一表达意图，则获取所述标准文本对应的处理策略，并基于所述处理策略进行意图反馈处理。
  8. 一种意图识别装置，所述意图识别装置包括：
    文本标注模块，用于通过第一语言模型对无标记文本进行字向量特征提取，得到无标记特征，并根据所述无标记特征对所述无标记文本进行标注，得到标注训练文本；
    模型构造模块，用于基于第二语言模型和所述标注训练文本构造注意力神经网络模型；
    特征提取模块，用于获取待识别文本，并通过所述注意力神经网络模型对所述待识别文本进行特征提取，得到候选特征集；
    意图识别模块，用于根据所述候选特征集计算所述待识别文本的相似度，并根据所述相似度判断所述待识别文本是否对应同一表达意图，得到意图识别结果。
  9. 一种意图识别设备，所述意图识别设备包括存储器、处理器及存储在所述存储器上并可在所述处理器上运行的计算机可读指令，所述计算机可读指令被处理器执行时实现以下步骤：
    通过第一语言模型对无标记文本进行字向量特征提取，得到无标记特征，并根据所述 无标记特征对所述无标记文本进行标注，得到标注训练文本；
    基于第二语言模型和所述标注训练文本构造注意力神经网络模型；
    获取待识别文本，并通过所述注意力神经网络模型对所述待识别文本进行特征提取，得到候选特征集；
    根据所述候选特征集计算所述待识别文本的相似度，并根据所述相似度判断所述待识别文本是否对应同一表达意图，得到意图识别结果。
  10. 如权利要求9所述的意图识别设备，所述通过第一语言模型对无标记文本进行字向量特征提取，得到无标记特征，并根据所述无标记特征对所述无标记文本进行标注，得到标注训练文本的步骤之后，还包括：
    基于复合决策规则对所述标注训练文本进行投票决策，以判断所述标注训练文本是否有效；
    所述基于第二语言模型和所述标注训练文本构造注意力神经网络模型的步骤包括：
    若所述标注训练文本有效，则基于第二语言模型和所述标注训练文本构造注意力神经网络模型。
  11. 如权利要求10所述的意图识别设备，所述基于复合决策规则对所述标注训练文本进行投票决策，以判断所述标注训练文本是否有效的步骤之后，还包括：
    若所述标注训练文本无效，则输出对应的人工标注提示；
    在接收到基于所述人工标注提示输入的人工标注时，根据所述人工标注对所述标注训练文本进行校正；
    所述基于第二语言模型和所述标注训练文本构造注意力神经网络模型的步骤还包括：
    基于第二语言模型和校正后的标注训练文本构造注意力神经网络模型。
  12. 如权利要求9所述的意图识别设备，所述注意力神经网络模型的损失函数为:
    loss(t)＝loss(attention)+λ*loss(sentence)+γ*L1_n orm(attention_p)
    其中，loss(t)为所述第一语言模型的总损失；
    loss(attention)为对齐损失；
    loss(sentence)为句子分类损失；
    attention_p为所述注意力神经网络模型中的注意力attention的预测值；
    λ、γ为预设参数，且均大于零并小于1。
  13. 如权利要求9所述的意图识别设备，所述获取待识别文本，并通过所述注意力神经网络模型对所述待识别文本进行特征提取，得到候选特征集的步骤之后，还包括：
    在所述待识别文本中确定所述候选特征集中各候选特征对应的特征文本；
    所述根据所述候选特征集计算所述待识别文本的相似度，并根据所述相似度判断所述待识别文本是否对应同一表达意图，得到意图识别结果的步骤之后，还包括：
    显示所述待识别文本和意图识别结果，并根据预设显示规则显示所述候选特征和特征文本。
  14. 如权利要求9所述的意图识别设备，所述获取待识别文本，并通过所述注意力神经网络模型对所述待识别文本进行特征提取，得到候选特征集的步骤包括：
    采集用户的用户语音，并将所述用户语音转换为对应的语音文本；
    从预设文本库中获取标准文本；
    将所述语音文本和所述标准文本作为待识别文本，并通过所述注意力神经网络模型对所述待识别文本进行特征提取，得到候选特征集。
  15. 如权利要求9所述的意图识别设备，所述根据所述候选特征集计算所述待识别文本的相似度，并根据所述相似度判断所述待识别文本是否对应同一表达意图，得到意图识别结果的步骤之后，还包括：
    若所述意图识别结果为所述待识别文本对应同一表达意图，则获取所述标准文本对应的处理策略，并基于所述处理策略进行意图反馈处理。
  16. 一种计算机可读存储介质，所述计算机可读存储介质内存储有计算机可读指令，所述计算机可读指令可被至少一个处理器所执行，以使所述至少一个处理器执行如下步骤：
    通过第一语言模型对无标记文本进行字向量特征提取，得到无标记特征，并根据所述无标记特征对所述无标记文本进行标注，得到标注训练文本；
    基于第二语言模型和所述标注训练文本构造注意力神经网络模型；
    获取待识别文本，并通过所述注意力神经网络模型对所述待识别文本进行特征提取，得到候选特征集；
    根据所述候选特征集计算所述待识别文本的相似度，并根据所述相似度判断所述待识别文本是否对应同一表达意图，得到意图识别结果。
  17. 如权利要求16所述的计算机可读存储介质，所述通过第一语言模型对无标记文本进行字向量特征提取，得到无标记特征，并根据所述无标记特征对所述无标记文本进行标注，得到标注训练文本的步骤之后，还包括：
    基于复合决策规则对所述标注训练文本进行投票决策，以判断所述标注训练文本是否有效；
    所述基于第二语言模型和所述标注训练文本构造注意力神经网络模型的步骤包括：
    若所述标注训练文本有效，则基于第二语言模型和所述标注训练文本构造注意力神经 网络模型。
  18. 如权利要求17所述的计算机可读存储介质，所述基于复合决策规则对所述标注训练文本进行投票决策，以判断所述标注训练文本是否有效的步骤之后，还包括：
    若所述标注训练文本无效，则输出对应的人工标注提示；
    在接收到基于所述人工标注提示输入的人工标注时，根据所述人工标注对所述标注训练文本进行校正；
    所述基于第二语言模型和所述标注训练文本构造注意力神经网络模型的步骤还包括：
    基于第二语言模型和校正后的标注训练文本构造注意力神经网络模型。
  19. 如权利要求16所述的计算机可读存储介质，所述注意力神经网络模型的损失函数为:
    loss(t)＝loss(attention)+λ*loss(sentence)+γ*L1_n orm(attention_p)
    其中，loss(t)为所述第一语言模型的总损失；
    loss(attention)为对齐损失；
    loss(sentence)为句子分类损失；
    attention_p为所述注意力神经网络模型中的注意力attention的预测值；
    λ、γ为预设参数，且均大于零并小于1。
  20. 如权利要求16所述的计算机可读存储介质，所述获取待识别文本，并通过所述注意力神经网络模型对所述待识别文本进行特征提取，得到候选特征集的步骤之后，还包括：
    在所述待识别文本中确定所述候选特征集中各候选特征对应的特征文本；
    所述根据所述候选特征集计算所述待识别文本的相似度，并根据所述相似度判断所述待识别文本是否对应同一表达意图，得到意图识别结果的步骤之后，还包括：
    显示所述待识别文本和意图识别结果，并根据预设显示规则显示所述候选特征和特征文本。
PCT/CN2020/093227 2020-01-16 2020-05-29 意图识别方法、装置、设备及计算机可读存储介质 WO2021143018A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010049994.1A CN111259625B (zh) 2020-01-16 2020-01-16 意图识别方法、装置、设备及计算机可读存储介质
CN202010049994.1 2020-01-16

Publications (1)

Publication Number Publication Date
WO2021143018A1 true WO2021143018A1 (zh) 2021-07-22

Family

ID=70945238

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/093227 WO2021143018A1 (zh) 2020-01-16 2020-05-29 意图识别方法、装置、设备及计算机可读存储介质

Country Status (2)

Country Link
CN (1) CN111259625B (zh)
WO (1) WO2021143018A1 (zh)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113591471A (zh) * 2021-08-20 2021-11-02 上海大参林医疗健康科技有限公司 一种基于字和词的语言特征提取装置及方法
CN113836945A (zh) * 2021-09-23 2021-12-24 平安科技（深圳）有限公司 意图识别方法、装置、电子设备和存储介质
CN114154509A (zh) * 2021-11-26 2022-03-08 深圳集智数字科技有限公司 一种意图确定方法及装置
CN117150305A (zh) * 2023-11-01 2023-12-01 杭州光云科技股份有限公司 融合检索和填空的文本数据增强方法、装置及电子设备
CN117910539A (zh) * 2024-03-19 2024-04-19 电子科技大学 一种基于异构半监督联邦学习的家庭特征识别方法
CN117910539B (zh) * 2024-03-19 2024-05-31 电子科技大学 一种基于异构半监督联邦学习的家庭特征识别方法

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111625641B (zh) * 2020-07-30 2020-12-01 浙江大学 一种基于多维度语义交互表征模型的对话意图识别方法及系统
CN112308370B (zh) * 2020-09-16 2024-03-05 湘潭大学 一种基于Transformer的面向思政课程的主观题自动评分方法
CN112364662A (zh) * 2020-11-13 2021-02-12 中国科学院软件研究所 一种基于神经网络的意图识别方法及电子装置
CN112329430B (zh) * 2021-01-04 2021-03-16 恒生电子股份有限公司 一种模型训练方法、文本相似度确定方法及装置
CN112800239B (zh) * 2021-01-22 2024-04-12 中信银行股份有限公司 意图识别模型训练方法、意图识别方法及装置
CN113011456B (zh) * 2021-02-05 2022-07-15 中国科学技术大学 用于图像分类的基于类别自适应模型的无监督域适应方法
CN112861519A (zh) * 2021-03-12 2021-05-28 云知声智能科技股份有限公司 医疗文本纠错方法、装置以及存储介质
CN113158692B (zh) * 2021-04-22 2023-09-12 中国平安财产保险股份有限公司 基于语义识别的多意图处理方法、系统、设备及存储介质
CN113672715A (zh) * 2021-08-20 2021-11-19 上海大参林医疗健康科技有限公司 一种意图识别系统及方法
CN113722492A (zh) * 2021-09-09 2021-11-30 国网电子商务有限公司 一种意图识别方法及装置

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107346340A (zh) * 2017-07-04 2017-11-14 北京奇艺世纪科技有限公司 一种用户意图识别方法及系统
WO2018212584A2 (ko) * 2017-05-16 2018-11-22 삼성전자 주식회사 딥 뉴럴 네트워크를 이용하여 문장이 속하는 클래스를 분류하는 방법 및 장치
US20180349350A1 (en) * 2017-06-01 2018-12-06 Beijing Baidu Netcom Science And Technology Co., Ltd. Artificial intelligence based method and apparatus for checking text
CN109829153A (zh) * 2019-01-04 2019-05-31 平安科技（深圳）有限公司 基于卷积神经网络的意图识别方法、装置、设备及介质
CN109857844A (zh) * 2018-12-29 2019-06-07 北京三快在线科技有限公司 基于点餐对话文本的意图识别方法、装置、电子设备

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11429405B2 (en) * 2017-11-28 2022-08-30 Intuit, Inc. Method and apparatus for providing personalized self-help experience
CN109902285B (zh) * 2019-01-08 2023-09-22 平安科技（深圳）有限公司 语料分类方法、装置、计算机设备及存储介质
CN109902157A (zh) * 2019-01-10 2019-06-18 平安科技（深圳）有限公司 一种训练样本有效性检测方法及装置
CN110083692B (zh) * 2019-04-22 2023-01-24 齐鲁工业大学 一种金融知识问答的文本交互匹配方法及装置
CN110287285B (zh) * 2019-05-31 2023-06-16 平安科技（深圳）有限公司 一种问题意图识别方法、装置、计算机设备及存储介质
CN110377911B (zh) * 2019-07-23 2023-07-21 中国工商银行股份有限公司 对话框架下的意图识别方法和装置
CN110534087B (zh) * 2019-09-04 2022-02-15 清华大学深圳研究生院 一种文本韵律层级结构预测方法、装置、设备及存储介质

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018212584A2 (ko) * 2017-05-16 2018-11-22 삼성전자 주식회사 딥 뉴럴 네트워크를 이용하여 문장이 속하는 클래스를 분류하는 방법 및 장치
US20180349350A1 (en) * 2017-06-01 2018-12-06 Beijing Baidu Netcom Science And Technology Co., Ltd. Artificial intelligence based method and apparatus for checking text
CN107346340A (zh) * 2017-07-04 2017-11-14 北京奇艺世纪科技有限公司 一种用户意图识别方法及系统
CN109857844A (zh) * 2018-12-29 2019-06-07 北京三快在线科技有限公司 基于点餐对话文本的意图识别方法、装置、电子设备
CN109829153A (zh) * 2019-01-04 2019-05-31 平安科技（深圳）有限公司 基于卷积神经网络的意图识别方法、装置、设备及介质

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113591471A (zh) * 2021-08-20 2021-11-02 上海大参林医疗健康科技有限公司 一种基于字和词的语言特征提取装置及方法
CN113836945A (zh) * 2021-09-23 2021-12-24 平安科技（深圳）有限公司 意图识别方法、装置、电子设备和存储介质
CN113836945B (zh) * 2021-09-23 2024-04-16 平安科技（深圳）有限公司 意图识别方法、装置、电子设备和存储介质
CN114154509A (zh) * 2021-11-26 2022-03-08 深圳集智数字科技有限公司 一种意图确定方法及装置
CN117150305A (zh) * 2023-11-01 2023-12-01 杭州光云科技股份有限公司 融合检索和填空的文本数据增强方法、装置及电子设备
CN117150305B (zh) * 2023-11-01 2024-02-27 杭州光云科技股份有限公司 融合检索和填空的文本数据增强方法、装置及电子设备
CN117910539A (zh) * 2024-03-19 2024-04-19 电子科技大学 一种基于异构半监督联邦学习的家庭特征识别方法
CN117910539B (zh) * 2024-03-19 2024-05-31 电子科技大学 一种基于异构半监督联邦学习的家庭特征识别方法

Also Published As

Publication number Publication date
CN111259625B (zh) 2023-06-27
CN111259625A (zh) 2020-06-09

Similar Documents

Publication Publication Date Title
WO2021143018A1 (zh) 意图识别方法、装置、设备及计算机可读存储介质
WO2020143844A1 (zh) 意图分析方法、装置、显示终端及计算机可读存储介质
WO2021253904A1 (zh) 测试案例集生成方法、装置、设备及计算机可读存储介质
CN109597994B (zh) 短文本问题语义匹配方法和系统
CN110580292B (zh) 一种文本标签生成方法、装置和计算机可读存储介质
WO2021212749A1 (zh) 命名实体标注方法、装置、计算机设备和存储介质
WO2020155766A1 (zh) 意图识别中的拒识方法、装置、设备及存储介质
CN111310438A (zh) 基于多粒度融合模型的中文句子语义智能匹配方法及装置
WO2021051598A1 (zh) 文本情感分析模型训练方法、装置、设备及可读存储介质
CN108038208B (zh) 上下文信息识别模型的训练方法、装置和存储介质
CN112270196A (zh) 实体关系的识别方法、装置及电子设备
WO2022052505A1 (zh) 基于依存句法的句子主干抽取方法、设备和可读存储介质
CN112883193A (zh) 一种文本分类模型的训练方法、装置、设备以及可读介质
CN111933127A (zh) 一种具备自学习能力的意图识别方法及意图识别系统
WO2022048194A1 (zh) 事件主体识别模型优化方法、装置、设备及可读存储介质
CN112101010B (zh) 一种基于bert的电信行业oa办公自动化文稿审核的方法
CN109036466A (zh) 面向情感语音识别的情感维度pad预测方法
CN110659392B (zh) 检索方法及装置、存储介质
CN111984780A (zh) 多意图识别模型训练方法和多意图识别方法及相关装置
CN115935959A (zh) 一种低资源黏着语序列标注的方法
CN116644183B (zh) 文本分类方法、装置及存储介质
CN112613451A (zh) 一种跨模态文本图片检索模型的建模方法
CN116361655A (zh) 模型训练方法、标准问题的预测方法、装置、设备及介质
CN116414958A (zh) 文本语料的生成方法、装置、存储介质及电子设备
CN114169447B (zh) 基于自注意力卷积双向门控循环单元网络的事件检测方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20914089

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20914089

Country of ref document: EP

Kind code of ref document: A1