CN115526171A - Intention identification method, device, equipment and computer readable storage medium - Google Patents
Intention identification method, device, equipment and computer readable storage medium Download PDFInfo
- Publication number
- CN115526171A CN115526171A CN202211190704.0A CN202211190704A CN115526171A CN 115526171 A CN115526171 A CN 115526171A CN 202211190704 A CN202211190704 A CN 202211190704A CN 115526171 A CN115526171 A CN 115526171A
- Authority
- CN
- China
- Prior art keywords
- label
- word
- mapper
- words
- predicted
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/279—Recognition of textual entities
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/205—Parsing
- G06F40/216—Parsing using statistical methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/30—Semantic analysis
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Probability & Statistics with Applications (AREA)
- Machine Translation (AREA)
Abstract
The invention discloses an intention identification method, an intention identification device, intention identification equipment and a computer readable storage medium, and relates to the technical field of NLP, wherein the method comprises the following steps: acquiring label similar words corresponding to each label by using a pre-training model, and constructing a label word mapper; acquiring small sample labeling data corresponding to each label by using a preset regular expression; according to the small sample labeling data, carrying out frequency adjustment and/or correlation adjustment on the label word mapper to obtain an adjusted label word mapper; matching a predicted word corresponding to an original sentence to be recognized and obtained by predicting a pre-training language model by utilizing an adjusting label word mapper to obtain a label corresponding to the predicted word; according to the label key words of each label, the small labeled sample data is obtained by regular extraction, and the problem of unbalanced sample distribution is solved; the label mapping words in the label word mapper are automatically configured by utilizing the pre-training model, and the accuracy of intention identification is improved through the optimization of the label word mapper.
Description
Technical Field
The present invention relates to the field of natural language processing technologies, and in particular, to an intention recognition method, apparatus, device, and computer readable storage medium.
Background
Intent recognition is a sub-task of Natural Language Processing (NLP), and for multiple rounds of dialog, understanding the content of a user's speech requires abstracting a semantic understanding definition of the user's speech by means of understanding the business and data analysis of the dialog text, i.e., an intent recognition module.
In the prior art, the classification method is the most effective method for identifying intentions; generally, the intention identification by using a pre-training model + fine-tune is widely applied, but under the conditions of few samples and unbalanced data distribution, the effect of the pre-training model + fine-tune is poor, a large amount of labeled data is needed to solve the problem of unbalanced sample distribution, and a large amount of manpower and material resources are needed to be consumed for labeling a large sample; when the intention is identified by using a pre-training model + Prompt (a NLP paradigm) + prediction, the label mapping words in the label word mapper are manually defined, and the method has the defects of strong subjectivity, small coverage and the like. Therefore, how to provide the intention recognition of the pre-training model + fine-tune by using less labeled data, solve the problem of unbalanced sample distribution, solve the problem that label mapping words are manually defined and have strong subjectivity and small coverage, and improve the accuracy of the intention recognition is a problem which needs to be solved urgently nowadays.
Disclosure of Invention
The invention aims to provide an intention recognition method, an intention recognition device, intention recognition equipment and a computer readable storage medium, which can perform intention recognition of a pre-training model + fine-tune by using less labeled data, solve the problem of unbalanced sample distribution and improve the accuracy of intention recognition.
In order to solve the above technical problem, the present invention provides an intention identifying method, including:
acquiring label similar words corresponding to each label by using a pre-training model, and constructing a label word mapper;
acquiring small sample labeling data corresponding to each label by using a preset regular expression; the preset regular expression comprises tag key words corresponding to the tags respectively;
according to the small sample labeling data, carrying out frequency adjustment and/or correlation adjustment on the label word mapper to obtain an adjusted label word mapper;
and matching the predicted words corresponding to the original sentences to be recognized and predicted by the pre-training language model by using the label word adjustment mapper to obtain the labels corresponding to the predicted words.
Optionally, before the pre-training model is used to obtain the labeled similar words corresponding to each label and construct the labeled word mapper, the method further includes:
determining a target label according to the acquired service scene information;
correspondingly, the obtaining of the label similar words corresponding to each label by using the pre-training model and the constructing of the label word mapper include:
and obtaining label similar words corresponding to each target label by using the pre-training model, and constructing the label word mapper.
Optionally, the obtaining, by using the pre-training model, a label similar word corresponding to each label, and constructing a label word mapper includes:
acquiring label similar words corresponding to each label by using a pre-training wobert model;
determining a first preset number of label words corresponding to each label according to the label similar words;
and constructing the label word mapper according to each label and the corresponding label word.
Optionally, the determining, according to the tag similar words, a first preset number of tag words corresponding to each of the tags includes:
acquiring target similar words in the label similar words; the target similar words do not comprise stop words and label similar words corresponding to the labels;
determining effective alternative positions of a second preset number corresponding to each label according to the vocabulary distribution probability of the target similar word position corresponding to each label; the effective substitution positions are the target similar positions of the second preset number before the vocabulary distribution probability in the target similar word positions corresponding to the labels is arranged in a descending order;
determining a first preset number of label words corresponding to each label according to a second preset number of effective alternative positions corresponding to each label; the label words are the target similar words of the first preset number in the target similar words corresponding to the labels, and the target similar words are arranged in descending order according to the times of appearing at the effective substitution positions.
Optionally, the performing frequency adjustment and/or correlation adjustment on the label word mapper according to the small sample labeling data to obtain an adjusted label word mapper includes:
detecting the small sample labeling data by using the pre-training model to obtain the occurrence frequency of label words corresponding to each label in the label word mapper;
according to the occurrence frequency, carrying out frequency adjustment on the label word mapper to obtain a preliminary optimized label word mapper;
determining importance information of each label corresponding to each label in the preliminary optimization label word mapper by using a TF-IDF formula;
and according to the importance information, performing relevance adjustment on the preliminary optimized tagged word mapper to obtain the adjusted tagged word mapper.
Optionally, the matching, by using the adjusted label word mapper, a predicted word corresponding to an original sentence to be recognized and predicted by a pre-training language model to obtain a label corresponding to the predicted word includes:
acquiring an original sentence to be identified;
predicting a prediction word corresponding to the original sentence to be recognized by utilizing a pre-training language RoBERTA model;
and matching the label corresponding to the predicted word by utilizing the adjusting label word mapper.
Optionally, the predicting, by using a pre-training language RoBERTa model, a predicted word corresponding to the original sentence to be recognized includes:
inputting the original sentence to be recognized into a preset prompt template of the RoBERTA model of the pre-training language, and predicting by utilizing an MLM (maximum likelihood model) to obtain a predicted word of an answer text empty position of the preset prompt template; the preset prompt template comprises the answer text empty position and an input empty position for inputting the original sentence to be recognized.
The present invention also provides an intention recognition apparatus, including:
the construction module is used for acquiring label similar words corresponding to the labels by utilizing the pre-training model and constructing a label word mapper;
the extraction module is used for acquiring small sample labeling data corresponding to each label by using a preset regular expression; the preset regular expression comprises label keywords corresponding to the labels respectively;
the adjusting module is used for carrying out frequency adjustment and/or correlation adjustment on the label word mapper according to the small sample labeling data to obtain an adjusted label word mapper;
and the matching module is used for matching the predicted words corresponding to the original sentences to be recognized and predicted by the pre-training language model by utilizing the label word adjusting mapper to obtain the labels corresponding to the predicted words.
The present invention also provides an intention identifying apparatus including:
a memory for storing a computer program;
a processor for implementing the steps of the intent recognition method as described above when executing the computer program.
Furthermore, the present invention also provides a computer-readable storage medium having stored thereon a computer program which, when being executed by a processor, carries out the steps of the intent recognition method as described above.
The invention provides an intention identification method, which comprises the following steps: acquiring label similar words corresponding to each label by using a pre-training model, and constructing a label word mapper; acquiring small sample labeling data corresponding to each label by using a preset regular expression; the preset regular expression comprises label keywords corresponding to the labels; according to the small sample labeling data, carrying out frequency adjustment and/or correlation adjustment on the label word mapper to obtain an adjusted label word mapper; matching a predicted word corresponding to the original sentence to be recognized, which is predicted by the pre-training language model, by using an adjusting label word mapper to obtain a label corresponding to the predicted word;
therefore, the small sample labeling data corresponding to each label is obtained by using the preset regular expression, and the labeled small sample data can be obtained by regular extraction according to the label key words of each label, so that the problem of unbalanced sample distribution is solved; the label mapping words in the label mapping device can be automatically configured by acquiring the label similar words corresponding to the labels by using the pre-training model and constructing the label mapping device; the frequency adjustment and/or the correlation adjustment are/is carried out on the label word mapper according to the small sample labeled data, the label word mapper is obtained and adjusted and optimized, and the intention identification accuracy is improved. In addition, the invention also provides an intention recognition device, equipment and a computer readable storage medium, and the intention recognition device, the equipment and the computer readable storage medium also have the beneficial effects.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
FIG. 1 is a flow chart of an intent recognition method according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a wobert model of another intention recognition method according to an embodiment of the present invention;
FIG. 3 is a flowchart illustrating another method for identifying intentions according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of a RoBERTA model of another intent recognition method according to an embodiment of the present invention;
FIG. 5 is a diagram illustrating matching of predicted words in another method for intent recognition according to an embodiment of the present invention;
FIG. 6 is a block diagram of an intention recognition apparatus according to an embodiment of the present invention;
fig. 7 is a schematic structural diagram of an intention identifying apparatus according to an embodiment of the present invention;
fig. 8 is a schematic structural diagram of an intention identifying device according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, fig. 1 is a flowchart illustrating an intention identifying method according to an embodiment of the invention. The method can comprise the following steps:
step 101: and (4) acquiring the label similar words corresponding to the labels by using a pre-training model, and constructing a label word mapper.
It can be understood that, in this step, the processor may search, by using the pre-training model, for respective corresponding similar words (i.e., label similar words) of each label (i.e., label name) to use the label similar words as label mapping words, and construct, by using the labels and the respective corresponding label similar words, a label word mapper containing a correspondence between each label and the respective corresponding label similar words; because the label similar words searched by the pre-training model have semantic similarity, the defects of strong subjectivity, small coverage and the like of manual definition can be avoided.
Specifically, the pre-training model is used in this step to obtain the label similar words corresponding to each label, and a specific manner for constructing the label Word mapper may be set by a designer according to a practical scene and a user requirement, for example, in this embodiment, the pre-training model may adopt a wobert (Word-based BERT, pre-training model based on Word granularity) model, that is, a pre-training wobert model, and in this step, the processor may obtain the label similar words corresponding to each label by using the pre-training wobert model, and construct the label Word mapper. The pre-training model in this embodiment may also adopt other types of training models, as long as the processor can obtain the label similar words corresponding to each label by using the pre-training model, and construct the label word mapper, which is not limited in this embodiment.
Correspondingly, the specific mode of acquiring the label similar words corresponding to the labels by utilizing the pre-trained wobert model and constructing the label word mapper can be set by a designer, for example, the processor can acquire the label similar words corresponding to the labels by utilizing the pre-trained wobert model; determining a first preset number of label words corresponding to each label according to the label similar words; constructing a label word mapper according to each label and each corresponding label word; that is, the processor may select a first preset number of tagged near words from the tagged near words respectively corresponding to each tag as tagged words (i.e., tagged mapped words) to construct the tag mapper. The processor may also directly use each label and all the corresponding label similar words to construct a label word mapper after obtaining the label similar words corresponding to each label. The present embodiment does not set any limit to this.
Correspondingly, for the specific mode that the processor determines the first preset number of label words corresponding to each label according to the label similar words, the specific mode can be set by a designer, for example, a target similar word in the label similar words is obtained; the target similar words do not comprise stop words and label similar words corresponding to the labels; determining effective alternative positions of a second preset number corresponding to each label according to the vocabulary distribution probability of the target similar word position corresponding to each label; the effective substitution positions are first preset number of target similar positions in which the vocabulary distribution probability in the target similar word positions corresponding to the labels is arranged in a descending order; determining a first preset number of label words corresponding to each label according to a second preset number of effective substitution positions corresponding to each label; the label words are the first preset number of target similar words in descending order according to the times of occurrence at the effective substitution positions in the target similar words corresponding to the labels.
As shown in fig. 2, a pre-trained wobert model is adopted, and each time a tag name (such as contrast) appears in a corpus, a sentence is input into the wobert model, and CLS is a mark placed at the head of the first sentence; then inputting the vector of the corresponding position of the tag name into a model comprising two full-connection layers and an activation function, performing softmax on the output of the model, outputting a vocabulary distribution probability, namely, for the tag name, each word can replace the probability of the tag name, then arranging the words in a descending order, and regarding top50 (namely, a second preset number) words as effective replacements; for all documents, 100 (i.e., a first preset number) words are respectively selected for each category (i.e., tags), and the arrangement order is the number of times these words appear in the valid alternatives, but the stop word and the words appearing in the plurality of tags are deleted, that is, the first preset number of tagged words before the number of times appearing in the valid alternatives are arranged in a descending order can be used as tagged words to construct a tagged word mapper.
Further, the method provided by this embodiment may further include, before step 101: determining a target label according to the acquired service scene information; correspondingly, in step 101, the processor may obtain the label similar words corresponding to each target label by using the pre-training model, and construct a label word mapper. That is, the processor may determine a specific service scenario identified by the user intention according to the acquired service scenario information, so as to determine a target tag (such as the user intention tag in fig. 3) identified by the user intention, that is, a category of the classification task, where the category may include user intention categories such as contrast, price dissimilarity, and WeChat adding; therefore, the accuracy of intention identification is determined by the classification of the classification task under the service scene.
Step 102: acquiring small sample labeling data corresponding to each label by using a preset regular expression; the preset regular expression comprises label keywords corresponding to the labels.
It can be understood that, in this step, the labeled small sample data (i.e., the small sample labeled data) can be obtained by using regular extraction according to the label keywords, the process is simple and fast, and the balance of sample distribution can be ensured.
Specifically, the specific setting of the preset regular expression in this step may be set by a designer according to a practical scene and a user requirement, for example, the label keyword in the preset regular expression corresponding to each label may include all or part of the label similar words corresponding to the label; other words set in advance can also be included; for example, the label: contrast, label keyword: comparing, comparing and competing for the product, wherein the preset regular expression can be (comparison | competition product), a part of sample data can be extracted as labeled small sample data (namely small sample labeled data) according to the preset regular expression, for example, sample data with a preset small sample number is extracted as small sample labeled data, and a label (for example, comparison) corresponding to the preset regular expression and/or a corresponding label keyword can be used as a labeled content.
Step 103: and according to the small sample labeling data, carrying out frequency adjustment and/or correlation adjustment on the label word mapper to obtain an adjusted label word mapper.
It is understood that, in this step, the processor may perform frequency adjustment and/or correlation adjustment on the tagged word mapper by using the obtained small sample tagging data, and optimize the tagged word mapper, so as to obtain an optimized tagged word mapper (i.e., adjust the tagged word mapper).
Specifically, the specific manner of adjusting the tag word mapper by performing frequency adjustment and/or correlation adjustment on the tag word mapper according to the small sample labeling data by the processor in this step can be set by a designer according to a practical scenario and a user requirement, for example, the processor may perform frequency adjustment and correlation adjustment on the tag word mapper according to the small sample labeling data to obtain an adjusted tag word mapper; for example, the processor may perform frequency adjustment on the tag word mapper by using the pre-training model according to the small sample labeled data to obtain a preliminary optimized tag word mapper; performing correlation adjustment on the preliminary optimized tagged word mapper by using a TF-IDF (term frequency-inverse text frequency index) formula to obtain an adjusted tagged word mapper; alternatively, the processor may perform the correlation adjustment before performing the frequency adjustment, or the correlation adjustment and the frequency adjustment may be performed simultaneously and in parallel. The present embodiment does not set any limit to this.
Correspondingly, the processor adjusts the frequency of the label word mapper by using the pre-training model according to the small sample label data to obtain a specific mode for preliminarily optimizing the label word mapper, which can be set by a designer, for example, the processor can detect the small sample label data by using the pre-training model (such as a pre-training wobert model) to obtain the occurrence frequency of the label words (i.e., label mapping words) corresponding to the labels in the label word mapper; and adjusting the frequency of the label word mapper according to the occurrence frequency to obtain a preliminary optimized label word mapper, for example, deleting the label words with the occurrence frequency smaller than a frequency threshold, or only keeping the third preset number of label words before the occurrence frequency corresponding to each label is arranged in a descending order. For example, in this step, the processor may use the output probability of the pre-trained wobert model itself for the tagged word as the prior probability of the tagged word according to the small sample labeled data to estimate the prior occurrence frequency of the tagged word, remove the tagged words with smaller frequency, and optimize the tagged word mapper to obtain a preliminary optimized tagged word mapper.
Similarly, for the processor, the processor uses the TF-IDF formula to perform correlation adjustment on the preliminary optimized tagged word mapper, and obtain a specific manner for adjusting the tagged word mapper, which can be set by a designer, for example, the processor can determine the importance information of the tagged words corresponding to each tag in the preliminary optimized tagged word mapper by using the TF-IDF formula; and according to the importance information, performing relevance adjustment on the preliminary optimization tagged word mapper to obtain an adjusted tagged word mapper, and deleting tagged words with the importance information smaller than a relevance threshold value. For example, the preliminarily optimized tagged word mapper obtained by the processor in this step is similar to { ' compare ', ' race ', ' compare ', ' so. }, because each tag (tag name) corresponds to a larger number of tagged words, such as 100 (i.e., the first preset number) of tagged words defined by the originally constructed tagged word mapper, but there are less related tagged words and tags, and some tagged words are confused with different tags at the same time, therefore, the idea of TF-IDF may be used to assign each tagged word an importance to a specific category (tag), that is, the processor may determine importance information of the tagged word corresponding to each tag in the preliminarily optimized tagged word mapper by using the following TF-IDF formula, such as obtaining the tagged words and importance information for comparing the tag as follows: { race quality: 0.1, comparison: 0.2, comparison: 0.1. }, so that the relevance adjustment can be performed on the primary optimized tagged word mapper by using the importance information, and the relevance of the tag and the corresponding tagged word is ensured.
In the above, R (v) is the importance of a certain tag word to a specific category (specific tag), such as the importance of the tag word v to the tag y; f (v) is the corresponding category of the tag word v; r (v, y) is the correlation between the label word v and the label y, similar to the DF term, i.e. if a word appears more frequently, the correlation is higher; the right entry is similar to the IDF entry, requiring that the entry be large, i.e., requiring that the tag word v and its non-corresponding tag have small dependencies.
Step 104: and matching the predicted words corresponding to the original sentences to be recognized, which are predicted by the pre-training language model, by using the adjusted label word mapper to obtain labels corresponding to the predicted words.
It can be understood that, in this embodiment, the labeled word mapper obtained through optimization in step 1033 (i.e., adjusting the labeled word mapper) may be used to match the predicted word (i.e., the predicted answer) corresponding to the original sentence to be recognized and predicted by the pre-trained language model, so as to obtain the label corresponding to the predicted word, i.e., the recognized user intention.
Correspondingly, the step can also comprise a process of predicting the predicted words corresponding to the original sentences to be recognized by utilizing the pre-training language model. For example, when the pre-training language model adopts the RoBERTa model (i.e., the pre-training language RoBERTa model) as shown in fig. 3, the processor may predict the predicted word corresponding to the original sentence to be recognized by using the pre-training language RoBERTa model; that is, in this step, the processor may obtain the original sentence to be recognized; predicting a prediction word corresponding to an original sentence to be recognized by utilizing a pre-training language RoBERTA model; and matching the corresponding label of the predicted word by utilizing the adjustment label word mapper.
Specifically, the specific manner of predicting the predicted word corresponding to the original sentence to be recognized by the processor by using the pre-training language RoBERTa model can be set by a designer, for example, the original sentence to be recognized is input into a preset prompt template of the pre-training language RoBERTa model, and the predicted word of the empty position of the answer text of the preset prompt template is predicted by using an MLM (mask language model); the preset prompt template comprises an answer text empty position and an input empty position for inputting an original sentence to be recognized. That is to say, in this embodiment, a label word mapper is constructed in a manner of combining the pre-training model + fine-tune with the pre-training model + Prompt, frequency fine tuning is performed by using the pre-training model wobert, correlation fine tuning is performed on a label word by using TF-IDF, user intent is recognized by using the Prompt-tuning under a small sample (raw-shot), and thus, raw _ shot user intent recognition based on the Prompt and wobert is realized.
The prompt is used for a classification task and can convert a classification problem into a problem of predicting and labeling related words; therefore, it is first necessary to construct a template containing [ MASK ] (i.e. a preset prompt template, such as the prompt template in FIG. 4), and then let the MLM model predict the words at the [ MASK ] position, so as to convert the classification task into a MASK language modeling problem. As in this embodiment, one or more preset prompt templates may be constructed by using a preset empty template, where the preset empty template is usually a segment of natural language and includes two empty positions: a location x for filling in the original sentence and a location for generating the answer text (i.e. [ MASK ]). For example: x is { "mask" }', it is intended { "mask" } { "mask": x } is about { "mask" }, { "mask": x } in this session, the client said about { "mask" }.
It should be noted that, in this step, the processor may match the predicted word obtained by predicting the pre-trained language model with the optimized tag word mapper (i.e., adjusting the tag word mapper) to the tag (tag name) sent by the predicted word, so as to avoid the limitation of the tag word; as shown in fig. 5, the word of the [ MASK ] position predicted by the model is the comparison tag word, and according to the optimized tag mapper, the comparison tag is the comparison tag, so that the user intention can be determined as the comparison tag.
In the embodiment of the invention, the small sample labeling data corresponding to each label is obtained by using the preset regular expression, and the labeled small sample data can be obtained by regular extraction according to the label key words of each label, so that the problem of unbalanced sample distribution is solved; the label mapping words in the label mapping device can be automatically configured by acquiring the label similar words corresponding to the labels by using the pre-training model and constructing the label mapping device; according to the small sample labeling data, frequency adjustment and/or correlation adjustment are/is carried out on the label word mapper, the adjusted label word mapper is obtained, the label word mapper is adjusted and optimized, and the intention identification accuracy is improved.
Corresponding to the above method embodiment, the embodiment of the present invention further provides an intention identification device, and an intention identification device described below and an intention identification method described above may be referred to in correspondence with each other.
Referring to fig. 6, fig. 6 is a block diagram illustrating an intention recognition device according to an embodiment of the present invention. The apparatus may include:
the building module 10 is configured to obtain tag similar words corresponding to each tag by using the pre-training model, and build a tag word mapper;
the extraction module 20 is configured to obtain small sample labeling data corresponding to each label by using a preset regular expression; the preset regular expression comprises label keywords corresponding to the labels;
the adjusting module 30 is configured to perform frequency adjustment and/or correlation adjustment on the tag word mapper according to the small sample labeled data, and obtain an adjusted tag word mapper;
and the matching module 40 is configured to match the predicted word corresponding to the original sentence to be recognized, which is predicted by the pre-training language model, by using the adjusted tag word mapper, so as to obtain a tag corresponding to the predicted word.
Optionally, the apparatus may further include:
the determining module is used for determining a target label according to the acquired service scene information;
correspondingly, the building module 10 may be specifically configured to obtain, by using the pre-training model, the label similar words corresponding to the target labels, and build the label word mapper.
Optionally, the building block 10 includes:
the model obtaining submodule is used for obtaining the label similar words corresponding to the labels by utilizing the pre-training wobert model;
the determining submodule is used for determining a first preset number of label words corresponding to each label according to the label similar words;
and the construction submodule is used for constructing a label word mapper according to each label and each corresponding label word.
Optionally, the determining sub-module may include:
a similarity obtaining unit, configured to obtain a target similar word from the tag similar words; the target similar words do not comprise stop words and label similar words corresponding to the labels;
the effective determining unit is used for determining the effective alternative positions of a second preset number corresponding to each label according to the vocabulary distribution probability of the target similar word position corresponding to each label; the effective substitution positions are the first preset number of target similar positions in which the vocabulary distribution probability in the target similar word positions corresponding to the labels is arranged in a descending order;
the label word determining unit is used for determining the label words of the first preset number corresponding to the labels according to the effective substitution positions of the second preset number corresponding to the labels; the label words are the first preset number of target similar words in descending order according to the times of occurrence at the effective substitution positions in the target similar words corresponding to the labels.
Optionally, the adjusting module 30 may include:
the model detection submodule is used for detecting the small sample labeling data by utilizing the pre-training model and acquiring the occurrence frequency of the label words corresponding to the labels in the label word mapper;
the frequency adjustment submodule is used for adjusting the frequency of the label word mapper according to the occurrence frequency to obtain a preliminary optimized label word mapper;
the importance determination submodule is used for determining importance information of the label words corresponding to each label in the preliminary optimization label word mapper by using a TF-IDF formula;
and the correlation adjustment submodule is used for performing correlation adjustment on the preliminary optimized tagged word mapper according to the importance information to obtain an adjusted tagged word mapper.
Optionally, the matching module 40 may include:
the recognition obtaining submodule is used for obtaining an original sentence to be recognized;
the recognition and prediction submodule is used for predicting a prediction word corresponding to the original sentence to be recognized by utilizing a pre-training language RoBERTA model;
and the identification matching submodule is used for matching the corresponding label of the predicted word by utilizing the adjustment label word mapper.
Optionally, the recognition and prediction submodule may be specifically configured to input the original sentence to be recognized into a preset prompt template of a pre-training language RoBERTa model, and obtain a predicted word of an answer text empty position of the preset prompt template by using an MLM model prediction; the preset prompt template comprises an answer text empty position and an input empty position for inputting an original sentence to be recognized.
In this embodiment, the extraction module 20 obtains the small sample labeling data corresponding to each label by using a preset regular expression, and can obtain the labeled small sample data by using regular extraction according to the label key words of each label, thereby solving the problem of unbalanced sample distribution; the building module 10 obtains the label similar words corresponding to each label by using the pre-training model, builds a label word mapper, and can automatically configure the label mapping words in the label word mapper; the adjustment module 30 adjusts the frequency and/or the correlation of the label word mapper according to the small sample labeling data to obtain an adjusted label word mapper, and adjusts and optimizes the label word mapper to improve the accuracy of intention identification.
Corresponding to the above method embodiment, the embodiment of the present invention further provides an intention identifying device, and an intention identifying device described below and an intention identifying method described above may be referred to correspondingly.
Referring to fig. 7, fig. 7 is a schematic structural diagram of an intention identifying device according to an embodiment of the present invention. The apparatus may include:
a memory D1 for storing a computer program;
a processor D2, configured to implement the steps of the intention identification method provided by the above method embodiments when executing the computer program.
Specifically, referring to fig. 8, fig. 8 is a schematic diagram of a specific structure of an intention identifying device according to an embodiment of the present invention, the intention identifying device 310 may have a relatively large difference due to different configurations or performances, and may include one or more processors (CPUs) 322 (e.g., one or more processors) and a memory 332, and one or more storage media 330 (e.g., one or more mass storage devices) storing an application 342 or data 344. Memory 332 and storage media 330 may be, among other things, transient storage or persistent storage. The program stored on the storage medium 330 may include one or more modules (not shown), each of which may include a series of instructions operating on a data processing device. Still further, central processor 322 may be configured to communicate with storage medium 330 to perform a series of instruction operations in storage medium 330 on intent recognition device 310.
The intent recognition device 310 may also include one or more power sources 326, one or more wired or wireless network interfaces 350, one or more input-output interfaces 358, and/or one or more operating systems 341. For example, windows Server, mac OS XTM, unixTM, linuxTM, freeBSDTM, etc.
The steps in the above-described intention identifying method may be implemented by the structure of the intention identifying apparatus.
Corresponding to the above method embodiment, the embodiment of the present invention further provides a computer-readable storage medium, and a computer-readable storage medium described below and an intention identification method described above may be correspondingly referred to each other.
A computer-readable storage medium, on which a computer program is stored, which computer program, when being executed by a processor, carries out the steps of the intention identification method provided by the above-mentioned method embodiments.
The computer-readable storage medium may be a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and various other readable storage media capable of storing program codes.
The embodiments are described in a progressive manner in the specification, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. The apparatuses, devices and computer-readable storage media disclosed in the embodiments correspond to the methods disclosed in the embodiments, so that the description is simple, and the relevant points can be referred to the description of the method.
The intention identifying method, device, equipment and computer readable storage medium provided by the invention are described in detail above. The principles and embodiments of the present invention are explained herein using specific examples, which are presented only to assist in understanding the method and its core concepts. It should be noted that, for those skilled in the art, without departing from the principle of the present invention, it is possible to make various improvements and modifications to the present invention, and those improvements and modifications also fall within the scope of the claims of the present invention.
Claims (10)
1. An intent recognition method, comprising:
acquiring label similar words corresponding to each label by using a pre-training model, and constructing a label word mapper;
acquiring small sample labeling data corresponding to each label by using a preset regular expression; the preset regular expression comprises tag key words corresponding to the tags respectively;
according to the small sample labeling data, carrying out frequency adjustment and/or correlation adjustment on the label word mapper to obtain an adjusted label word mapper;
and matching the predicted words corresponding to the original sentences to be recognized and predicted by the pre-training language model by using the label word adjustment mapper to obtain the labels corresponding to the predicted words.
2. The method of claim 1, wherein before the pre-training model is used to obtain labeled proximal words corresponding to labels and construct a labeled word mapper, the method further comprises:
determining a target label according to the acquired service scene information;
correspondingly, the obtaining of the label similar words corresponding to each label by using the pre-training model and the constructing of the label word mapper include:
and obtaining label similar words corresponding to each target label by using the pre-training model, and constructing the label word mapper.
3. The method for identifying intentions of claim 1, wherein the obtaining of the labeled similar words corresponding to the labels by using the pre-training model and constructing a labeled word mapper comprise:
acquiring label similar words corresponding to each label by using a pre-training wobert model;
determining a first preset number of label words corresponding to each label according to the label similar words;
and constructing the label word mapper according to each label and the corresponding label word.
4. The method according to claim 3, wherein the determining a first preset number of label words corresponding to each label according to the label proximal words comprises:
acquiring a target similar word in the label similar words; the target similar words do not comprise stop words and label similar words corresponding to the labels;
determining effective alternative positions of a second preset number corresponding to each label according to the vocabulary distribution probability of the target similar word position corresponding to each label; the effective substitution positions are the target similar positions of the second preset number before the vocabulary distribution probability in the target similar word positions corresponding to the labels is arranged in a descending order;
determining a first preset number of label words corresponding to each label according to a second preset number of effective alternative positions corresponding to each label; the label words are the target similar words of the first preset number in the target similar words corresponding to the labels, and the target similar words are arranged in descending order according to the times of appearing at the effective substitution positions.
5. The method according to claim 1, wherein the performing frequency adjustment and/or correlation adjustment on the tag word mapper according to the small sample labeling data to obtain an adjusted tag word mapper comprises:
detecting the small sample labeling data by using the pre-training model to obtain the occurrence frequency of the label words corresponding to the labels in the label word mapper;
according to the occurrence frequency, carrying out frequency adjustment on the label word mapper to obtain a preliminary optimized label word mapper;
determining importance information of each label corresponding to each label in the preliminary optimization label word mapper by using a TF-IDF formula;
and according to the importance information, performing correlation adjustment on the preliminary optimization tagged word mapper to obtain the adjusted tagged word mapper.
6. The intention recognition method according to any one of claims 1 to 5, wherein the matching, by the adjusted tagged word mapper, the predicted word corresponding to the original sentence to be recognized, which is predicted by the pre-trained language model, to obtain the tag corresponding to the predicted word comprises:
acquiring an original sentence to be identified;
predicting a prediction word corresponding to the original sentence to be recognized by utilizing a pre-training language RoBERTA model;
and matching the corresponding label of the predicted word by utilizing the label-word adjusting mapper.
7. The intention recognition method of claim 6, wherein the predicting the predicted word corresponding to the original sentence to be recognized by using a pre-trained language RoBERTA model comprises:
inputting the original sentence to be recognized into a preset prompt template of the RoBERTA model of the pre-training language, and predicting by utilizing an MLM (maximum likelihood model) to obtain a predicted word of an answer text empty position of the preset prompt template; the preset prompt template comprises the answer text empty position and an input empty position for inputting the original sentence to be recognized.
8. An intention recognition device, comprising:
the construction module is used for acquiring label similar words corresponding to the labels by utilizing the pre-training model and constructing a label word mapper;
the extraction module is used for acquiring small sample labeling data corresponding to each label by using a preset regular expression; the preset regular expression comprises tag key words corresponding to the tags respectively;
the adjusting module is used for carrying out frequency adjustment and/or correlation adjustment on the label word mapper according to the small sample labeling data to obtain an adjusted label word mapper;
and the matching module is used for matching the predicted words corresponding to the original sentences to be recognized and predicted by the pre-training language model by utilizing the label word adjusting mapper to obtain the labels corresponding to the predicted words.
9. An intent recognition device, comprising:
a memory for storing a computer program;
a processor for implementing the steps of the intent recognition method of any of claims 1-7 when executing the computer program.
10. A computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, which computer program, when being executed by a processor, carries out the steps of the intent recognition method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211190704.0A CN115526171A (en) | 2022-09-28 | 2022-09-28 | Intention identification method, device, equipment and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211190704.0A CN115526171A (en) | 2022-09-28 | 2022-09-28 | Intention identification method, device, equipment and computer readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115526171A true CN115526171A (en) | 2022-12-27 |
Family
ID=84699590
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211190704.0A Pending CN115526171A (en) | 2022-09-28 | 2022-09-28 | Intention identification method, device, equipment and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115526171A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116304063A (en) * | 2023-05-19 | 2023-06-23 | 吉林大学 | Simple emotion knowledge enhancement prompt tuning aspect-level emotion classification method |
| CN116580408A (en) * | 2023-06-06 | 2023-08-11 | 上海任意门科技有限公司 | Image generation method and device, electronic equipment and storage medium |
-
2022
- 2022-09-28 CN CN202211190704.0A patent/CN115526171A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116304063A (en) * | 2023-05-19 | 2023-06-23 | 吉林大学 | Simple emotion knowledge enhancement prompt tuning aspect-level emotion classification method |
| CN116304063B (en) * | 2023-05-19 | 2023-07-21 | 吉林大学 | Simple emotion knowledge enhancement prompt tuning aspect-level emotion classification method |
| CN116580408A (en) * | 2023-06-06 | 2023-08-11 | 上海任意门科技有限公司 | Image generation method and device, electronic equipment and storage medium |
| CN116580408B (en) * | 2023-06-06 | 2023-11-03 | 上海任意门科技有限公司 | Image generation method and device, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108829893B (en) | Method and device for determining video label, storage medium and terminal equipment | |
| CN108363790B (en) | Method, device, equipment and storage medium for evaluating comments | |
| CN109086265B (en) | Semantic training method and multi-semantic word disambiguation method in short text | |
| CN111783518A (en) | Training sample generation method and device, electronic equipment and readable storage medium | |
| CN112817561B (en) | Transaction type functional point structured extraction method and system for software demand document | |
| CN115526171A (en) | Intention identification method, device, equipment and computer readable storage medium | |
| CN107102993B (en) | User appeal analysis method and device | |
| CN112188312B (en) | Method and device for determining video material of news | |
| US11699034B2 (en) | Hybrid artificial intelligence system for semi-automatic patent infringement analysis | |
| CN110334186A (en) | Data query method, apparatus, computer equipment and computer readable storage medium | |
| CN114661872B (en) | Beginner-oriented API self-adaptive recommendation method and system | |
| CN113157859A (en) | Event detection method based on upper concept information | |
| CN114595327A (en) | Data enhancement method and device, electronic equipment and storage medium | |
| CN115859980A (en) | Semi-supervised named entity identification method, system and electronic equipment | |
| CN110795942B (en) | Keyword determination method and device based on semantic recognition and storage medium | |
| CN112527977A (en) | Concept extraction method and device, electronic equipment and storage medium | |
| CN111159381A (en) | Data searching method and device | |
| RU2546064C1 (en) | Distributed system and method of language translation | |
| CN116933782A (en) | E-commerce text keyword extraction processing method and system | |
| CN114742062B (en) | Text keyword extraction processing method and system | |
| CN116090450A (en) | Text processing method and computing device | |
| CN112487154B (en) | Intelligent search method based on natural language | |
| US20210311985A1 (en) | Method and apparatus for image processing, electronic device, and computer readable storage medium | |
| CN114254622A (en) | Intention identification method and device | |
| CN114118049A (en) | Information acquisition method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |