CN110928997A - Intention recognition method and device, electronic equipment and readable storage medium - Google Patents
Intention recognition method and device, electronic equipment and readable storage medium Download PDFInfo
- Publication number
- CN110928997A CN110928997A CN201911232049.9A CN201911232049A CN110928997A CN 110928997 A CN110928997 A CN 110928997A CN 201911232049 A CN201911232049 A CN 201911232049A CN 110928997 A CN110928997 A CN 110928997A
- Authority
- CN
- China
- Prior art keywords
- vector
- semantic
- input text
- model
- lstm
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 81
- 239000013598 vector Substances 0.000 claims abstract description 430
- 238000000605 extraction Methods 0.000 claims abstract description 55
- 238000006243 chemical reaction Methods 0.000 claims abstract description 24
- 230000006870 function Effects 0.000 claims description 26
- 230000015654 memory Effects 0.000 claims description 19
- 238000013528 artificial neural network Methods 0.000 claims description 17
- 238000013527 convolutional neural network Methods 0.000 claims description 12
- 238000011176 pooling Methods 0.000 claims description 12
- 238000004364 calculation method Methods 0.000 claims description 10
- 238000004590 computer program Methods 0.000 claims description 7
- 230000007246 mechanism Effects 0.000 claims description 6
- 238000004806 packaging method and process Methods 0.000 claims description 6
- 238000012545 processing Methods 0.000 abstract description 8
- 230000008569 process Effects 0.000 description 36
- 238000004891 communication Methods 0.000 description 9
- 238000012549 training Methods 0.000 description 9
- 238000010586 diagram Methods 0.000 description 7
- 239000011159 matrix material Substances 0.000 description 5
- 230000004913 activation Effects 0.000 description 4
- 230000002457 bidirectional effect Effects 0.000 description 4
- 230000006403 short-term memory Effects 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 238000004422 calculation algorithm Methods 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 239000013604 expression vector Substances 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 230000007787 long-term memory Effects 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000008034 disappearance Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000007477 logistic regression Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 230000014759 maintenance of location Effects 0.000 description 1
- 230000000873 masking effect Effects 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 238000007637 random forest analysis Methods 0.000 description 1
- 230000002441 reversible effect Effects 0.000 description 1
- 230000011664 signaling Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/3331—Query processing
- G06F16/334—Query execution
- G06F16/3344—Query execution using natural language analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Evolutionary Computation (AREA)
- Biophysics (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Databases & Information Systems (AREA)
- Machine Translation (AREA)
Abstract
The application provides an intention identification method, an intention identification device, electronic equipment and a readable storage medium, and relates to the technical field of language processing. The method comprises the following steps: acquiring an input text to be identified; performing word embedding vector conversion on the input text through a language model to obtain a first semantic vector corresponding to the input text; semantic feature extraction is carried out on the first semantic vector through a feature extraction model, and a second semantic vector corresponding to the input text is obtained; and determining an intention category corresponding to the input text based on the second semantic vector through an intention prediction model. According to the scheme, the three models are matched with each other, semantic features of the input text are extracted in sequence, so that more context information and deeper semantic relations in the input text can be obtained, more effective information is obtained, and the accuracy of the intention prediction of the input text can be effectively improved.
Description
Technical Field
The present application relates to the field of language processing technologies, and in particular, to an intention recognition method, an intention recognition apparatus, an electronic device, and a readable storage medium.
Background
With the continuous development of artificial intelligence technology, application scenarios based on intention recognition are increasing, for example, intention recognition is involved in the fields of intelligent customer service, intelligent question and answer, intelligent assistant, intelligent robot and the like.
In the prior art, traditional machine learning algorithms are generally adopted for intention recognition, such as methods of random forests, logistic regression, classifiers and the like, however, these methods classify according to values or categories of feature dimensions of structured data, so that the recognition accuracy is low when intention recognition is performed.
Disclosure of Invention
An embodiment of the present application provides an intention identification method, an intention identification device, an electronic apparatus, and a readable storage medium, so as to solve the problem of low accuracy of intention identification in the prior art.
In a first aspect, an embodiment of the present application provides an intention recognition method for performing intention recognition on an input text, where the method includes:
acquiring an input text to be identified;
performing word embedding vector conversion on the input text through a language model to obtain a first semantic vector corresponding to the input text, wherein the first semantic vector represents semantic information of each word in the input text;
semantic feature extraction is carried out on the first semantic vector through a feature extraction model, and a second semantic vector corresponding to the input text is obtained, wherein the second semantic vector represents context semantic information of each word in the input text;
and determining an intention category corresponding to the input text based on the second semantic vector through an intention prediction model.
In the implementation process, the three models are mutually matched to sequentially extract the semantic features of the input text, so that more context information and deeper semantic relation in the input text can be obtained, more effective information can be obtained, and the accuracy of the intention prediction of the input text can be effectively improved.
Optionally, the language model includes an embedding layer and M coding layers, where M is an integer greater than or equal to 2, and the performing word embedding vector conversion on the input text through the language model to obtain a first semantic vector corresponding to the input text includes:
performing word embedding vector conversion on the input text through the embedding layer to obtain a word vector, a sentence vector and a position vector corresponding to the input text, and adding the word vector, the sentence vector and the position vector to obtain a word embedding vector;
semantic coding is carried out on the word embedding vector through a first coding layer, so that a first coding vector corresponding to the input text is obtained, wherein the first coding vector comprises a coding vector corresponding to each word;
and sequentially taking i as 2 to M, continuing semantic coding on the i-1 coding vector through the ith coding layer to obtain the ith coding vector, and obtaining the Mth coding vector when i is taken as M, wherein the Mth coding vector is the first semantic vector corresponding to the input text.
In the implementation process, the semantic information of each word in the input text can be better extracted through the language model.
Optionally, each coding layer includes a self-attention layer, a feedforward neural network layer, and a residual layer, and the semantic coding is continued on the i-1 th coding vector through the i-th encoder to obtain the i-th coding vector, including:
performing attention mechanism calculation on the ith-1 encoding vector through the self-attention layer to obtain an ith output vector;
summing and normalizing the ith-1 encoding vector and the ith output vector through the residual error layer to obtain an ith normalized vector;
semantic feature extraction is carried out on the ith normalized vector through the feedforward neural network layer, and an ith semantic feature vector is obtained;
and summing and normalizing the ith semantic feature vector and the ith normalized vector through the residual error layer to obtain an ith coding vector.
In the implementation process, the language model can use each coding layer to code the vector, that is, the parameters of each coding layer can be used to extract the features of the input text, so that the associated features between words can be effectively extracted.
Optionally, the feature extraction model is a Bi-directional long-short term memory neural network Bi-LSTM model, and performing semantic feature extraction on the first semantic vector through the feature extraction model to obtain a second semantic vector corresponding to the input text, including:
calculating to obtain an output value of a forgetting gate through a sigmoid function based on the first semantic vector input in the LSTM unit in the Bi-LSTM model at the current moment and the second semantic vector output by the hidden layer of the LSTM unit at the last moment;
calculating to obtain an output value of an input gate through a sigmoid function based on the first semantic vector input in the LSTM unit in the Bi-LSTM model at the current moment and a second semantic vector output by a hidden layer of the LSTM unit at the last moment;
calculating a value of a temporary LSTM unit cell state through a tanh function based on the first semantic vector input in the LSTM unit in the Bi-LSTM model at the current moment and the second semantic vector output by the hidden layer of the LSTM unit at the last moment;
calculating to obtain the value of the LSTM unit cell state at the current moment based on the output value of the forgetting gate, the output value of the input gate, the value of the temporary LSTM unit cell state and the value of the LSTM unit cell state at the last moment;
calculating to obtain an output value of an output gate through a sigmoid function based on the first semantic vector input in the LSTM unit in the Bi-LSTM model at the current moment and a second semantic vector output by a hidden layer of the LSTM unit at the last moment;
obtaining an output vector of a hidden state at the current moment according to the output value of the output gate and the value of the cell state of the LSTM unit at the current moment;
obtaining an output vector of a forward LSTM network in the Bi-LSTM model according to the output vector of the hidden state at each moment;
obtaining an output vector of a backward LSTM network in the Bi-LSTM model according to the output vector of the hidden state at each moment;
and splicing the output vector of the forward LSTM network and the output vector of the backward LSTM network to obtain a second semantic vector output by the Bi-LSTM model.
In the implementation process, the context dependency relationship among word sequences can be better processed by utilizing the Bi-LSTM model, so that deeper semantic information of each word in the input text can be extracted.
Optionally, the intention prediction model is a convolutional neural network model, the convolutional neural network model includes a convolutional layer, a max pooling layer, and a full connection layer, and the determining, by the intention prediction model, an intention category corresponding to the input text based on the second semantic vector includes:
performing dot product operation on the second semantic vector and a convolution kernel through the convolution layer, and extracting a local maximum value through the maximum pooling layer to perform high-dimensional semantic feature extraction to obtain a semantic feature vector;
and calculating and obtaining the probability that the semantic feature vector belongs to each intention category through the full connection layer, and obtaining the intention category corresponding to the input text based on the probability.
In the implementation process, redundant information in the input text can be eliminated by utilizing the convolutional neural network model, and only important characteristic information of the redundant information is extracted, so that the accuracy is higher when the intention type prediction is carried out.
Optionally, the method further comprises:
identifying the input text, and searching a problem text matched with the input text from a database corresponding to the intention category;
and acquiring a corresponding reply text from the database according to the question text, and taking the reply text as a reply to the input text.
In the implementation process, the corresponding reply text is obtained according to the intention type and the input text, so that the input of the user can be responded in time, and the user experience is improved.
Optionally, the method further comprises:
and packaging an intention recognition model formed by the language model, the feature extraction model and the intention prediction model into an application service interface, and calling the intention recognition model through the application service interface to perform intention recognition on the input text.
In the implementation process, the intention recognition model is packaged as an application service interface, so that other users can use the intention recognition model directly through the application service interface.
In a second aspect, an embodiment of the present application provides an intention recognition apparatus for performing intention recognition on an input text, the apparatus including:
the text acquisition module is used for acquiring an input text to be identified;
the first semantic vector acquisition module is used for performing word embedding vector conversion on the input text through a language model to acquire a first semantic vector corresponding to the input text, wherein the first semantic vector represents semantic information of each word in the input text;
the second semantic vector acquisition module is used for extracting semantic features of the first semantic vector through a feature extraction model to obtain a second semantic vector corresponding to the input text, wherein the second semantic vector represents context semantic information of each word in the input text;
and the intention category determining module is used for determining an intention category corresponding to the input text based on the second semantic vector through an intention prediction model.
Optionally, the language model includes an embedded layer and M coding layers, where M is an integer greater than or equal to 2, and the first semantic vector obtaining module is configured to:
performing word embedding vector conversion on the input text through the embedding layer to obtain a word vector, a sentence vector and a position vector corresponding to the input text, and adding the word vector, the sentence vector and the position vector to obtain a word embedding vector;
semantic coding is carried out on the word embedding vector through a first coding layer, so that a first coding vector corresponding to the input text is obtained, wherein the first coding vector comprises a coding vector corresponding to each word;
and sequentially taking i as 2 to M, continuing semantic coding on the i-1 coding vector through the ith coding layer to obtain the ith coding vector, and obtaining the Mth coding vector when i is taken as M, wherein the Mth coding vector is the first semantic vector corresponding to the input text.
Optionally, each coding layer includes a self-attention layer, a feedforward neural network layer, and a residual layer, and the first semantic vector obtaining module is configured to:
performing attention mechanism calculation on the ith-1 encoding vector through the self-attention layer to obtain an ith output vector;
summing and normalizing the ith-1 encoding vector and the ith output vector through the residual error layer to obtain an ith normalized vector;
semantic feature extraction is carried out on the ith normalized vector through the feedforward neural network layer, and an ith semantic feature vector is obtained;
and summing and normalizing the ith semantic feature vector and the ith normalized vector through the residual error layer to obtain an ith coding vector.
Optionally, the feature extraction model is a Bi-directional long-short term memory neural network Bi-LSTM model, and the second semantic vector obtaining module is configured to:
calculating to obtain an output value of a forgetting gate through a sigmoid function based on the first semantic vector input in the LSTM unit in the Bi-LSTM model at the current moment and the second semantic vector output by the hidden layer of the LSTM unit at the last moment;
calculating to obtain an output value of an input gate through a sigmoid function based on the first semantic vector input in the LSTM unit in the Bi-LSTM model at the current moment and a second semantic vector output by a hidden layer of the LSTM unit at the last moment;
calculating a value of a temporary LSTM unit cell state through a tanh function based on the first semantic vector input in the LSTM unit in the Bi-LSTM model at the current moment and the second semantic vector output by the hidden layer of the LSTM unit at the last moment;
calculating to obtain the value of the LSTM unit cell state at the current moment based on the output value of the forgetting gate, the output value of the input gate, the value of the temporary LSTM unit cell state and the value of the LSTM unit cell state at the last moment;
calculating to obtain an output value of an output gate through a sigmoid function based on the first semantic vector input in the LSTM unit in the Bi-LSTM model at the current moment and a second semantic vector output by a hidden layer of the LSTM unit at the last moment;
obtaining an output vector of a hidden state at the current moment according to the output value of the output gate and the value of the cell state of the LSTM unit at the current moment;
obtaining an output vector of a forward LSTM network in the Bi-LSTM model according to the output vector of the hidden state at each moment;
obtaining an output vector of a backward LSTM network in the Bi-LSTM model according to the output vector of the hidden state at each moment;
and splicing the output vector of the forward LSTM network and the output vector of the backward LSTM network to obtain a second semantic vector output by the Bi-LSTM model.
Optionally, the intention prediction model is a convolutional neural network model, the convolutional neural network model includes a convolutional layer, a max pooling layer, and a full connection layer, and the intention category determining module is configured to:
performing dot product operation on the second semantic vector and a convolution kernel through the convolution layer, and extracting a local maximum value through the maximum pooling layer to perform high-dimensional semantic feature extraction to obtain a semantic feature vector;
and calculating and obtaining the probability that the semantic feature vector belongs to each intention category through the full connection layer, and obtaining the intention category corresponding to the input text based on the probability.
Optionally, the apparatus further comprises:
the reply module is used for identifying the input text and searching a problem text matched with the input text from a database corresponding to the intention category; and acquiring a corresponding reply text from the database according to the question text, and taking the reply text as a reply to the input text.
Optionally, the apparatus further comprises:
and the interface packaging module is used for packaging an intention recognition model formed by the language model, the feature extraction model and the intention prediction model into an application service interface so as to call the intention recognition model through the application service interface to perform intention recognition on the input text.
In a third aspect, an embodiment of the present application provides an electronic device, including a processor and a memory, where the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the steps in the method as provided in the first aspect are executed.
In a fourth aspect, embodiments of the present application provide a readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, performs the steps in the method as provided in the first aspect.
Additional features and advantages of the present application will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by the practice of the embodiments of the present application. The objectives and other advantages of the application may be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are required to be used in the embodiments of the present application will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and that those skilled in the art can also obtain other related drawings based on the drawings without inventive efforts.
Fig. 1 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure;
FIG. 2 is a flow chart of an intent recognition method provided by an embodiment of the present application;
fig. 3 is a schematic structural diagram of a BERT model according to an embodiment of the present application;
FIG. 4 is a schematic structural diagram of a Bi-LSTM model provided in an embodiment of the present application;
fig. 5 is a block diagram of an intention identifying apparatus according to an embodiment of the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. The components of the embodiments of the present application, generally described and illustrated in the figures herein, can be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present application, presented in the accompanying drawings, is not intended to limit the scope of the claimed application, but is merely representative of selected embodiments of the application. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present application without making any creative effort, shall fall within the protection scope of the present application.
The embodiment of the application provides an intention identification method, which is characterized in that semantic features of an input text are extracted through a language model, a feature extraction model and an intention prediction model, so that more context information and deeper semantic relation in the input text can be obtained, more effective information is obtained, and the accuracy of intention prediction of the input text can be effectively improved.
The intention recognition method provided by the embodiment of the application is described in detail below with reference to the accompanying drawings.
Referring to fig. 1, fig. 1 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure, where the electronic device may include: at least one processor 110, such as a CPU, at least one communication interface 120, at least one memory 130, and at least one communication bus 140. Wherein the communication bus 140 is used for realizing direct connection communication of these components. The communication interface 120 of the device in the embodiment of the present application is used for performing signaling or data communication with other node devices. The memory 130 may be a high-speed RAM memory or a non-volatile memory (e.g., at least one disk memory). Memory 130 may optionally be at least one memory device located remotely from the aforementioned processor. The memory 130 stores computer readable instructions which, when executed by the processor 110, cause the electronic device to perform the method processes of fig. 2 described below. For example, the memory 130 may be configured to store data such as an input text, a semantic vector, a language model, a feature extraction model, and an intention prediction model, when the processor 110 performs intention recognition, the processor 110 may obtain the input text from the memory 130, then obtain the language model, perform word-embedded vector conversion on the input text through the language model to obtain a first semantic vector, where the first semantic vector may also be temporarily stored in the memory 130, and the processor 110 may obtain the feature extraction model again to perform semantic feature extraction on the first semantic vector to obtain a second semantic vector, where the second semantic vector may also be temporarily stored in the memory 130, and then obtain the intention type prediction model for the intention type prediction, so as to obtain the intention type to which the input text belongs.
Referring to fig. 2, fig. 2 is a flowchart of an intention recognition method according to an embodiment of the present application, where the method is used for performing intention recognition on an input text, and includes the following steps:
step S110: and acquiring an input text to be recognized.
The embodiment of the application can be applied to a plurality of different scenes, such as scenes of intelligent customer service questions and answers, robot interaction, self-service and the like in various fields, and the embodiment of the application does not specifically limit the scenes. In a robot interaction scene, the electronic equipment can be a robot, and in an intelligent customer service question and answer and self-service scene, the electronic equipment can be a user terminal.
In order to facilitate understanding of the embodiments of the present application, the embodiments of the present application are described by taking an intelligent customer service question and answer as an example, in such a scenario, a user may input information that the user wants to interact or inquire in an electronic device, and then recognize an intention of the user through the electronic device, and then may feed back corresponding information to the user based on the intention of the user, so as to complete a question and answer service.
In this case, the input text may refer to text information directly input by the user in the electronic device or text information converted from voice information, that is, for convenience of subsequent processing, when the user inputs voice information, the electronic device may further recognize the voice information, convert the voice information into corresponding text information, and perform subsequent processing.
Step S120: and performing word embedding vector conversion on the input text through a language model to obtain a first semantic vector corresponding to the input text.
The language model is used for performing word embedding vector conversion on the input text, and when the word embedding vector conversion is performed through the language model, each word in the input text is converted into vector representation with fixed length, so that mathematical processing is facilitated.
The word embedding is to convert sparse vector representation of words into dense and continuous vector space, and can identify similarity and reference relationship between words, so that a first semantic vector can be obtained after word embedding vector conversion is carried out on an input text through a language model, and the first semantic vector represents semantic information of each word in the input text.
The language model may be a Word2Vec model, a GloVe model, a bidirectional depth Transformer from Transformer (BERT) model, or the like.
Step S130: and semantic feature extraction is carried out on the first semantic vector through a feature extraction model, and a second semantic vector corresponding to the input text is obtained.
Although the language model can extract semantic information of each word in the input text to a certain extent, in order to further explore context sequence dependency information and improve the accuracy of performing intention type recognition on the input text, the context semantic information of each word in the input text is continuously extracted, that is, semantic features are extracted from the first semantic vector through the feature extraction model, so that a second semantic vector corresponding to the input text can be obtained, and the second semantic vector can represent the context semantic information of each word in the input text.
The feature extraction model is a model that can be used for semantic feature extraction, for example, the feature extraction model may be a Long Short-Term Memory network (LSTM) model or a Bidirectional Long Short-Term Memory network (Bi-LSTM) model.
Step S140: and determining an intention category corresponding to the input text based on the second semantic vector through an intention prediction model.
After obtaining the depth semantic information of the input text, an intention category corresponding to the input text can be determined based on the second semantic vector through an intention prediction model.
The intention category can be classified according to actual situations, such as negative intention, positive intention, query intention and other intention categories.
The intention prediction model may specifically be a classifier model which may classify the input text by intention categories based on the second semantic vector, outputting a probability that the input text belongs to each intention category, thereby determining a true intention category of the input text based on the probabilities.
For example, the intention prediction model may output a probability that the input text belongs to a negative intention, a probability that a positive intention is input, and a probability that the input text belongs to a query intention, and after determining the probabilities of the respective intention categories, the intention category having the highest probability may be taken as the final intention category to which the input text belongs.
In the implementation process, the word embedding vector conversion can be carried out on the input text through the language model to obtain a first semantic vector corresponding to the input text, the first semantic vector represents the semantic information of each word in the input text, then the semantic feature extraction is carried out on the first semantic vector through a feature extraction model to obtain a second semantic vector corresponding to the input text, the second semantic vector represents the context semantic information of each word in the input text, and the intention category corresponding to the input text is determined based on the second semantic vector through an intention prediction model, so that, the scheme can carry out semantic feature extraction on the input text in turn through the mutual cooperation of the three models, therefore, more context information and deeper semantic relation in the input text can be obtained, more effective information can be obtained, and the accuracy of the intention prediction of the input text can be effectively improved.
In addition, the language model, the feature extraction model and the intention prediction model are trained in advance, and since the intention of the user may be different in different scenes, the above models may be trained for different scenes in order to accurately recognize the intention of the user, so that the intention of the user may be accurately recognized in different scenes. Certainly, in different fields, training sample data of the model are different, for example, for an insurance field, the training sample has more professional words, and when the model is trained, the obtained training sample may include more professional words, so that the model can accurately identify the intention type of the input text in the application process.
As an example, in order to better extract semantic information of each word in an input text, a BERT model may be used as a language model in the embodiment of the present application, and the BERT model is more accurate than a general word embedding vector generation method, and can effectively solve the problems of word ambiguity and unknown words.
That is to say, the word embedding vector conversion may be performed on the input text through the BERT model, so as to obtain a first semantic vector corresponding to the input text.
The BERT model is characterized by a transform-based bidirectional encoder, the root of which is the transform, wherein the bidirectional meaning means that the BERT model can take the information of words in front of and behind a word into consideration when processing the word, thereby acquiring the semantic meaning of the context. The BERT model aims to obtain vector representation of texts containing rich semantic information by utilizing large-scale unmarked corpus training, namely the BERT model can carry out vectorization representation on each word in the input texts so as to obtain semantic features of the input texts. The main input of the BERT model is an original Word Vector of each Word in the text, the Vector can be initialized randomly, or pre-trained by using algorithms such as Word2Vector and the like to be used as an initial value, and the output of the BERT model is a Vector representation of each Word in the text after being fused with full-text semantic information.
In the process of vector conversion of the input text by the BERT model, the BERT model can firstly convert each word in the input text into a one-dimensional vector by querying a word vector table, and then the BERT model can also obtain a text vector of the input text, wherein the value of the text vector is automatically learned by the BERT model in the training process, is used for depicting the global semantic information of the input text and is fused with the semantic information of the word. In addition, because semantic information carried by words appearing at different positions of the input text is different, the BERT model can also obtain position vectors of the words appearing at different positions of the input text in the input text, namely the BERT model adds one position vector to each word at different positions to distinguish the words.
Therefore, the BERT model can obtain the word vector, the sentence vector and the position vector of the input text, then sum the vectors, and then perform semantic analysis on the vectors, so that the vector converted from the word/word vector output by the BERT model can contain more accurate semantic information.
Referring to fig. 3, as an example, the language model includes an embedding layer and M coding layers, where M is an integer greater than or equal to 2, the embedding layer is configured to convert an input text into the word vector, the position vector, and the sentence vector, then input a word embedding vector formed by a sum of the three vectors into the coding layer, and perform semantic coding on the word embedding vector by the coding layer to obtain a corresponding coding vector.
It is understood that if the language model is a BERT model, it generally includes 12 coding layers, and of course, the number of coding layers may be reduced or increased according to actual requirements, i.e., the BERT model may include a smaller or larger number of coding layers.
In the process of training the BERT model, a masking language model (Masked LanguageModel) can be applied to randomly mask 15% of Chinese characters in a corpus, the Masked Chinese characters are predicted based on the Chinese characters which are not Masked in the sequence to obtain a word vector word embedding model, and then word vectors of an input text can be obtained through the word vector word embedding model; using paired sentences as input by applying Next Sentence Prediction (NSP), predicting whether a second sentence is the next sentence of a first sentence, carrying out depth coding on a sentence vector through a Transformer model to obtain a sentence vector word embedding model, and obtaining the sentence vector of an input text through the sentence vector word embedding model; and calculating the position of each word in the sentence through a position coding formula to obtain a position code, wherein the position coding formula is as follows:
where pos represents the position information of the Chinese character, i is used to express the coding dimension, dmodelIs the maximum sequence length of the model, d in this examplemodelMay be 512, then i is 0 to 255.
The encoding layer is a transform encoder and is used for extracting semantic information. The embedding layer carries out word embedding vector conversion on an input text to obtain word embedding vectors, then the word embedding vectors are input into the first coding layer, the word embedding vectors are subjected to semantic coding by the first coding layer to obtain first coding vectors, and the first coding vectors comprise coding vectors corresponding to each word. And each coding layer carries out semantic coding on the coding vector, namely, i is sequentially selected from 2 to M, the ith coding vector is continuously coded through the ith coding layer to obtain the ith coding vector, and the Mth coding vector is obtained until i is selected from M, and is the first semantic vector corresponding to the input text.
For example, after the first coding vector is obtained, the first coding vector is input into the 2 nd coding layer to continue semantic coding, so as to obtain the second coding vector, then the second coding vector is input into the 3 rd coding layer to continue semantic coding, so as to sequentially code the coding vectors, and finally the M-th coding vector is obtained.
Wherein each coding layer comprises a self-attention layer, a feedforward neural network layer and a residual layer. The self-attention layers can well capture the word-word reference relation in the input text, and the self-attention layers are connected by the feedforward neural network layer and the residual layer for summation normalization so as to improve the model fitting efficiency and prevent the problem of gradient disappearance.
It can be understood that each coding layer has the same network structure, the input text is input to the attention layer after being subjected to vector conversion by the embedded layer, the ith output vector can be obtained by performing attention mechanism calculation on the ith-1 coding vector through the attention layer, that is, the output vector fusing attention head information through the attention layer reflects the degree of influence of other words in the input text on the current word, one attention head corresponds to one output vector, if 12 such attention heads exist, the 12 attention heads are spliced into a total output vector, and each word corresponds to one row vector in the output vector.
Wherein, the ith output vector is obtained by adopting attention mechanism calculation to the ith-1 encoding vector through the self-attention layer, and the calculation process is as follows:
wherein, Attention is a self-Attention layer calculation formula, Q is a query vector obtained by multiplying a weight obtained in the process of training the model by an input code vector, K is a key vector obtained by multiplying a weight obtained in the process of training the model by an input code vector, and V isA value vector obtained by multiplying the weight obtained in the training process of the model by the input code vector, dkIs an empirical constant. The core idea is that the correlation of each word in the input text to all words in the input text is calculated, then the correlation between the words is considered to reflect the correlation and the importance degree between different words in the input text to a certain extent, the correlation can be reused to adjust the importance degree of each word, so that a new expression vector of each word can be obtained, the new expression vector not only contains the semantics of the word, but also contains the semantics of other words referred to or contained by the word, and therefore more comprehensive semantic information of each word in the input text can be obtained.
In order to extract more characteristic information in the original vector, a residual error layer is arranged between the self-attention layer and the feedforward neural network layer, and the ith normalized vector is obtained by summing and normalizing the ith encoding vector and the ith output vector matrix through the residual error layer.
The feedforward neural network layer carries out semantic feature extraction on the ith normalized vector to obtain an ith semantic feature vector, namely the feedforward neural network layer can multiply the ith normalized vector by the weight of a hidden layer of the ith normalized vector, add the ith normalized vector by a bias term and then process the sum through an activation function to obtain the ith semantic feature vector, and the calculation formula is as follows: and S is Relu (xb + w), wherein S is the ith semantic feature line, Relu is the activation function, x is the ith normalized vector, b is the weight, and w is the bias term.
And summing and normalizing the ith semantic feature vector and the ith normalized vector through the residual error layer to obtain an ith encoding vector, and outputting the ith encoding vector to the next encoding layer to continue encoding according to the mode.
In the implementation process, the language model can use each coding layer to code the vector, that is, the parameters of each coding layer can be used to extract the characteristics of the input text, so that the reference characteristics among the words can be effectively extracted.
In the process, the semantic information of each word in the input text can be extracted through the BERT model, but in order to integrate the semantic information of the context, a feature extraction model is further adopted to continuously extract the features of the output vector of the language model. In the embodiment of the application, the feature extraction model can adopt a Bi-LSTM model, and the Bi-LSTM model can better process the sequence relation between word context sequences, so that the feature extraction model can extract the semantic information of each word in the input text at a deeper level in the context.
Therefore, after the first semantic vector corresponding to the input text is obtained through the language model, in order to further extract context semantic information of each word in the input text, semantic feature extraction can be further continuously performed on the input text through the feature extraction model. As shown in fig. 4, fig. 4 is a schematic structural diagram of a Bi-LSTM model, where the Bi-LSTM model includes two independent long-term and short-term memory networks, i.e., a forward LSTM network and a backward LSTM network, and the input sequence order of the two independent long-term and short-term memory networks is opposite, so that, for an input text sequence, two implicit state sequence outputs can be obtained, and then the Bi-LSTM model concatenates (Concat) a vector set of the two implicit state sequence outputs according to a word to obtain a vector, and then outputs the vector.
The Bi-LSTM model is actually a combination of forward LSTM and backward LSTM, so the internal data processing process is similar to that of the LSTM model, namely, the Bi-LSTM model is used for extracting semantic features of the first semantic vector, and the process of obtaining the second semantic vector corresponding to the input text is as follows:
and calculating to obtain an output value of the forgetting gate by a sigmod function based on the first semantic vector input in the LSTM unit in the Bi-LSTM model at the current moment and the second semantic vector output by the hidden layer of the LSTM unit at the last moment.
Wherein, the above process can adopt the formula ft＝σ(Wf·[ht-1,xt]+bf) Is represented by ftI.e. the output value of the forgetting gate, xtA first semantic vector, h, which is the input of the LSTM unit at the current timet-1A second semantic vector, W, output for the hidden layer of the LSTM cell at the previous instantfWeight matrix for forgetting the state of the gate cell, bfTo forget the deviation of the state of the door unitAnd (5) setting a vector.
And then calculating to obtain an output value of the input gate through a sigmoid function based on the first semantic vector input in the LSTM unit in the current time Bi-LSTM model and the second semantic vector output by the hidden layer of the LSTM unit at the previous time.
Wherein the process may employ formula it＝σ(Wi·[ht-1,xt]+bi) Is represented by itIs the output value of the input gate, sigma is sigmoid activation function, WiWeight matrix being the state of the input gate cell, biIs the offset vector of the input gate cell state.
And calculating the value of the cell state of the temporary LSTM unit by using a tanh function based on the first semantic vector input in the LSTM unit in the Bi-LSTM model at the current moment and the second semantic vector output by the hidden layer of the LSTM unit at the last moment.
Wherein the process may employ a formula
It is shown that,
i.e. the value of the cell state of the temporary LSTM cell, and tanh is a hyperbolic tangent function, WcWeight matrix being the state of the Bi-LSTM cell, bcIs the bias vector for the LSTM cell state.
And calculating to obtain the value of the LSTM unit cell state at the current moment based on the output value of the forgetting gate, the output value of the input gate, the value of the temporary LSTM unit cell state and the value of the LSTM unit cell state at the last moment.
Wherein the process may employ a formula
Is represented by CtIs the value of the cell state of the LSTM unit at the current moment, Ct-1Is the value of the cellular state of the LSTM unit at the last time.
And then calculating to obtain an output value of the output gate through a sigmoid function based on the first semantic vector input in the LSTM unit in the current time Bi-LSTM model and the second semantic vector output by the hidden layer of the LSTM unit at the previous time.
Wherein the process may employ the formula ot＝σ(WO·[ht-1,xt]+bo) Is represented by itIs the output value of the output gate, sigma is sigmoid activation function, WOWeight matrix being the state of the output gate cell, boIs an offset vector of the output gate cell state.
And obtaining an output vector of the hidden state at the current moment according to the output value of the output gate and the value of the cell state of the LSTM unit at the current moment.
Wherein, the process can be expressed by the following formula:
t t th＝o*tanh(C)
otfor outputting the output of the gate unit, htThe output vector of the hidden state at the current moment.
And obtaining the output vector of the forward LSTM network in the Bi-LSTM model according to the output vector of the hidden state at each moment.
For the forward LSTM network, the output vector of the hidden state at each time can be obtained according to the above process, and thus the hidden state sequence can be obtained as the output vector of the forward LSTM network
And obtaining an output vector of a backward LSTM network in the Bi-LSTM model according to the output vector of the hidden state at each moment.
For the backward LSTM network, the output vector of the hidden state at each moment can be obtained according to the above process, and thus the hidden state sequence can be obtained as the output vector of the backward LSTM network
And splicing the output vector of the forward LSTM network and the output vector of the backward LSTM network to obtain a second semantic vector output by the Bi-LSTM model.
Wherein the second semantic vector may be represented as
In the process, the retention degree of the sequence information input before forgetting to control the gate in the LSTM network in the Bi-LSTM model and the influence degree of the current sequence unit can be controlled by the input gate, the two LSTM networks can fully consider the forward sequence information and the reverse sequence information, the context characteristics are mined, two implicit state sequence vector sets are generated, and then the two vector sets are spliced according to words to obtain the final second semantic vector.
Therefore, the second semantic vector can be obtained according to the method, and the obtained second semantic vector contains the semantic features of each word. For example, the first semantic vector sequence set of inputs is denoted as x ═ (x)1,x2,x3,x4,x5) After each word vector is input into a forward LSTM network in the Bi-LSTM model, five implicit state vector sets are respectively obtained
After each word vector is input into a backward LSTM network in the Bi-LSTM model, five implicit state vector sets are respectively obtained
Then the output of the forward LSTM network and the output of the backward LSTM network are spliced to obtain the output of the Bi-LSTM model, namely a second semantic vector set
Is marked as
Thus, a second semantic vector output by the Bi-LSTM model can be obtained based on the above algorithm.
In the implementation process, the Bi-LSTM model can be used for better processing the relation among the word sequences, so that the semantic information of each word in the input text, which is deeper in context, can be extracted.
And after a second semantic vector containing rich semantic information is obtained, identifying the intention type of the input text based on the second semantic vector through an intention prediction model. In the embodiment of the application, the intention prediction model may be a convolutional neural network model, the convolutional neural network model may well extract helpful characteristics for classification, then perform pooling operation on the extracted characteristics to obtain a final classification characteristic representation, and then normalize the class probability by using a full connection layer, so that an intention class to which the input text finally belongs may be obtained.
As an example, the convolutional neural network model includes a convolutional layer, a maximum pooling layer and a full-link layer, the second semantic vector and the convolutional kernel are subjected to dot product operation through the convolutional layer, a local maximum value is extracted through the maximum pooling layer to perform high-dimensional semantic abstract feature extraction, a semantic feature vector is obtained, then, the probability that the semantic feature vector belongs to each intention category is obtained through full-link layer calculation, and the intention category corresponding to the input text is obtained based on the probability.
For example, assuming that the second semantic vector is X ═ X1, X2, X3, X4, X5, X6], the trained convolution kernel is F ═ F1, F2, F3], then the convolution operation procedure is:
Y1＝x1*f1+x2*f2+x3*f3；
Y2＝x2*f1+x3*f2+x4*f3；
Y3＝x3*f1+x4*f2+x5*f3；
Y4＝x4*f1+x5*f2+x6*f3；
and the convolution output is Y ═ Y1, Y2, Y3 and Y4, and the convolution output is subjected to 1 × 2 maximum pooling operation, so that the final output is Y ═ max (Y1, Y2) and max (Y3 and Y4), and then the final output enters the full-link layer and is normalized by softmax to obtain the final probability distribution.
Specifically, redundant information can be removed through the convolutional layer, the most important features can be extracted, overfitting can be prevented to a certain extent, and the generalization capability of the prediction result is improved.
In addition, in the field of intelligent customer service question answering, after the intention category corresponding to the input text is obtained, the reply text to the input text can be obtained, and then the reply text can be fed back to the user as the reply to the input text.
The process of acquiring the reply text may be as follows: in order to quickly obtain the corresponding reply text, a corresponding database can be established for each intention category, so that after the intention category is determined, the input text can be identified, the problem text matched with the input text is searched in the database corresponding to the intention category, then the corresponding reply text is obtained from the database according to the problem text, and the reply text is used as the reply to the input text.
For example, if the intention category is a question, the input text may be identified, and then a question text matching the input text is searched for from a database corresponding to the question category, where the question text may be the same text as the input text or a text with a higher similarity to the input text, for example, each character in the input text may be subjected to similarity matching with each character in each question text, so that a question text satisfying a certain similarity may be obtained. And when a plurality of reply texts are available, the reply texts can be fed back to the user together as replies to the input text, or one of the reply texts can be arbitrarily selected as the reply to the input text.
The database is preset with various question texts corresponding to different intention categories in corresponding fields and corresponding reply texts thereof, and then the answer texts are fed back to the user, the feedback mode can be a voice broadcast answer mode or a character answer mode, if the corresponding question texts are not found, default information can be fed back to the user, if the answer information is not returned or unrecognizable prompt information is returned, and the like, so that the input of the user can be responded in time, and the user experience is improved.
In addition, in order to facilitate other users to directly adopt the intention recognition method provided by the embodiment of the application, an intention recognition model formed by a language model, a feature extraction model and an intention prediction model can be packaged as an application service interface, so that the intention recognition model is called by the application service interface to perform intention recognition on the input text.
It can be understood that the application service interface may be an http interface, that is, the intention recognition model may form an http link address, and when a user needs to call the intention recognition model to perform intention recognition, the corresponding http link address may be directly input in a web page, and an input text is transferred in a GET or POST manner, so that the obtained input text may be input into the intention recognition model to be recognized, so as to obtain a corresponding intention category.
In the implementation process, the intention recognition model is packaged as an application service interface, so that other users can use the intention recognition model directly through the application service interface.
Referring to fig. 5, fig. 5 is a structural block diagram of an intention identifying apparatus 200 according to an embodiment of the present application, where the apparatus 200 may be a module, a program segment, or code on an electronic device. It should be understood that the apparatus 200 corresponds to the above-mentioned embodiment of the method of fig. 2, and can perform various steps related to the embodiment of the method of fig. 2, and the specific functions of the apparatus 200 can be referred to the above description, and the detailed description is appropriately omitted here to avoid redundancy.
Optionally, the apparatus 200 comprises:
a text obtaining module 210, configured to obtain an input text to be identified;
a first semantic vector obtaining module 220, configured to perform word-embedded vector conversion on the input text through a language model, and obtain a first semantic vector corresponding to the input text, where the first semantic vector represents semantic information of each word in the input text;
a second semantic vector obtaining module 230, configured to perform semantic feature extraction on the first semantic vector through a feature extraction model, to obtain a second semantic vector corresponding to the input text, where the second semantic vector represents context semantic information of each word in the input text;
an intention category determining module 240, configured to determine, by an intention prediction model, an intention category corresponding to the input text based on the second semantic vector.
Optionally, the language model includes an embedded layer and M coding layers, where M is an integer greater than or equal to 2, and the first semantic vector obtaining module 220 is configured to:
performing word embedding vector conversion on the input text through the embedding layer to obtain a word embedding vector corresponding to the input text;
performing word embedding vector conversion on the input text through the embedding layer to obtain a word vector, a sentence vector and a position vector corresponding to the input text, and adding the word vector, the sentence vector and the position vector to obtain a word embedding vector;
semantic coding is carried out on the word embedding vector through a first coding layer, so that a first coding vector corresponding to the input text is obtained, wherein the first coding vector comprises a coding vector corresponding to each word;
and sequentially taking i as 2 to M, continuing semantic coding on the i-1 coding vector through the ith coding layer to obtain the ith coding vector, and obtaining the Mth coding vector when i is taken as M, wherein the Mth coding vector is the first semantic vector corresponding to the input text.
Optionally, each coding layer includes a self-attention layer, a feedforward neural network layer, and a residual layer, and the first semantic vector obtaining module 220 is configured to:
performing attention mechanism calculation on the ith-1 encoding vector through the self-attention layer to obtain an ith output vector;
summing and normalizing the ith-1 encoding vector and the ith output vector through the residual error layer to obtain an ith normalized vector;
semantic feature extraction is carried out on the ith normalized vector through the feedforward neural network layer, and an ith semantic feature vector is obtained;
and summing and normalizing the ith semantic feature vector and the ith normalized vector through the residual error layer to obtain an ith coding vector.
Optionally, the feature extraction model is a Bi-directional long-short term memory neural network Bi-LSTM model, and the second semantic vector obtaining module 230 is configured to:
calculating to obtain an output value of a forgetting gate through a sigmoid function based on the first semantic vector input in the LSTM unit in the Bi-LSTM model at the current moment and the second semantic vector output by the hidden layer of the LSTM unit at the last moment;
calculating to obtain an output value of an input gate through a sigmoid function based on the first semantic vector input in the LSTM unit in the Bi-LSTM model at the current moment and a second semantic vector output by a hidden layer of the LSTM unit at the last moment;
calculating a value of a temporary LSTM unit cell state through a tanh function based on the first semantic vector input in the LSTM unit in the Bi-LSTM model at the current moment and the second semantic vector output by the hidden layer of the LSTM unit at the last moment;
calculating to obtain the value of the LSTM unit cell state at the current moment based on the output value of the forgetting gate, the output value of the input gate, the value of the temporary LSTM unit cell state and the value of the LSTM unit cell state at the last moment;
calculating to obtain an output value of an output gate through a sigmoid function based on the first semantic vector input in the LSTM unit in the Bi-LSTM model at the current moment and a second semantic vector output by a hidden layer of the LSTM unit at the last moment;
obtaining an output vector of a hidden state at the current moment according to the output value of the output gate and the value of the cell state of the Bi-LSTM unit at the current moment;
obtaining an output vector of a forward LSTM network in the Bi-LSTM model according to the output vector of the hidden state at each moment;
obtaining an output vector of a backward LSTM network in the Bi-LSTM model according to the output vector of the hidden state at each moment;
and splicing the output vector of the forward LSTM network and the output vector of the backward LSTM network to obtain a second semantic vector output by the Bi-LSTM model.
Optionally, the intention prediction model is a convolutional neural network model, the convolutional neural network model includes a convolutional layer, a max pooling layer, and a full connection layer, and the intention category determining module 240 is configured to:
performing dot product operation on the second semantic vector and a convolution kernel through the convolution layer, and extracting a local maximum value through the maximum pooling layer to perform high-dimensional semantic feature extraction to obtain a semantic feature vector;
and calculating and obtaining the probability that the semantic feature vector belongs to each intention category through the full connection layer, and obtaining the intention category corresponding to the input text based on the probability.
Optionally, the apparatus 200 further comprises:
the reply module is used for identifying the input text and searching a problem text matched with the input text from a database corresponding to the intention category; and acquiring a corresponding reply text from the database according to the question text, and taking the reply text as a reply to the input text.
Optionally, the apparatus 200 further comprises:
and the interface packaging module is used for packaging an intention recognition model formed by the language model, the feature extraction model and the intention prediction model into an application service interface so as to call the intention recognition model through the application service interface to perform intention recognition on the input text.
The embodiment of the present application provides a readable storage medium, and when being executed by a processor, the computer program performs the method process performed by the electronic device in the method embodiment shown in fig. 2.
The present embodiments disclose a computer program product comprising a computer program stored on a non-transitory computer readable storage medium, the computer program comprising program instructions which, when executed by a computer, enable the computer to perform the methods provided by the above-described method embodiments, for example, comprising: acquiring an input text to be identified; performing word embedding vector conversion on the input text through a language model to obtain a first semantic vector corresponding to the input text, wherein the first semantic vector represents semantic information of each word in the input text; semantic feature extraction is carried out on the first semantic vector through a feature extraction model, and a second semantic vector corresponding to the input text is obtained, wherein the second semantic vector represents context semantic information of each word in the input text; and determining an intention category corresponding to the input text based on the second semantic vector through an intention prediction model.
In summary, the embodiments of the present application provide an intention recognition method, an intention recognition device, an electronic device, and a readable storage medium, in which the method performs semantic feature extraction on an input text in sequence through mutual cooperation of three models, so that the method can obtain more context information and deeper semantic relationships in the input text, obtain more effective information, and thus can effectively improve accuracy of intention prediction on the input text.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one logical division, and there may be other divisions when actually implemented, and for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of devices or units through some communication interfaces, and may be in an electrical, mechanical or other form.
In addition, units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
Furthermore, the functional modules in the embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.
In this document, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions.
The above description is only an example of the present application and is not intended to limit the scope of the present application, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application.
Claims (10)
1. An intention recognition method for performing intention recognition on an input text, the method comprising:
acquiring an input text to be identified;
performing word embedding vector conversion on the input text through a language model to obtain a first semantic vector corresponding to the input text, wherein the first semantic vector represents semantic information of each word in the input text;
semantic feature extraction is carried out on the first semantic vector through a feature extraction model, and a second semantic vector corresponding to the input text is obtained, wherein the second semantic vector represents context semantic information of each word in the input text;
and determining an intention category corresponding to the input text based on the second semantic vector through an intention prediction model.
2. The method according to claim 1, wherein the language model includes an embedding layer and M coding layers, M is an integer greater than or equal to 2, and the obtaining a first semantic vector corresponding to the input text by performing word embedding vector conversion on the input text through the language model includes:
performing word embedding vector conversion on the input text through the embedding layer to obtain a word vector, a sentence vector and a position vector corresponding to the input text, and adding the word vector, the sentence vector and the position vector to obtain a word embedding vector;
semantic coding is carried out on the word embedding vector through a first coding layer, so that a first coding vector corresponding to the input text is obtained, wherein the first coding vector comprises a coding vector corresponding to each word;
and sequentially taking i as 2 to M, continuing semantic coding on the i-1 coding vector through the ith coding layer to obtain the ith coding vector, and obtaining the Mth coding vector when i is taken as M, wherein the Mth coding vector is the first semantic vector corresponding to the input text.
3. The method of claim 2, wherein each coding layer comprises a self-attention layer, a feedforward neural network layer and a residual layer, and the semantic coding of the i-1 th coding vector by the i-th encoder is continued to obtain the i-th coding vector, and the method comprises the following steps:
performing attention mechanism calculation on the ith-1 encoding vector through the self-attention layer to obtain an ith output vector;
summing and normalizing the ith-1 encoding vector and the ith output vector through the residual error layer to obtain an ith normalized vector;
semantic feature extraction is carried out on the ith normalized vector through the feedforward neural network layer, and an ith semantic feature vector is obtained;
and summing and normalizing the ith semantic feature vector and the ith normalized vector through the residual error layer to obtain an ith coding vector.
4. The method of claim 1, wherein the feature extraction model is a Bi-directional long-short term memory neural network (Bi-LSTM) model, and the semantic feature extraction performed on the first semantic vector by the feature extraction model to obtain a second semantic vector corresponding to the input text comprises:
calculating to obtain an output value of a forgetting gate through a sigmoid function based on the first semantic vector input in the LSTM unit in the Bi-LSTM model at the current moment and the second semantic vector output by the hidden layer of the LSTM unit at the last moment;
calculating to obtain an output value of an input gate through a sigmoid function based on the first semantic vector input in the LSTM unit in the Bi-LSTM model at the current moment and a second semantic vector output by a hidden layer of the LSTM unit at the last moment;
calculating a value of a temporary LSTM unit cell state through a tanh function based on the first semantic vector input in the LSTM unit in the Bi-LSTM model at the current moment and the second semantic vector output by the hidden layer of the LSTM unit at the last moment;
calculating to obtain the value of the LSTM unit cell state at the current moment based on the output value of the forgetting gate, the output value of the input gate, the value of the temporary LSTM unit cell state and the value of the LSTM unit cell state at the last moment;
calculating to obtain an output value of an output gate through a sigmoid function based on the first semantic vector input in the LSTM unit in the Bi-LSTM model at the current moment and a second semantic vector output by a hidden layer of the LSTM unit at the last moment;
obtaining an output vector of a hidden state at the current moment according to the output value of the output gate and the value of the cell state of the LSTM unit at the current moment;
obtaining an output vector of a forward LSTM network in the Bi-LSTM model according to the output vector of the hidden state at each moment;
obtaining an output vector of a backward LSTM network in the Bi-LSTM model according to the output vector of the hidden state at each moment;
and splicing the output vector of the forward LSTM network and the output vector of the backward LSTM network to obtain a second semantic vector output by the Bi-LSTM model.
5. The method of claim 1, wherein the intent prediction model is a convolutional neural network model, the convolutional neural network model comprises a convolutional layer, a max-pooling layer, and a full-link layer, and the determining, by the intent prediction model, the intent category corresponding to the input text based on the second semantic vector comprises:
performing dot product operation on the second semantic vector and a convolution kernel through the convolution layer, and extracting a local maximum value through the maximum pooling layer to perform high-dimensional semantic feature extraction to obtain a semantic feature vector;
and calculating and obtaining the probability that the semantic feature vector belongs to each intention category through the full connection layer, and obtaining the intention category corresponding to the input text based on the probability.
6. The method according to any one of claims 1-5, further comprising:
identifying the input text, and searching a problem text matched with the input text from a database corresponding to the intention category;
and acquiring a corresponding reply text from the database according to the question text, and taking the reply text as a reply to the input text.
7. The method according to any one of claims 1-5, further comprising:
and packaging an intention recognition model formed by the language model, the feature extraction model and the intention prediction model into an application service interface, and calling the intention recognition model through the application service interface to perform intention recognition on the input text.
8. An intention recognition apparatus for performing intention recognition on an input text, the apparatus comprising:
the text acquisition module is used for acquiring an input text to be identified;
the first semantic vector acquisition module is used for performing word embedding vector conversion on the input text through a language model to acquire a first semantic vector corresponding to the input text, wherein the first semantic vector represents semantic information of each word in the input text;
the second semantic vector acquisition module is used for extracting semantic features of the first semantic vector through a feature extraction model to obtain a second semantic vector corresponding to the input text, wherein the second semantic vector represents context semantic information of each word in the input text;
and the intention category determining module is used for determining an intention category corresponding to the input text based on the second semantic vector through an intention prediction model.
9. An electronic device comprising a processor and a memory, the memory storing computer readable instructions that, when executed by the processor, perform the method of any of claims 1-7.
10. A readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the method according to any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911232049.9A CN110928997A (en) | 2019-12-04 | 2019-12-04 | Intention recognition method and device, electronic equipment and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911232049.9A CN110928997A (en) | 2019-12-04 | 2019-12-04 | Intention recognition method and device, electronic equipment and readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110928997A true CN110928997A (en) | 2020-03-27 |
Family
ID=69857105
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911232049.9A Pending CN110928997A (en) | 2019-12-04 | 2019-12-04 | Intention recognition method and device, electronic equipment and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110928997A (en) |
Cited By (42)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111460302A (en) * | 2020-03-31 | 2020-07-28 | 拉扎斯网络科技（上海）有限公司 | Data processing method and device, electronic equipment and computer readable storage medium |
| CN111462735A (en) * | 2020-04-10 | 2020-07-28 | 网易（杭州）网络有限公司 | Voice detection method and device, electronic equipment and storage medium |
| CN111460303A (en) * | 2020-03-31 | 2020-07-28 | 拉扎斯网络科技（上海）有限公司 | Data processing method and device, electronic equipment and computer readable storage medium |
| CN111462752A (en) * | 2020-04-01 | 2020-07-28 | 北京思特奇信息技术股份有限公司 | Client intention identification method based on attention mechanism, feature embedding and BI-L STM |
| CN111553363A (en) * | 2020-04-20 | 2020-08-18 | 北京易道博识科技有限公司 | End-to-end seal identification method and system |
| CN111625641A (en) * | 2020-07-30 | 2020-09-04 | 浙江大学 | Dialog intention recognition method and system based on multi-dimensional semantic interaction representation model |
| CN111709223A (en) * | 2020-06-02 | 2020-09-25 | 上海硬通网络科技有限公司 | Method and device for generating sentence vector based on bert and electronic equipment |
| CN111737458A (en) * | 2020-05-21 | 2020-10-02 | 平安国际智慧城市科技股份有限公司 | Intention identification method, device and equipment based on attention mechanism and storage medium |
| CN111813896A (en) * | 2020-07-13 | 2020-10-23 | 重庆紫光华山智安科技有限公司 | Text triple relation identification method and device, training method and electronic equipment |
| CN111858848A (en) * | 2020-05-22 | 2020-10-30 | 深圳创新奇智科技有限公司 | Semantic classification method and device, electronic equipment and storage medium |
| CN111931513A (en) * | 2020-07-08 | 2020-11-13 | 泰康保险集团股份有限公司 | Text intention identification method and device |
| CN112086086A (en) * | 2020-10-22 | 2020-12-15 | 平安科技（深圳）有限公司 | Speech synthesis method, device, equipment and computer readable storage medium |
| CN112182231A (en) * | 2020-12-01 | 2021-01-05 | 佰聆数据股份有限公司 | Text processing method, system and storage medium based on sentence vector pre-training model |
| CN112256864A (en) * | 2020-09-23 | 2021-01-22 | 北京捷通华声科技股份有限公司 | Multi-intention recognition method and device, electronic equipment and readable storage medium |
| CN112257432A (en) * | 2020-11-02 | 2021-01-22 | 北京淇瑀信息科技有限公司 | Self-adaptive intention identification method and device and electronic equipment |
| CN112446219A (en) * | 2020-12-11 | 2021-03-05 | 四川长虹电器股份有限公司 | Chinese request text intention analysis method |
| CN112463922A (en) * | 2020-11-25 | 2021-03-09 | 中国测绘科学研究院 | Risk user identification method and storage medium |
| CN112686023A (en) * | 2020-12-29 | 2021-04-20 | 平安普惠企业管理有限公司 | Text data processing method and device, electronic equipment and storage medium |
| CN112699213A (en) * | 2020-12-23 | 2021-04-23 | 平安普惠企业管理有限公司 | Speech intention recognition method and device, computer equipment and storage medium |
| CN112699686A (en) * | 2021-01-05 | 2021-04-23 | 浙江诺诺网络科技有限公司 | Semantic understanding method, device, equipment and medium based on task type dialog system |
| CN112765959A (en) * | 2020-12-31 | 2021-05-07 | 康佳集团股份有限公司 | Intention recognition method, device, equipment and computer readable storage medium |
| CN112765328A (en) * | 2021-01-28 | 2021-05-07 | 珠海格力电器股份有限公司 | Text similarity determination method, system, storage medium and equipment |
| CN112818086A (en) * | 2021-02-04 | 2021-05-18 | 上海畅圣计算机科技有限公司 | Multi-label classification method for acquiring client intention label by robot |
| CN112883221A (en) * | 2020-12-11 | 2021-06-01 | 广州橙行智动汽车科技有限公司 | Semantic information correction method and device and intelligent cabin |
| CN112949317A (en) * | 2021-02-26 | 2021-06-11 | 平安普惠企业管理有限公司 | Text semantic recognition method and device, computer equipment and storage medium |
| CN113064973A (en) * | 2021-04-12 | 2021-07-02 | 平安国际智慧城市科技股份有限公司 | Text classification method, device, equipment and storage medium |
| CN113094473A (en) * | 2021-04-30 | 2021-07-09 | 平安国际智慧城市科技股份有限公司 | Keyword weight calculation method and device, computer equipment and storage medium |
| CN113190670A (en) * | 2021-05-08 | 2021-07-30 | 重庆第二师范学院 | Information display method and system based on big data platform |
| CN113297374A (en) * | 2021-04-29 | 2021-08-24 | 军事科学院系统工程研究院网络信息研究所 | Text classification method based on BERT and word feature fusion |
| CN113343711A (en) * | 2021-06-29 | 2021-09-03 | 南方电网数字电网研究院有限公司 | Work order generation method, device, equipment and storage medium |
| WO2021208696A1 (en) * | 2020-11-19 | 2021-10-21 | 平安科技（深圳）有限公司 | User intention analysis method, apparatus, electronic device, and computer storage medium |
| CN113609768A (en) * | 2021-08-02 | 2021-11-05 | 国网浙江省电力有限公司台州供电公司 | Calculation method of line loss rate in station area based on bidirectional LSTM network |
| CN113724697A (en) * | 2021-08-27 | 2021-11-30 | 北京百度网讯科技有限公司 | Model generation method, emotion recognition method, device, equipment and storage medium |
| CN113806470A (en) * | 2020-06-15 | 2021-12-17 | 华为技术有限公司 | Semantic recognition method and device |
| CN113935330A (en) * | 2021-10-22 | 2022-01-14 | 平安科技（深圳）有限公司 | Voice-based disease early warning method, device, equipment and storage medium |
| CN114020930A (en) * | 2021-11-03 | 2022-02-08 | 宁波深擎信息科技有限公司 | User label identification method and device based on session scene |
| CN114036943A (en) * | 2021-11-15 | 2022-02-11 | 平安普惠企业管理有限公司 | Intent recognition method, apparatus, computer equipment and storage medium |
| CN114492447A (en) * | 2020-10-23 | 2022-05-13 | 华为技术有限公司 | A text data processing method and smart device |
| WO2022123695A1 (en) * | 2020-12-09 | 2022-06-16 | 日本電信電話株式会社 | Learning device, search device, learning method, search method, and program |
| CN117221839A (en) * | 2023-11-09 | 2023-12-12 | 北京中科网芯科技有限公司 | 5G signaling identification method and system thereof |
| CN117574918A (en) * | 2024-01-15 | 2024-02-20 | 青岛冠成软件有限公司 | Intelligent interaction method based on LSTM |
| CN118413923A (en) * | 2024-06-27 | 2024-07-30 | 杭州字棒棒科技有限公司 | Intelligent desk lamp learning auxiliary system and method based on AI voice interaction |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106326984A (en) * | 2016-08-09 | 2017-01-11 | 北京京东尚科信息技术有限公司 | User intention identification method and device and automatic answering system |
| US20180143964A1 (en) * | 2016-11-18 | 2018-05-24 | Microsoft Technology Licensing, Llc | Data input system using trained keypress encoder |
| CN109063035A (en) * | 2018-07-16 | 2018-12-21 | 哈尔滨工业大学 | A kind of man-machine more wheel dialogue methods towards trip field |
| CN109241255A (en) * | 2018-08-20 | 2019-01-18 | 华中师范大学 | A kind of intension recognizing method based on deep learning |
| US20190057036A1 (en) * | 2018-10-15 | 2019-02-21 | Amrita MATHURIYA | Programmable interface to in-memory cache processor |
| CN110046221A (en) * | 2019-03-01 | 2019-07-23 | 平安科技（深圳）有限公司 | A kind of machine dialogue method, device, computer equipment and storage medium |
| CN110245348A (en) * | 2019-05-17 | 2019-09-17 | 北京百度网讯科技有限公司 | A kind of intension recognizing method and system |
| CN110298403A (en) * | 2019-07-02 | 2019-10-01 | 郭刚 | The sentiment analysis method and system of enterprise dominant in a kind of financial and economic news |
-
2019
- 2019-12-04 CN CN201911232049.9A patent/CN110928997A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106326984A (en) * | 2016-08-09 | 2017-01-11 | 北京京东尚科信息技术有限公司 | User intention identification method and device and automatic answering system |
| US20180143964A1 (en) * | 2016-11-18 | 2018-05-24 | Microsoft Technology Licensing, Llc | Data input system using trained keypress encoder |
| CN109063035A (en) * | 2018-07-16 | 2018-12-21 | 哈尔滨工业大学 | A kind of man-machine more wheel dialogue methods towards trip field |
| CN109241255A (en) * | 2018-08-20 | 2019-01-18 | 华中师范大学 | A kind of intension recognizing method based on deep learning |
| US20190057036A1 (en) * | 2018-10-15 | 2019-02-21 | Amrita MATHURIYA | Programmable interface to in-memory cache processor |
| CN110046221A (en) * | 2019-03-01 | 2019-07-23 | 平安科技（深圳）有限公司 | A kind of machine dialogue method, device, computer equipment and storage medium |
| CN110245348A (en) * | 2019-05-17 | 2019-09-17 | 北京百度网讯科技有限公司 | A kind of intension recognizing method and system |
| CN110298403A (en) * | 2019-07-02 | 2019-10-01 | 郭刚 | The sentiment analysis method and system of enterprise dominant in a kind of financial and economic news |
Cited By (66)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111460302A (en) * | 2020-03-31 | 2020-07-28 | 拉扎斯网络科技（上海）有限公司 | Data processing method and device, electronic equipment and computer readable storage medium |
| CN111460303A (en) * | 2020-03-31 | 2020-07-28 | 拉扎斯网络科技（上海）有限公司 | Data processing method and device, electronic equipment and computer readable storage medium |
| CN111460303B (en) * | 2020-03-31 | 2023-07-04 | 拉扎斯网络科技（上海）有限公司 | Data processing method, device, electronic equipment and computer readable storage medium |
| CN111460302B (en) * | 2020-03-31 | 2023-08-08 | 拉扎斯网络科技（上海）有限公司 | Data processing method, device, electronic equipment and computer readable storage medium |
| CN111462752A (en) * | 2020-04-01 | 2020-07-28 | 北京思特奇信息技术股份有限公司 | Client intention identification method based on attention mechanism, feature embedding and BI-L STM |
| CN111462752B (en) * | 2020-04-01 | 2023-10-13 | 北京思特奇信息技术股份有限公司 | Attention mechanism, feature embedding and BI-LSTM (business-to-business) based customer intention recognition method |
| CN111462735A (en) * | 2020-04-10 | 2020-07-28 | 网易（杭州）网络有限公司 | Voice detection method and device, electronic equipment and storage medium |
| CN111462735B (en) * | 2020-04-10 | 2023-11-28 | 杭州网易智企科技有限公司 | Voice detection method, device, electronic equipment and storage medium |
| CN111553363A (en) * | 2020-04-20 | 2020-08-18 | 北京易道博识科技有限公司 | End-to-end seal identification method and system |
| CN111553363B (en) * | 2020-04-20 | 2023-08-04 | 北京易道博识科技有限公司 | End-to-end seal identification method and system |
| CN111737458A (en) * | 2020-05-21 | 2020-10-02 | 平安国际智慧城市科技股份有限公司 | Intention identification method, device and equipment based on attention mechanism and storage medium |
| CN111737458B (en) * | 2020-05-21 | 2024-05-21 | 深圳赛安特技术服务有限公司 | Attention mechanism-based intention recognition method, device, equipment and storage medium |
| CN111858848A (en) * | 2020-05-22 | 2020-10-30 | 深圳创新奇智科技有限公司 | Semantic classification method and device, electronic equipment and storage medium |
| CN111858848B (en) * | 2020-05-22 | 2024-03-15 | 青岛创新奇智科技集团股份有限公司 | Semantic classification method and device, electronic equipment and storage medium |
| CN111709223A (en) * | 2020-06-02 | 2020-09-25 | 上海硬通网络科技有限公司 | Method and device for generating sentence vector based on bert and electronic equipment |
| CN111709223B (en) * | 2020-06-02 | 2023-08-08 | 上海硬通网络科技有限公司 | Sentence vector generation method and device based on bert and electronic equipment |
| CN113806470A (en) * | 2020-06-15 | 2021-12-17 | 华为技术有限公司 | Semantic recognition method and device |
| CN111931513B (en) * | 2020-07-08 | 2023-11-14 | 泰康保险集团股份有限公司 | Text intention recognition method and device |
| CN111931513A (en) * | 2020-07-08 | 2020-11-13 | 泰康保险集团股份有限公司 | Text intention identification method and device |
| CN111813896B (en) * | 2020-07-13 | 2022-12-02 | 重庆紫光华山智安科技有限公司 | Text triple relation identification method and device, training method and electronic equipment |
| CN111813896A (en) * | 2020-07-13 | 2020-10-23 | 重庆紫光华山智安科技有限公司 | Text triple relation identification method and device, training method and electronic equipment |
| CN111625641A (en) * | 2020-07-30 | 2020-09-04 | 浙江大学 | Dialog intention recognition method and system based on multi-dimensional semantic interaction representation model |
| CN112256864A (en) * | 2020-09-23 | 2021-01-22 | 北京捷通华声科技股份有限公司 | Multi-intention recognition method and device, electronic equipment and readable storage medium |
| CN112256864B (en) * | 2020-09-23 | 2024-05-14 | 北京捷通华声科技股份有限公司 | Multi-intention recognition method, device, electronic equipment and readable storage medium |
| CN112086086A (en) * | 2020-10-22 | 2020-12-15 | 平安科技（深圳）有限公司 | Speech synthesis method, device, equipment and computer readable storage medium |
| CN112086086B (en) * | 2020-10-22 | 2024-06-25 | 平安科技（深圳）有限公司 | Speech synthesis method, device, equipment and computer readable storage medium |
| CN114492447B (en) * | 2020-10-23 | 2025-01-17 | 华为技术有限公司 | Text data processing method and intelligent device |
| CN114492447A (en) * | 2020-10-23 | 2022-05-13 | 华为技术有限公司 | A text data processing method and smart device |
| CN112257432A (en) * | 2020-11-02 | 2021-01-22 | 北京淇瑀信息科技有限公司 | Self-adaptive intention identification method and device and electronic equipment |
| WO2021208696A1 (en) * | 2020-11-19 | 2021-10-21 | 平安科技（深圳）有限公司 | User intention analysis method, apparatus, electronic device, and computer storage medium |
| CN112463922A (en) * | 2020-11-25 | 2021-03-09 | 中国测绘科学研究院 | Risk user identification method and storage medium |
| CN112182231B (en) * | 2020-12-01 | 2021-03-09 | 佰聆数据股份有限公司 | Text processing method, system and storage medium based on sentence vector pre-training model |
| CN112182231A (en) * | 2020-12-01 | 2021-01-05 | 佰聆数据股份有限公司 | Text processing method, system and storage medium based on sentence vector pre-training model |
| WO2022123695A1 (en) * | 2020-12-09 | 2022-06-16 | 日本電信電話株式会社 | Learning device, search device, learning method, search method, and program |
| CN112446219A (en) * | 2020-12-11 | 2021-03-05 | 四川长虹电器股份有限公司 | Chinese request text intention analysis method |
| CN112883221A (en) * | 2020-12-11 | 2021-06-01 | 广州橙行智动汽车科技有限公司 | Semantic information correction method and device and intelligent cabin |
| CN112883221B (en) * | 2020-12-11 | 2024-03-08 | 广州橙行智动汽车科技有限公司 | Semantic information correction method and device and intelligent cabin |
| CN112699213A (en) * | 2020-12-23 | 2021-04-23 | 平安普惠企业管理有限公司 | Speech intention recognition method and device, computer equipment and storage medium |
| CN112686023A (en) * | 2020-12-29 | 2021-04-20 | 平安普惠企业管理有限公司 | Text data processing method and device, electronic equipment and storage medium |
| CN112765959B (en) * | 2020-12-31 | 2024-05-28 | 康佳集团股份有限公司 | Intent recognition method, device, equipment and computer readable storage medium |
| CN112765959A (en) * | 2020-12-31 | 2021-05-07 | 康佳集团股份有限公司 | Intention recognition method, device, equipment and computer readable storage medium |
| CN112699686A (en) * | 2021-01-05 | 2021-04-23 | 浙江诺诺网络科技有限公司 | Semantic understanding method, device, equipment and medium based on task type dialog system |
| CN112699686B (en) * | 2021-01-05 | 2024-03-08 | 浙江诺诺网络科技有限公司 | Semantic understanding method, device, equipment and medium based on task type dialogue system |
| CN112765328A (en) * | 2021-01-28 | 2021-05-07 | 珠海格力电器股份有限公司 | Text similarity determination method, system, storage medium and equipment |
| CN112765328B (en) * | 2021-01-28 | 2024-05-31 | 珠海格力电器股份有限公司 | Text similarity determination method, system, storage medium and equipment |
| CN112818086A (en) * | 2021-02-04 | 2021-05-18 | 上海畅圣计算机科技有限公司 | Multi-label classification method for acquiring client intention label by robot |
| CN112949317B (en) * | 2021-02-26 | 2022-10-11 | 平安普惠企业管理有限公司 | Text semantic recognition method and device, computer equipment and storage medium |
| CN112949317A (en) * | 2021-02-26 | 2021-06-11 | 平安普惠企业管理有限公司 | Text semantic recognition method and device, computer equipment and storage medium |
| CN113064973A (en) * | 2021-04-12 | 2021-07-02 | 平安国际智慧城市科技股份有限公司 | Text classification method, device, equipment and storage medium |
| CN113297374B (en) * | 2021-04-29 | 2023-09-12 | 军事科学院系统工程研究院网络信息研究所 | Text classification method based on BERT and word feature fusion |
| CN113297374A (en) * | 2021-04-29 | 2021-08-24 | 军事科学院系统工程研究院网络信息研究所 | Text classification method based on BERT and word feature fusion |
| CN113094473A (en) * | 2021-04-30 | 2021-07-09 | 平安国际智慧城市科技股份有限公司 | Keyword weight calculation method and device, computer equipment and storage medium |
| CN113190670A (en) * | 2021-05-08 | 2021-07-30 | 重庆第二师范学院 | Information display method and system based on big data platform |
| CN113343711B (en) * | 2021-06-29 | 2024-05-10 | 南方电网数字电网研究院有限公司 | Work order generation method, device, equipment and storage medium |
| CN113343711A (en) * | 2021-06-29 | 2021-09-03 | 南方电网数字电网研究院有限公司 | Work order generation method, device, equipment and storage medium |
| CN113609768A (en) * | 2021-08-02 | 2021-11-05 | 国网浙江省电力有限公司台州供电公司 | Calculation method of line loss rate in station area based on bidirectional LSTM network |
| CN113724697A (en) * | 2021-08-27 | 2021-11-30 | 北京百度网讯科技有限公司 | Model generation method, emotion recognition method, device, equipment and storage medium |
| CN113935330A (en) * | 2021-10-22 | 2022-01-14 | 平安科技（深圳）有限公司 | Voice-based disease early warning method, device, equipment and storage medium |
| CN113935330B (en) * | 2021-10-22 | 2024-10-25 | 平安科技（深圳）有限公司 | Disease early warning method, device, equipment and storage medium based on voice |
| CN114020930A (en) * | 2021-11-03 | 2022-02-08 | 宁波深擎信息科技有限公司 | User label identification method and device based on session scene |
| CN114036943A (en) * | 2021-11-15 | 2022-02-11 | 平安普惠企业管理有限公司 | Intent recognition method, apparatus, computer equipment and storage medium |
| CN117221839B (en) * | 2023-11-09 | 2024-01-16 | 北京中科网芯科技有限公司 | 5G signaling identification method and system thereof |
| CN117221839A (en) * | 2023-11-09 | 2023-12-12 | 北京中科网芯科技有限公司 | 5G signaling identification method and system thereof |
| CN117574918B (en) * | 2024-01-15 | 2024-05-03 | 青岛冠成软件有限公司 | Intelligent interaction method based on LSTM |
| CN117574918A (en) * | 2024-01-15 | 2024-02-20 | 青岛冠成软件有限公司 | Intelligent interaction method based on LSTM |
| CN118413923A (en) * | 2024-06-27 | 2024-07-30 | 杭州字棒棒科技有限公司 | Intelligent desk lamp learning auxiliary system and method based on AI voice interaction |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110928997A (en) | Intention recognition method and device, electronic equipment and readable storage medium | |
| CN110598206B (en) | Text semantic recognition method and device, computer equipment and storage medium | |
| CN111460807B (en) | Sequence labeling method, device, computer equipment and storage medium | |
| CN112613308B (en) | User intention recognition method, device, terminal equipment and storage medium | |
| US11893345B2 (en) | Inducing rich interaction structures between words for document-level event argument extraction | |
| CN108959246B (en) | Answer selection method and device based on improved attention mechanism and electronic equipment | |
| CN109033068B (en) | Method and device for reading and understanding based on attention mechanism and electronic equipment | |
| US11544456B2 (en) | Interpretable label-attentive encoder-decoder parser | |
| CN108959482B (en) | Single-round dialogue data classification method and device based on deep learning and electronic equipment | |
| CN111930942B (en) | Text classification method, language model training method, device and equipment | |
| CN113987187B (en) | Public opinion text classification method, system, terminal and medium based on multi-label embedding | |
| CN111222305A (en) | Information structuring method and device | |
| US20230205995A1 (en) | Methods and Systems for Automated Detection of Personal Information Using Neural Networks | |
| CN111241232B (en) | Business service processing method and device, service platform and storage medium | |
| CN112699686B (en) | Semantic understanding method, device, equipment and medium based on task type dialogue system | |
| CN109344242B (en) | A dialogue question answering method, device, equipment and storage medium | |
| CN113204970A (en) | BERT-BilSTM-CRF named entity detection model and device | |
| CN116796744A (en) | Entity relation extraction method and system based on deep learning | |
| CN112307179A (en) | Text matching method, apparatus, device and storage medium | |
| CN113536784B (en) | Text processing method, device, computer equipment and storage medium | |
| Zhen et al. | The research of convolutional neural network based on integrated classification in question classification | |
| CN112347783A (en) | Method for identifying types of alert condition record data events without trigger words | |
| CN113221553A (en) | Text processing method, device and equipment and readable storage medium | |
| CN111597816A (en) | Self-attention named entity recognition method, device, equipment and storage medium | |
| CN111680136A (en) | Method and device for matching spoken language and semantics |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: Room 401, building 4, area C, Dongsheng Science Park, 66 xixiaokou Road, Haidian District, Beijing Applicant after: Zhongdian Jinxin Software Co.,Ltd. Address before: Room 401, building 4, area C, Dongsheng Science Park, 66 xixiaokou Road, Haidian District, Beijing Applicant before: Beijing Wensi Haihui Jinxin Software Co.,Ltd. |
|
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200327 |