CN111966811A - Intention recognition and slot filling method and device, readable storage medium and terminal equipment - Google Patents

Intention recognition and slot filling method and device, readable storage medium and terminal equipment Download PDF

Info

Publication number
CN111966811A
CN111966811A CN202011022747.9A CN202011022747A CN111966811A CN 111966811 A CN111966811 A CN 111966811A CN 202011022747 A CN202011022747 A CN 202011022747A CN 111966811 A CN111966811 A CN 111966811A
Authority
CN
China
Prior art keywords
sequence
slot filling
symbol
target text
intention
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202011022747.9A
Other languages
Chinese (zh)
Inventor
阎守卫
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Ping An Zhitong Consulting Co Ltd Shanghai Branch
Original Assignee
Ping An Zhitong Consulting Co Ltd Shanghai Branch
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ping An Zhitong Consulting Co Ltd Shanghai Branch filed Critical Ping An Zhitong Consulting Co Ltd Shanghai Branch
Priority to CN202011022747.9A priority Critical patent/CN111966811A/en
Publication of CN111966811A publication Critical patent/CN111966811A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/33—Querying
    • G06F16/332—Query formulation
    • G06F16/3329—Natural language query formulation or dialogue systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/33—Querying
    • G06F16/3331—Query processing
    • G06F16/334—Query execution
    • G06F16/3343—Query execution using phonetics
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/20—Natural language analysis
    • G06F40/279—Recognition of textual entities
    • G06F40/284—Lexical analysis, e.g. tokenisation or collocates
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/30—Semantic analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods

Abstract

The application belongs to the technical field of natural language processing, and particularly relates to an intention recognition and slot filling method, an intention recognition and slot filling device, a computer readable storage medium and a terminal device. The method comprises the steps of obtaining a target text to be processed; segmenting the target text to obtain a symbol sequence of the target text; and performing intention recognition and slot filling joint processing on the symbol sequence by using a preset neural network model to obtain an intention recognition result and a slot filling result of the target text. Compared with the prior art that intention identification and slot filling are completed as two independent tasks, the embodiment of the application makes full use of the association between the intention identification and the slot filling, namely the intention identification has strong influence on the slot filling effect, and vice versa, and the accuracy of the final result is greatly improved by performing the combined processing of the intention identification and the slot filling. In addition, the application also relates to an artificial intelligence technology and a block chain technology, and can be applied to the field of smart cities.

Description

Intention recognition and slot filling method and device, readable storage medium and terminal equipment
Technical Field
The application belongs to the technical field of natural language processing, and particularly relates to an intention recognition and slot filling method, an intention recognition and slot filling device, a computer readable storage medium and a terminal device.
Background
Intention recognition and slot filling are very important tasks in the field of natural language processing (especially in the field of intelligent question answering) and are key to influencing the quality of an intelligent question answering system. The intention recognition can help the intelligent question-answering system understand the user, and the slot filling can extract and refine key information required by the intelligent question-answering system to meet the intention of the user. In the prior art, CNN, RNN, LSTM and the like are generally used for intention identification, and Bi-LSTM + CRF and the like are used for groove filling, so that the accuracy is low.
Disclosure of Invention
In view of this, embodiments of the present application provide an intention identification and slot filling method, an intention identification and slot filling device, a computer-readable storage medium, and a terminal device, so as to solve the problem of a low accuracy rate when performing intention identification and slot filling in the prior art.
A first aspect of an embodiment of the present application provides an intention identification and slot filling method, which may include:
acquiring a target text to be processed;
segmenting the target text to obtain a symbol sequence of the target text;
and performing intention recognition and slot filling joint processing on the symbol sequence by using a preset neural network model to obtain an intention recognition result and a slot filling result of the target text.
A second aspect of embodiments of the present application provides an intention recognition and slot filling apparatus, which may include:
the text acquisition module is used for acquiring a target text to be processed;
the text segmentation module is used for carrying out segmentation processing on the target text to obtain a symbol sequence of the target text;
and the joint processing module is used for carrying out intention recognition and slot filling joint processing on the symbol sequence by using a preset neural network model to obtain an intention recognition result and a slot filling result of the target text.
A third aspect of embodiments of the present application provides a computer-readable storage medium storing computer-readable instructions that, when executed by a processor, implement the steps of:
acquiring a target text to be processed;
segmenting the target text to obtain a symbol sequence of the target text;
and performing intention recognition and slot filling joint processing on the symbol sequence by using a preset neural network model to obtain an intention recognition result and a slot filling result of the target text.
A fourth aspect of the embodiments of the present application provides a terminal device, including a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor, where the processor executes the computer-readable instructions to implement the following steps:
acquiring a target text to be processed;
segmenting the target text to obtain a symbol sequence of the target text;
and performing intention recognition and slot filling joint processing on the symbol sequence by using a preset neural network model to obtain an intention recognition result and a slot filling result of the target text.
Compared with the prior art, the embodiment of the application has the advantages that: the method comprises the steps of obtaining a target text to be processed; segmenting the target text to obtain a symbol sequence of the target text; and performing intention recognition and slot filling joint processing on the symbol sequence by using a preset neural network model to obtain an intention recognition result and a slot filling result of the target text. Compared with the prior art that intention identification and slot filling are completed as two independent tasks, the embodiment of the application makes full use of the association between the intention identification and the slot filling, namely the intention identification has strong influence on the slot filling effect, and vice versa, and the accuracy of the final result is greatly improved by performing the combined processing of the intention identification and the slot filling.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments or the prior art descriptions will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive exercise.
FIG. 1 is a flow chart of one embodiment of an intent recognition and slot filling method in an embodiment of the present application;
FIG. 2 is a schematic flow diagram of the joint process of intent recognition and slot filling for the symbol sequence;
FIG. 3 is a schematic diagram of generating a superposition sequence corresponding to the symbol sequence;
FIG. 4 is a schematic illustration of an attention mechanism;
FIG. 5 is a block diagram of one embodiment of an intent recognition and slot filling apparatus in an embodiment of the present application;
fig. 6 is a schematic block diagram of a terminal device in an embodiment of the present application.
Detailed Description
In order to make the objects, features and advantages of the present invention more apparent and understandable, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application, and it is apparent that the embodiments described below are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
Referring to fig. 1, an embodiment of an intention identification and slot filling method in an embodiment of the present application may include:
and step S101, acquiring a target text to be processed.
In a specific implementation of the embodiment of the application, a user can directly input a text through an entity key or a virtual key in a designated human-computer interaction interface and send the text to a preset terminal device, and the terminal device can process the text as the target text after receiving the text input by the user.
In another specific implementation of the embodiment of the application, the user can interact with the terminal device in a voice mode, the terminal device collects the conversation voice of the user through a microphone, a microphone and other voice collecting devices carried by the terminal device, and performs text processing on the collected conversation voice to obtain the target text.
And S102, segmenting the target text to obtain a symbol sequence of the target text.
For the target text, it needs to be preprocessed first, and then divided into individual sentences. In the embodiment of the present application, the target text may be divided by commas, periods, question marks, exclamation marks, and the like as sentence dividers. For convenience of subsequent processing, the embodiment of the present application may insert a special symbol in the target text to mark the segmentation result, for example, a preset first symbol (the symbol is denoted as [ CLS ]) may be inserted before the first segmented sentence, a preset second symbol (the symbol is denoted as [ SEP ]) may be added after each segmented sentence, and the [ CLS ] and the [ SEP ] and the characters therebetween are used as a set of token sequences, that is, the symbol sequences of the target text.
It should be noted that the "character" mentioned in the embodiments of the present application is the smallest unit of natural language processing, and in general, each word may be regarded as one character for english text or other alphabetical phonetic text, and each kanji may be regarded as one character for chinese text.
And S103, performing intention recognition and slot filling joint processing on the symbol sequence by using a preset neural network model to obtain an intention recognition result and a slot filling result of the target text.
In a specific implementation of the embodiment of the present application, an artificial intelligence technique is adopted, the neural network model may be a BERT (bidirectional Encoder reconstruction from transforms) model, a model architecture of the BERT is based on multi-layer bidirectional transform decoding, because a decoder (decoder) cannot obtain information to be predicted, main innovation points of the model are all on a pre-training method, that is, two methods, named master LM and Next sequence Prediction, are used to capture expressions at a word level and a Sentence level, respectively. Where "bi-directional" indicates that the model can use both the preceding word and the following word when processing a word, the "bi-directional" comes from the fact that BERT, unlike conventional language models, does not predict the most likely current word given all preceding words, but rather randomly masks some words and uses all unmasked words for prediction. The BERT model is pre-trained in a large amount of unlabeled text with two strategies, one is a mask language model and the other is next sequence prediction. The pre-trained BERT model provides a powerful sentence representation containing context dependencies, which can be used to process multiple Natural Language Processing (NLP) tasks, such as the intent recognition and word slot filling tasks in the embodiments of the present application.
As shown in fig. 2, step S103 may specifically include the following steps:
and step S1031, generating a superposition sequence corresponding to the symbol sequence.
Specifically, a word embedding sequence, a segment embedding sequence and a position embedding sequence corresponding to the symbol sequence may be first generated, where the word embedding sequence includes word vectors of each symbol in the symbol sequence, the segment embedding sequence includes segment information to which each symbol in the symbol sequence belongs, and the position embedding sequence includes position information of each symbol in the symbol sequence;
and then, overlapping the word embedding sequence, the segment embedding sequence and the position embedding sequence to generate the overlapping sequence.
For each symbol (including characters and inserted special symbols) in the symbol sequence of the target text, there are three Embedding (Embedding) forms, namely word Embedding (Token Embedding), Segment Embedding (Segment Embedding) and Position Embedding (Position Embedding), wherein the word Embedding represents a word vector of the symbol, and the first symbol is [ CLS ], which can be used for a subsequent classification task; the segmentation embedding is used for distinguishing different sentences, if the target text is composed of a plurality of sentences, each sentence has an integral sentence embedding item corresponding to each symbol; position embedding represents position information, which needs to be encoded here because word order in NLP is an important feature; and superposing the three embeddings corresponding to each symbol to form a superposed sequence corresponding to the symbol sequence.
As shown in fig. 3, the symbol sequence includes two natural sentences, namely, a sentence a [ my dog is cut ] and a sentence B [ he tokens playing ], and each symbol and a special symbol need to be converted into a word embedding vector at first, because the neural network can only perform numerical calculation. The special symbol [ SEP ] is a symbol for dividing two sentences, the former half sentence is added with a division code A, and the latter half sentence is added with a division code B. Because of the relationships between sentences to be modeled, BERT has a task of predicting whether a sentence B is a sentence that follows a sentence a, and this classification task is performed by means of the first special symbol [ CLS ] of a/B sentences, which can be considered as a collection of tokens for the entire sequence of symbols. The final position encoding is determined by the encoder architecture itself, because the full attention-based approach cannot encode the position relationship between words like CNN or RNN, but because of this property the relationship between two words can be modeled regardless of the distance. Therefore, in order for the encoder to sense the positional relationship between words, it is necessary here to add positional information to each symbol using positional encoding.
Step S1032 performs hidden layer processing on the superimposed sequence to obtain a hidden layer state of each symbol in the superimposed sequence.
In this embodiment of the present application, the superposition sequence is used as an input of the BERT, and the hidden layer state of each symbol in the superposition sequence can be obtained through hidden layer processing of the BERT. Here, the first symbol in the superimposed sequence, i.e., the hidden layer state of [ CLS ], may be denoted as h _1, the hidden layer state of the second symbol in the superimposed sequence may be denoted as h _2, the hidden layer state of the third symbol in the superimposed sequence may be denoted as h _ 3.
And step S1033, generating an intention matrix of the target text according to the hidden layer state of the first symbol in the superposition sequence.
The hidden layer state of the first symbol in the superimposed sequence is a vector with 768 dimensions, as follows:
[1,1,1,…]
the hidden layer state of the first symbol in the superimposed sequence is processed by the full link layer of BERT, and output vectors of all link layers, that is, 12 vectors of 768 dimensions, can be obtained as follows:
[1,1,1,…]
[1,1,1,…]
……
and splicing the output vectors of all the full connection layers to obtain an intention matrix of the target text. The intention matrix is a matrix with 12 rows and 758 columns, wherein each row is a full-connection layer output vector, and the intention matrix is information representation of intention classification, and is as follows:
[1,1,1,…
1,1,1,…
……
1,1,1,…]
it should be noted that the above processes are all described by taking the all-1 vector and the all-1 matrix as examples, and it should be clear to those skilled in the art that this is only a specific example, and is not a limitation to specific values of the vector and the matrix.
Step S1034, processing the hidden layer states of the symbols in the intention matrix and the superimposed sequence by using an attention mechanism, and obtaining an attention matrix of the target text.
The attention mechanism is a technology which enables a model to pay attention to important information and learn, and is commonly used in a seq2seq model. During the model encoding phase, the attention mechanism will provide the hidden layer states of all nodes as contexts to the decoder. In the decoding phase, the decoder will adopt a selection mechanism to select the hidden layer state (hidden state) that best fits the current position, which includes the following three steps: determining which hidden layer state is closest to the current node; calculating a score value of each hidden layer state; softmax calculations were performed for each score. As shown in FIG. 4, the nature of the attention mechanism can be described as a mapping of one query (query) to a series of key-value pairs. Firstly, similarity calculation is carried out on the query and each key to obtain a weight, then softmax normalization calculation is carried out on each weight, and finally the weight and the corresponding key Value are weighted and summed to obtain an Attention Value (Attention Value).
In the method and the device, intention classification information is fused into a word slot filling process, token sequences are focused through intention classification vectors, namely an attention mechanism is adopted to enhance the word slot filling effect, and potential information association between the intention and the word slot is better utilized through information representation and combined training of intention recognition and word slot filling in a model architecture.
Specifically, in the embodiment of the present application, the attention matrix may be calculated according to the following formula:
H～＝Attention(Q,K,V)＝Attention(H_(2:T),Intent,Intent)
＝softmax(H_(2:T)*Intent^T)*Intent
wherein H _ (2: T) is a hidden layer state of each symbol except a first symbol in the superimposed sequence, Intent is the Intent matrix, softmax is a softmax function, Attention is a Attention mechanism processing function, Q is a query parameter in an Attention mechanism, Q is H _ (2: T), K is a key parameter in the Attention mechanism, K is Intent, V is a value parameter in the Attention mechanism, and V is Intent, H～Is the attention matrix, and H～＝[(h_2)～,…,(h_T)～]，(h_2)～Is the stacking orderAttention vector of the second symbol in the column, (h _3)～Is the attention vector of the third symbol in the superimposed sequence～Attention vector for the T-th symbol in the superimposed sequence, and so on.
And step S1035, determining an intention recognition result and a slot filling result of the target text according to the attention moment array of the target text.
Specifically, the attention matrix may be input into a softmax function, and a classification result, that is, an intention recognition result and a slot filling result of the target text may be obtained.
Corresponding to the combined model architecture for intention recognition and word slot filling proposed in the above process, the embodiment of the present application may perform joint training on the combined model architecture by using an objective function as shown below:
Figure BDA0002701181250000081
wherein x is a training text, N is the serial number of words in the training text, N is more than or equal to 1 and less than or equal to N, N is the total number of words in the training text, i is the serial number of each intention recognition result of the training text, s is the serial number of each slot filling result of the training text,
Figure BDA0002701181250000082
probability of filling result for nth word in training text as s type slot, p (y)i| x) is the probability that the training text is the ith intention recognition result, p (y)i,ysAnd | x) is the probability that the training text is the ith intention recognition result and is the filling result of the s-th slot.
The final goal of the joint training is to maximize p (y)i,ys| x), performing end-to-end fine-tune by minimizing cross entropy loss to obtain a final model, and then performing joint analysis of intention recognition and slot filling on the text by using the model.
Preferably, in order to further ensure the privacy and security of the intention recognition result and the slot filling result of the target text, the intention recognition result and the slot filling result of the target text may also be stored in a node of a block chain.
It should be noted that the blockchain in the present invention is a novel application mode of computer technologies such as distributed data storage, point-to-point transmission, consensus mechanism, and encryption algorithm. A block chain (Blockchain), which is essentially a decentralized database, is a series of data blocks associated by using a cryptographic method, and each data block contains information of a batch of network transactions, so as to verify the validity (anti-counterfeiting) of the information and generate a next block. The blockchain may include a blockchain underlying platform, a platform product service layer, an application service layer, and the like.
Simultaneously, this scheme can be applied to in the smart city field to promote smart city's construction.
To sum up, the embodiment of the application obtains a target text to be processed; segmenting the target text to obtain a symbol sequence of the target text; and performing intention recognition and slot filling joint processing on the symbol sequence by using a preset neural network model to obtain an intention recognition result and a slot filling result of the target text. Compared with the prior art that intention identification and slot filling are completed as two independent tasks, the embodiment of the application makes full use of the association between the intention identification and the slot filling, namely the intention identification has strong influence on the slot filling effect, and vice versa, and the accuracy of the final result is greatly improved by performing the combined processing of the intention identification and the slot filling.
It should be understood that, the sequence numbers of the steps in the foregoing embodiments do not imply an execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.
Fig. 5 is a block diagram of an embodiment of an intention identifying and slot filling apparatus according to an embodiment of the present application, corresponding to an intention identifying and slot filling method according to the above embodiment.
In this embodiment, an intention recognition and slot filling apparatus may include:
a text obtaining module 501, configured to obtain a target text to be processed;
a text segmentation module 502, configured to perform segmentation processing on the target text to obtain a symbol sequence of the target text;
a joint processing module 503, configured to perform intent recognition and slot filling joint processing on the symbol sequence by using a preset neural network model, so as to obtain an intent recognition result and a slot filling result of the target text.
Further, the joint processing module may include:
a superimposed sequence generating unit configured to generate a superimposed sequence corresponding to the symbol sequence;
a hidden layer processing unit, configured to perform hidden layer processing on the superimposed sequence to obtain a hidden layer state of each symbol in the superimposed sequence;
the intention matrix generating unit is used for generating an intention matrix of the target text according to the hidden layer state of the first symbol in the superposition sequence;
the attention moment array computing unit is used for processing the intention matrix and the hidden layer state of each symbol in the superposition sequence by using an attention mechanism to obtain an attention matrix of the target text;
and the result determining unit is used for determining the intention recognition result and the slot filling result of the target text according to the attention moment array of the target text.
Further, the superimposition sequence generation unit may include:
a sequence generating subunit, configured to generate a word embedding sequence, a segment embedding sequence, and a position embedding sequence corresponding to the symbol sequence, respectively, where the word embedding sequence includes word vectors of symbols in the symbol sequence, the segment embedding sequence includes segment information to which each symbol in the symbol sequence belongs, and the position embedding sequence includes position information of each symbol in the symbol sequence;
and the sequence overlapping subunit is used for overlapping the word embedding sequence, the segment embedding sequence and the position embedding sequence to generate the overlapping sequence.
Further, the intention matrix generating unit may include:
a full-connection layer processing subunit, configured to perform full-connection layer processing on the hidden layer state of the first symbol in the superposition sequence to obtain output vectors of all full-connection layers;
and the splicing processing subunit is used for splicing the output vectors of all the full connection layers to obtain an intention matrix of the target text.
Further, the attention matrix calculation unit is specifically configured to calculate the attention matrix according to the following equation:
H～＝Attention(Q,K,V)＝Attention(H_(2:T),Intent,Intent)
＝softmax(H_(2:T)*Intent^T)*Intent
wherein T is a total number of symbols in the superimposed sequence, H _ (2: T) is a hidden layer state of each symbol in the superimposed sequence except a first symbol, Intent is the Intent matrix, softmax is a softmax function, Attention is a Attention mechanism processing function, Q is a query parameter in the Attention mechanism, and Q is H _ (2: T), K is a key parameter in the Attention mechanism, and K is Intent, V is a value parameter in the Attention mechanism, and V is Intent, H _ (2: T), and～is the attention matrix, and H～＝[(h_2)～,…,(h_T)～]，(h_2)～Attention vector for the second symbol in the superimposed sequence, (h _3)～Is the attention vector of the third symbol in the superimposed sequence～Attention vector for the T-th symbol in the superimposed sequence, and so on.
Further, the intention recognition and slot filling apparatus may further include:
the model training module is used for training the initial neural network model by using a preset target function to obtain a trained neural network model; the objective function is:
Figure BDA0002701181250000111
wherein x is a training text, N is the serial number of words in the training text, N is more than or equal to 1 and less than or equal to N, N is the total number of words in the training text, i is the serial number of each intention recognition result of the training text, s is the serial number of each slot filling result of the training text,
Figure BDA0002701181250000112
probability of filling result for nth word in training text as s type slot, p (y)i| x) is the probability that the training text is the ith intention recognition result, p (y)i,ysAnd | x) is the probability that the training text is the ith intention recognition result and is the filling result of the s-th slot.
It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described apparatuses, modules and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and reference may be made to the related descriptions of other embodiments for parts that are not described or illustrated in a certain embodiment.
Fig. 6 shows a schematic block diagram of a terminal device provided in an embodiment of the present application, and only shows a part related to the embodiment of the present application for convenience of description.
In this embodiment, the terminal device 6 may be a computing device such as a desktop computer, a notebook, a palm computer, and a cloud server. The terminal device 6 may include: a processor 60, a memory 61, and computer readable instructions 62 stored in the memory 61 and executable on the processor 60, such as computer readable instructions to perform the intent recognition and slot filling methods described above. The processor 60, when executing the computer readable instructions 62, implements the steps in the various intent recognition and slot filling method embodiments described above, such as steps S101-S103 shown in fig. 1. Alternatively, the processor 60, when executing the computer readable instructions 62, implements the functions of the modules/units in the above-described device embodiments, such as the functions of the modules 501 to 503 shown in fig. 5.
Illustratively, the computer readable instructions 62 may be partitioned into one or more modules/units that are stored in the memory 61 and executed by the processor 60 to accomplish the present application. The one or more modules/units may be a series of computer-readable instruction segments capable of performing specific functions, which are used to describe the execution process of the computer-readable instructions 62 in the terminal device 6.
The Processor 60 may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic device, discrete hardware component, etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The memory 61 may be an internal storage unit of the terminal device 6, such as a hard disk or a memory of the terminal device 6. The memory 61 may also be an external storage device of the terminal device 6, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like, which are provided on the terminal device 6. Further, the memory 61 may also include both an internal storage unit and an external storage device of the terminal device 6. The memory 61 is used for storing the computer readable instructions and other instructions and data required by the terminal device 6. The memory 61 may also be used to temporarily store data that has been output or is to be output.
Each functional unit in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be substantially implemented or contributed to by the prior art, or all or part of the technical solution may be embodied in a software product, which is stored in a storage medium and includes a plurality of computer readable instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and the like, which can store computer readable instructions.
The above-mentioned embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions in the embodiments of the present application.

Claims (10)

1. An intent recognition and slot filling method, comprising:
acquiring a target text to be processed;
segmenting the target text to obtain a symbol sequence of the target text;
and performing intention recognition and slot filling joint processing on the symbol sequence by using a preset neural network model to obtain an intention recognition result and a slot filling result of the target text.
2. The method for intention recognition and slot filling according to claim 1, wherein the joint process of intention recognition and slot filling on the symbol sequence by using a preset neural network model comprises:
generating a superposition sequence corresponding to the symbol sequence;
carrying out hidden layer processing on the superposed sequence to obtain the hidden layer state of each symbol in the superposed sequence;
generating an intention matrix of the target text according to the hidden layer state of the first symbol in the superposition sequence;
processing hidden layer states of all symbols in the intention matrix and the superposition sequence by using an attention mechanism to obtain an attention matrix of the target text;
and determining an intention recognition result and a slot filling result of the target text according to the attention moment array of the target text.
3. The intent recognition and slot filling method of claim 2, wherein said generating a superposition sequence corresponding to the symbol sequence comprises:
respectively generating a word embedding sequence, a segment embedding sequence and a position embedding sequence corresponding to the symbol sequence, wherein the word embedding sequence comprises word vectors of all symbols in the symbol sequence, the segment embedding sequence comprises segment information of all symbols in the symbol sequence, and the position embedding sequence comprises position information of all symbols in the symbol sequence;
and superposing the word embedding sequence, the segment embedding sequence and the position embedding sequence to generate the superposed sequence.
4. The intent recognition and slot filling method of claim 2, wherein said generating an intent matrix for the target text from the hidden layer state of the first symbol in the superimposed sequence comprises:
carrying out full-connection layer processing on the hidden layer state of the first symbol in the superposed sequence to obtain output vectors of all full-connection layers;
and splicing the output vectors of all the full connection layers to obtain an intention matrix of the target text.
5. The intent recognition and slot filling method of claim 2, wherein said processing the hidden layer states of each symbol in the intent matrix and the superimposed sequence using an attention mechanism to obtain an attention moment matrix of the target text comprises:
calculating the attention matrix according to:
Figure FDA0002701181240000021
wherein T is a total number of symbols in the superimposed sequence, H _ (2: T) is a hidden layer state of each symbol in the superimposed sequence except a first symbol, Intent is the Intent matrix, softmax is a softmax function, Attention is a Attention mechanism processing function, Q is a query parameter in an Attention mechanism, and Q is H _ (2: T), K is a key parameter in the Attention mechanism, and K is Intent, V is a value parameter in the Attention mechanism, and V is Intent,
Figure FDA0002701181240000022
is the attention matrix.
6. The intent recognition and slot filling method according to any of claims 1 to 5, further comprising, before performing intent recognition and slot filling joint processing on the symbol sequence using a preset neural network model:
training the initial neural network model by using a preset target function to obtain a trained neural network model; the objective function is:
Figure FDA0002701181240000023
wherein x is a training text, N is the serial number of words in the training text, N is more than or equal to 1 and less than or equal to N, N is the total number of words in the training text, i is the serial number of each intention recognition result of the training text, s is the serial number of each slot filling result of the training text,
Figure FDA0002701181240000024
probability of filling result for nth word in training text as s type slot, p (y)i| x) is the probability that the training text is the ith intention recognition result, p (y)i,ysAnd | x) is the probability that the training text is the ith intention recognition result and is the filling result of the s-th slot.
7. An intent recognition and slot filling apparatus, comprising:
the text acquisition module is used for acquiring a target text to be processed;
the text segmentation module is used for carrying out segmentation processing on the target text to obtain a symbol sequence of the target text;
and the joint processing module is used for carrying out intention recognition and slot filling joint processing on the symbol sequence by using a preset neural network model to obtain an intention recognition result and a slot filling result of the target text.
8. The intent recognition and slot filling apparatus of claim 7, wherein said joint processing module comprises:
a superimposed sequence generating unit configured to generate a superimposed sequence corresponding to the symbol sequence;
a hidden layer processing unit, configured to perform hidden layer processing on the superimposed sequence to obtain a hidden layer state of each symbol in the superimposed sequence;
the intention matrix generating unit is used for generating an intention matrix of the target text according to the hidden layer state of the first symbol in the superposition sequence;
the attention moment array computing unit is used for processing the intention matrix and the hidden layer state of each symbol in the superposition sequence by using an attention mechanism to obtain an attention matrix of the target text;
and the result determining unit is used for determining the intention recognition result and the slot filling result of the target text according to the attention moment array of the target text.
9. A computer readable storage medium storing computer readable instructions, which when executed by a processor implement the steps of the intent recognition and slot filling method of any of claims 1-6.
10. A terminal device comprising a memory, a processor and computer readable instructions stored in the memory and executable on the processor, characterized in that the processor when executing the computer readable instructions implements the steps of the intent recognition and slot filling method according to any of claims 1 to 6.
CN202011022747.9A 2020-09-25 2020-09-25 Intention recognition and slot filling method and device, readable storage medium and terminal equipment Pending CN111966811A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011022747.9A CN111966811A (en) 2020-09-25 2020-09-25 Intention recognition and slot filling method and device, readable storage medium and terminal equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011022747.9A CN111966811A (en) 2020-09-25 2020-09-25 Intention recognition and slot filling method and device, readable storage medium and terminal equipment

Publications (1)

Publication Number Publication Date
CN111966811A true CN111966811A (en) 2020-11-20

Family

ID=73387060

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011022747.9A Pending CN111966811A (en) 2020-09-25 2020-09-25 Intention recognition and slot filling method and device, readable storage medium and terminal equipment

Country Status (1)

Country Link
CN (1) CN111966811A (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113051930A (en) * 2021-03-25 2021-06-29 润联软件系统(深圳)有限公司 Intent recognition method and device based on Bert model and related equipment
CN113239690A (en) * 2021-03-24 2021-08-10 浙江工业大学 Chinese text intention identification method based on integration of Bert and fully-connected neural network
CN113836928A (en) * 2021-09-28 2021-12-24 平安科技（深圳）有限公司 Text entity generation method, device, equipment and storage medium

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113239690A (en) * 2021-03-24 2021-08-10 浙江工业大学 Chinese text intention identification method based on integration of Bert and fully-connected neural network
CN113051930A (en) * 2021-03-25 2021-06-29 润联软件系统(深圳)有限公司 Intent recognition method and device based on Bert model and related equipment
CN113836928A (en) * 2021-09-28 2021-12-24 平安科技（深圳）有限公司 Text entity generation method, device, equipment and storage medium
CN113836928B (en) * 2021-09-28 2024-02-27 平安科技（深圳）有限公司 Text entity generation method, device, equipment and storage medium

Similar Documents

Publication Publication Date Title
CN112084331A (en) Text processing method, text processing device, model training method, model training device, computer equipment and storage medium
CN110134946B (en) Machine reading understanding method for complex data
CN111026857B (en) Conversation state tracking method, man-machine conversation method and system
CN111966811A (en) Intention recognition and slot filling method and device, readable storage medium and terminal equipment
CN111104512B (en) Game comment processing method and related equipment
CN110990555B (en) End-to-end retrieval type dialogue method and system and computer equipment
CN112800292B (en) Cross-modal retrieval method based on modal specific and shared feature learning
CN110678882A (en) Selecting answer spans from electronic documents using machine learning
CN112200664A (en) Repayment prediction method based on ERNIE model and DCNN model
CN111145914B (en) Method and device for determining text entity of lung cancer clinical disease seed bank
CN114021582B (en) Spoken language understanding method, device, equipment and storage medium combined with voice information
CN111914553A (en) Financial information negative subject judgment method based on machine learning
CN113779190B (en) Event causal relationship identification method, device, electronic equipment and storage medium
CN114417823A (en) Aspect level emotion analysis method and device based on syntax and graph convolution network
CN116362242A (en) Small sample slot value extraction method, device, equipment and storage medium
CN113362852A (en) User attribute identification method and device
CN111639500A (en) Semantic role labeling method and device, computer equipment and storage medium
CN116450829A (en) Medical text classification method, device, equipment and medium
CN114138954A (en) User consultation problem recommendation method, system, computer equipment and storage medium
CN110852066A (en) Multi-language entity relation extraction method and system based on confrontation training mechanism
CN114417891A (en) Reply sentence determination method and device based on rough semantics and electronic equipment
CN111428005A (en) Standard question and answer pair determining method and device and electronic equipment
CN114818644B (en) Text template generation method, device, equipment and storage medium
CN112036439B (en) Dependency relationship classification method and related equipment
CN113822018B (en) Entity relation joint extraction method

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination