CN112883179B - Man-machine conversation method and system - Google Patents
Man-machine conversation method and system Download PDFInfo
- Publication number
- CN112883179B CN112883179B CN202110200723.6A CN202110200723A CN112883179B CN 112883179 B CN112883179 B CN 112883179B CN 202110200723 A CN202110200723 A CN 202110200723A CN 112883179 B CN112883179 B CN 112883179B
- Authority
- CN
- China
- Prior art keywords
- sentence
- word
- dialogue
- function
- answer
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 101
- 239000013598 vector Substances 0.000 claims abstract description 121
- 238000012549 training Methods 0.000 claims abstract description 56
- 238000013145 classification model Methods 0.000 claims abstract description 36
- 238000007781 pre-processing Methods 0.000 claims abstract description 24
- 230000008569 process Effects 0.000 claims description 45
- 230000004913 activation Effects 0.000 claims description 14
- 238000012545 processing Methods 0.000 claims description 12
- 230000011218 segmentation Effects 0.000 claims description 9
- 230000006870 function Effects 0.000 description 133
- 238000012360 testing method Methods 0.000 description 12
- 238000013528 artificial neural network Methods 0.000 description 10
- 238000011156 evaluation Methods 0.000 description 6
- 230000003993 interaction Effects 0.000 description 6
- 238000002372 labelling Methods 0.000 description 6
- 230000009471 action Effects 0.000 description 3
- 238000013527 convolutional neural network Methods 0.000 description 3
- 125000004122 cyclic group Chemical group 0.000 description 3
- 238000012937 correction Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000000306 recurrent effect Effects 0.000 description 2
- 239000013589 supplement Substances 0.000 description 2
- 230000008901 benefit Effects 0.000 description 1
- 239000003795 chemical substances by application Substances 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003058 natural language processing Methods 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000010200 validation analysis Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/332—Query formulation
- G06F16/3329—Natural language query formulation or dialogue systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/3331—Query processing
- G06F16/334—Query execution
- G06F16/3344—Query execution using natural language analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/35—Clustering; Classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/10—Text processing
- G06F40/12—Use of codes for handling textual entities
- G06F40/126—Character encoding
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/205—Parsing
- G06F40/216—Parsing using statistical methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/279—Recognition of textual entities
- G06F40/289—Phrasal analysis, e.g. finite state techniques or chunking
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Mathematical Physics (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Databases & Information Systems (AREA)
- Human Computer Interaction (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Evolutionary Computation (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Software Systems (AREA)
- Probability & Statistics with Applications (AREA)
- Machine Translation (AREA)
Abstract
The invention discloses a man-machine conversation method and a system, which are used for preprocessing an acquired current conversation sentence to obtain a word vector sequence, inputting the word vector sequence into a sentence classification model to obtain a sentence function label of the current conversation sentence, inputting the word vector sequence and the sentence function label into a generated conversation model, and outputting an answer sentence of the current conversation sentence by adopting the generated conversation method. The invention adds a certain proportion of question sentences in the sentence function labels of the training sentence classification model, so that the answer sentences output by the generated dialogue model contain a certain proportion of question sentences, and in order to avoid the situation that the generated dialogue model outputs question sentences while other sentence patterns are wrongly output, a loss function is introduced.
Description
Technical Field
The invention relates to the technical field of natural language processing, in particular to a man-machine conversation method and a man-machine conversation system.
Background
Dialog systems, also known as interactive dialog agents or chat robots, have a very wide range of applications, such as technical support, entertainment, etc. In a dialogue system, the question asked by a general user can be solved through one round of dialogue, but in practical application, a more common scene is multiple rounds of dialogue, and the user often carries out new question for the answer of the dialogue system or expands related interaction.
However, the research on the dialogue system is mostly in a form of asking a question by a user, and the dialogue system answers a question, and the question-and-answer mode is similar to the information retrieval mode, and because the dialogue system obtains less effective information from the user, the effectiveness of dialogue interaction is affected, so that man-machine dialogue cannot be well continued, and a great difference exists between the man-machine dialogue and the dialogue in daily life, so that the experience effect of the user is poor.
Disclosure of Invention
In view of the above, the invention discloses a man-machine conversation method and a system thereof, so as to realize active conversation during man-machine conversation and improve user experience.
A human-machine conversation method, comprising:
acquiring a current dialogue sentence;
preprocessing the current dialogue sentence to obtain a word vector sequence;
inputting the word vector sequence into a pre-trained sentence classification model to obtain a sentence function label of the current dialogue sentence, wherein the sentence classification model is obtained by training the word vector sequence of the dialogue sentence as a training sample and the sentence function label as a label, and the questionable sentence proportion in the sentence function label of the sentence classification model is trained in a preset proportion interval;
And inputting the word vector sequence and the sentence function label into a pre-trained generated dialogue model, and outputting the answer sentence of the current dialogue sentence by adopting a generated dialogue method, wherein the generated dialogue model adopts a back propagation algorithm to learn in the training process, and a loss function converges to a minimum value, and the loss function is used for representing the loss condition of the classification probability of the answer sentence output by the generated dialogue model relative to the classification probability of a reference answer sentence.
Optionally, the preprocessing the current dialogue sentence to obtain a word vector sequence specifically includes:
word segmentation is carried out on the current dialogue sentence to obtain a word sequence;
And carrying out word embedding processing on each word in the word sequence, and converting each word into a word vector to obtain the word vector sequence.
Optionally, the inputting the word vector sequence and the sentence function tag into a pre-trained generating dialogue model, and outputting the answer sentence of the current dialogue sentence by adopting a generating dialogue method specifically includes:
encoding the word vector sequence by using an encoder to obtain a hidden state;
Decoding the hidden state by using a decoder to obtain an output sequence, and stopping decoding when the last character of the output sequence is a stop symbol;
and inputting the decoded output sequence and the sentence function tag into the generated dialogue model, and outputting the answer sentence of the current dialogue sentence by adopting a generated dialogue method.
Optionally, the expression of the hidden state is as follows:
ht＝f(xt,ht-1);c＝ht；
wherein h t is the hidden state, f is an activation function, the activation function is a nonlinear function, x t is a word vector in the t-th word vector sequence, h t-1 is a context vector of the word vector sequence corresponding to the time t-1, and c is a context vector.
Optionally, the loss function is a weighted average of the output statement loss factor and the classification loss factor, and the expression of the loss function is as follows:
Wherein loss is the loss function, alpha and beta represent weights of weighted averages, alpha and beta are both values between 0 and 1, and alpha+beta=1;
Wherein, the expression of the output statement loss factor is as follows:
Wherein lossA represents the output sentence loss factor, n represents the total length of the current dialogue sentence, i represents the independent variable in the summation process, represents the current dialogue sentence length in the answer sentence generation process, P represents the probability from the 1 st word to the n-1 st word for generating the n-th word, y n represents the n-th word in the answer sentence, y 1 represents the 1 st word in the answer sentence, and y n-1 represents the n-1 st word in the answer sentence;
the expression of the classification loss factor is as follows:
in the formula, k represents a sentence function class, i represents a variable in the summation process, the value is 1-k, Y represents a classification label corresponding to the answer sentence, U represents a classification label corresponding to the reference answer V, P (Y|k) represents the probability of obtaining the classification label Y by the answer sentence, and P (U|k) represents the probability of obtaining the classification label U by the generated dialogue model output reference answer V.
A human-machine conversation system, comprising:
A dialogue sentence acquisition unit for acquiring a current dialogue sentence;
the preprocessing unit is used for preprocessing the current dialogue sentence to obtain a word vector sequence;
The function tag obtaining unit is used for inputting the word vector sequence into a pre-trained sentence classification model to obtain a sentence function tag of the current dialogue sentence, wherein the sentence classification model is obtained by training the word vector sequence of the dialogue sentence as a training sample and the sentence function tag as a tag, and the question mark proportion in the sentence function tag for training the sentence classification model is within a preset proportion interval;
The answer sentence acquisition unit is used for inputting the word vector sequence and the sentence function label into a pre-trained generation type dialogue model, and outputting the answer sentence of the current dialogue sentence by adopting a generation type dialogue method, wherein the generation type dialogue model adopts a back propagation algorithm to learn in the training process, and a loss function is converged to a minimum value, and the loss function is used for representing the loss condition of the classification probability of the answer sentence output by the generation type dialogue model relative to the classification probability of a reference answer sentence.
Optionally, the preprocessing unit is specifically configured to:
word segmentation is carried out on the current dialogue sentence to obtain a word sequence;
And carrying out word embedding processing on each word in the word sequence, and converting each word into a word vector to obtain the word vector sequence.
Optionally, the answer sentence acquisition unit is specifically configured to:
encoding the word vector sequence by using an encoder to obtain a hidden state;
Decoding the hidden state by using a decoder to obtain an output sequence, and stopping decoding when the last character of the output sequence is a stop symbol;
and inputting the decoded output sequence and the sentence function tag into the generated dialogue model, and outputting the answer sentence of the current dialogue sentence by adopting a generated dialogue method.
Optionally, the expression of the hidden state is as follows:
ht＝f(xt,ht-1);c＝ht；
wherein h t is the hidden state, f is an activation function, the activation function is a nonlinear function, x t is a word vector in the t-th word vector sequence, h t-1 is a context vector of the word vector sequence corresponding to the time t-1, and c is a context vector.
Optionally, the loss function is a weighted average of the output statement loss factor and the classification loss factor, and the expression of the loss function is as follows:
Wherein loss is the loss function, alpha and beta represent weights of weighted averages, alpha and beta are both values between 0 and 1, and alpha+beta=1;
Wherein, the expression of the output statement loss factor is as follows:
Wherein lossA represents the output sentence loss factor, n represents the total length of the current dialogue sentence, i represents the independent variable in the summation process, represents the current dialogue sentence length in the answer sentence generation process, P represents the probability from the 1 st word to the n-1 st word for generating the n-th word, y n represents the n-th word in the answer sentence, y 1 represents the 1 st word in the answer sentence, and y n-1 represents the n-1 st word in the answer sentence;
the expression of the classification loss factor is as follows:
in the formula, k represents a sentence function class, i represents a variable in the summation process, the value is 1-k, Y represents a classification label corresponding to the answer sentence, U represents a classification label corresponding to the reference answer V, P (Y|k) represents the probability of obtaining the classification label Y by the answer sentence, and P (U|k) represents the probability of obtaining the classification label U by the generated dialogue model output reference answer V.
As can be seen from the above technical solution, the present invention discloses a man-machine conversation method and system, wherein a word vector sequence is obtained by preprocessing an obtained current conversation sentence, the word vector sequence is input into a sentence classification model to obtain a sentence function tag of the current conversation sentence, the word vector sequence and the sentence function tag are input into a generated conversation model, and a generated conversation method is adopted to output an answer sentence of the current conversation sentence. According to the invention, a certain proportion of question sentences are added in the sentence function labels of the training sentence classification model, so that the answer sentences output by the generated dialogue model comprise a certain proportion of question sentences, and in order to avoid the situation that the generated dialogue model outputs question sentences and wrongly outputs sentence patterns other than question sentences, a loss function is introduced, wherein the loss function represents the loss condition of the classification probability of the answer sentences output by the generated dialogue model relative to the classification probability of the reference answer sentences.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, and it is obvious that the drawings in the following description are only embodiments of the present invention, and that other drawings can be obtained according to the disclosed drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of a man-machine conversation method disclosed in an embodiment of the invention;
FIG. 2 is a flowchart of a method for inputting a word vector sequence and sentence function tags into a pre-trained generated dialogue model and outputting answer sentences of a current dialogue sentence by using the generated dialogue method according to the embodiment of the invention;
fig. 3 is a schematic structural diagram of a man-machine conversation system according to an embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The embodiment of the invention discloses a man-machine conversation method and a man-machine conversation system, which are used for preprocessing an acquired current conversation sentence to obtain a word vector sequence, inputting the word vector sequence into a sentence classification model to obtain a sentence function label of the current conversation sentence, inputting the word vector sequence and the sentence function label into a generated conversation model, and outputting an answer sentence of the current conversation sentence by adopting the generated conversation method. According to the invention, a certain proportion of question sentences are added in the sentence function labels of the training sentence classification model, so that the answer sentences output by the generated dialogue model comprise a certain proportion of question sentences, and in order to avoid the situation that the generated dialogue model outputs question sentences and wrongly outputs sentence patterns other than question sentences, a loss function is introduced, wherein the loss function represents the loss condition of the classification probability of the answer sentences output by the generated dialogue model relative to the classification probability of the reference answer sentences.
Referring to fig. 1, a flow chart of a man-machine conversation method disclosed in an embodiment of the present invention includes:
step S101, acquiring a current dialogue statement;
step S102, preprocessing the current dialogue sentence to obtain a word vector sequence;
the preprocessing process for the current dialogue statement specifically comprises the following steps:
Word segmentation is carried out on the current dialogue sentence to obtain a word sequence;
And carrying out word embedding processing on each word in the word sequence, and converting each word into a word vector to obtain a word vector sequence.
It should be noted that, word segmentation is performed on the current dialogue sentence, that is, the dialogue sentence is segmented into a sequence taking a word as a unit, and the sequence is a word sequence.
In this embodiment, a word vector sequence may be obtained by performing word embedding processing on each word in the word sequence, where word embedding is a process of converting a word into a word vector of a preset length.
Step S103, inputting the word vector sequence into a pre-trained sentence classification model to obtain a sentence function tag of the current dialogue sentence;
wherein, sentence function tags include four types: question sentences, statement sentences, imperative sentences and exclamation sentences.
The sentence classification model is obtained by training a word vector sequence of dialogue sentences and sentence function labels as training samples, wherein the ratio of question sentences in the sentence function labels of the training sentence classification model is in a preset ratio interval, and the value of the preset ratio interval is determined according to actual needs, such as 50% -60%.
Sentence function is the purpose of a user when expressing a sentence, and is one of the important attributes in linguistics, namely, an important factor representing the interaction process between the user and the dialogue system. Sentence functions are divided into four types: question sentences, statement sentences, imperative sentences and exclamation sentences. Since sentence functions are related to the purpose of speaking, sentence functions are important factors in characterizing the interaction process of a user with a dialog system. Before classifying dialogue sentences according to functions, the dialogue sentences need to be preprocessed, and the preprocessing process comprises the following steps: and segmenting the dialogue sentence to obtain a word sequence, carrying out word embedding processing on each word in the word sequence, and converting each word into a word vector to obtain a word vector sequence.
The dialogue system in this embodiment may be formalized, specifically: the given dialog statement context set d=w 1,W2,.....,Wn-1,Wn,Wn generates a reply Y n. Where W n represents the nth dialog sentence in the dialog sentence context set. In the invention, the mere use of dialogue sentence context sets and replies as training data is insufficient, and because the generated dialogue model cannot directly acquire the current dialogue state information, a sentence function information is required as a supplement. Sentence function information is used to avoid the generated dialogue model outputting monotonous statement sentences each time a question is answered, but can transform multiple sentence types, including actively asking questions to the user.
Since the sentence classification process is supervised learning according to the sentence function, all the acquired history dialogue sentences need to be divided. In this embodiment, all the historical dialogue sentences are divided into a training set and a testing set, the training set may account for 10% -20% of all the historical dialogue sentences, and each historical dialogue sentence in the training set is labeled with a sentence function label. The specific method of labeling is not required, and can be manual labeling or manual correction after labeling by using other models. The sentence function labels are marked by sentence classification models obtained through training of historical dialogue sentences in the test set.
In addition to the historical dialogue sentences in the training set, a certain proportion of statement sentences and question sentences are required to be added, so that the active question capability of the dialogue system is improved. The training data is in the form of: in the preprocessing of the dialogue sentences, the input sentences X and the output sentences Y (X, Y) are subjected to sentence function classification according to the method in step S103, so as to obtain sentence function labels D (Y), wherein D (Y) is the question sentences, and the proportion of D (Y) is 50% -60%.
When the dialogue sentences are functionally classified, the sentence classification model is input as dialogue sentences U, consists of word vector sequences x 1,x2,.....,xn and is output as sentence functional labels L.
The method of classifying dialogue sentences according to the present embodiment is not particularly limited, and for example, classification can be performed using a cyclic convolutional neural network (Recurrent Convolutional Neural Network, RCNN). The structure commonly used for solving the time sequence information such as the text processing problem is a cyclic neural network, sentences with different sentence function types also contain different keyword features, and the relative positions of the sentences do not influence the classification of the sentences, so that the feature extraction capability of the convolutional neural network can be utilized. The algorithmic model constructed by the sentence classification method is referred to as a sentence classification model.
In this embodiment, a supervised learning method is used to train to obtain a sentence classification model, and each set of sample data includes a pair of data, namely a word vector sequence as input data, and a sentence function tag as desired output data. The supervised learning algorithm is not limited in this regard, and the present embodiment analyzes training data and generates an inference function that can be used to predict data from a validation set or a test set.
And step S104, inputting the word vector sequence and the sentence function label into a pre-trained generation type dialogue model, and outputting the answer sentence of the current dialogue sentence by adopting a generation type dialogue method.
The generated dialogue model is learned by adopting a back propagation algorithm in the training process, and a loss function is converged to a minimum value, wherein the loss function is used for representing the loss condition of the classification probability of the answer sentence output by the generated dialogue model relative to the classification probability of the reference answer sentence.
In this embodiment, the generative dialogue model outputs answer sentences using the generative dialogue method. The generated dialogue is to train an end-to-end deep neural network to obtain a generated dialogue model to generate answer sentences of dialogue sentences, so the generated dialogue model can generate more flexible answer sentences for the dialogue sentences.
In the training process, the generated dialogue model provides the real word vector sequence in the dialogue sentence to the generated dialogue model, and the generated dialogue model can be learned through a back propagation algorithm. The back propagation algorithm is defined as an algorithm that trains some given feedforward neural network for a given input pattern given a known classification. The back propagation algorithm may converge the loss function to a minimum given the training set.
The dialog system trains the model using the chat corpus and generates context-adaptive answer sentences from the dialog sentences entered by the user. In the generated dialog, an end-to-end deep neural network, also referred to as a codec network, is employed.
The generated dialogue model learns from input to output mapping relation, and given the generated dialogue model, an input sequence u=x 1,x2,.....,xn,xn represents word vectors and sentence function labels L, wherein the input sequence U can be a continuous sequence consisting of n sentences, n is generally 3-5, the dialogue output is answer sentence y n, and the generated dialogue model learns by maximizing the conditional probability of y n. In the training process of the generated dialogue model, reference answers V are needed, wherein V represents real answers to U in training corpus, and the model is used as reference in training. The specific implementation of the end-to-end deep neural network can be varied, but the implementation steps should meet the following requirements:
Referring to fig. 2, a method flowchart for inputting a word vector sequence and a sentence function tag into a pre-trained generated dialogue model and outputting an answer sentence of a current dialogue sentence by using the generated dialogue method is disclosed in the embodiment of the invention, and the method comprises the following steps:
Step S201, encoding the word vector sequence by using an encoder to obtain a hidden state;
the word vector sequence is read word by an encoder and converted into context vectors through a recurrent neural network.
The hidden state of the word vector sequence is a context vector, and the expression of the hidden state is shown in formula (1), wherein formula (1) is as follows: ;
ht＝f(xt,ht-1);c＝ht (1)；
Wherein h t is a hidden state, f is an activation function, the activation function is a nonlinear function, which may be tanh, a gating unit of a long-short-term memory network, or other nonlinear functions, x t is a word vector in the t-th word vector sequence, h t-1 is a context vector of the word vector sequence corresponding to the time t-1, that is, the time t above, and c is a context vector.
Step S202, decoding the hidden state by using a decoder to obtain an output sequence, and stopping decoding when the last character of the output sequence is a stop symbol;
The generation probability p (y t|yt-1,...,y1 |x) of the t-th word is shown in formula (2), and formula (2) is as follows:
p(yt|yt-1,…,y1|x)＝g(yt-1,st,ct) (2)；
Where y t is the word vector of the t-th word in the output sequence, s t is the hidden state of the decoder at time t, y t-1 is the word vector of the t-1 st word in the output sequence, y tl is the word vector of the 1 st word in the output sequence, x is the hidden state, represents the coding of the input sequence, g is the output function of the cyclic neural network unit of the decoder, s t is the hidden state of the decoder at time t, and c t is the background variable of time step t.
The expression of the generation probability of the t-th word includes, but is not limited to, the expression shown in formula (2).
The expression of s t is shown in formula (3), and formula (3) is as follows:
st＝f(yt-1,st-1,ct) (3)；
where s t-1 is the hidden state of the decoder at time t-1, and f is an activation function, where the activation function is a nonlinear function, and may be tanh, a gating unit of a long-short-term memory network, or other nonlinear functions.
And step 203, inputting the decoded output sequence and the sentence function tag into the generated dialogue model, and outputting the answer sentence of the current dialogue sentence by adopting a generated dialogue method.
In the training process of the generated dialogue model, the real output sequence in the dialogue statement is provided for the generated dialogue model, and learning can be performed through a back propagation algorithm. The back propagation algorithm is defined as an algorithm that trains some given feedforward neural network for a given input pattern given a known classification. The back propagation algorithm may minimize the loss function given the training set. The form of the loss function will be limited in the present invention.
During the training process, the generated dialogue model can obtain an output sequence y n (i.e. an answer sentence) from the input sequence U (i.e. the word vector sequence and the sentence function tag) mentioned in step S104, wherein the output sentence loss factor of the loss function is defined as lossA, the classification loss factor is defined as lossB, which is a key step for implementing the active dialogue in the present invention.
The loss function is a weighted average of the output statement loss factor and the class loss factor, the expression of the loss function loss is shown in formula (4), and the formula (4) is as follows:
Where α and β represent weights of a weighted average, α and β each take a value between 0 and 1, and α+β=1.
The expression of the output statement loss factor of the loss function in the formula (4) is shown as formula (5), and the formula (5) is as follows:
Wherein lossA denotes an output sentence loss factor of the loss function, n denotes a total length of a current dialogue sentence, i denotes an argument in a summation process, represents a current dialogue sentence length in an answer sentence generation process, P denotes a probability from a1 st word to an n-1 st word of generation of the n-th word, y n denotes the n-th word in the answer sentence, y 1 denotes the 1 st word in the answer sentence, and y n-1 denotes the n-1 st word in the answer sentence.
Assuming that the sentence types are k, since the last step of the classifier is used as the output layer, the probability distribution of classification is a discrete probability distribution, and the discrete random variable takes a value from 1 to k.
The expression of the classification loss factor lossB is shown in formula (6), and formula (6) is as follows:
In the formula, k represents a sentence function class, i represents a variable in the summation process, the value is 1-k, Y represents a classification label corresponding to an answer sentence output by a generated dialogue model, U represents a classification label corresponding to a reference answer V, P (Y|k) represents the probability of obtaining the classification label Y by the answer sentence output by the generated dialogue model, and P (U|k) represents the probability of obtaining the classification label U by the reference answer V output by the generated dialogue model.
If the classification of the output answer sentence is consistent with the classification in the dialogue sentence, the classification loss factor takes a smaller value, and if the classification is not consistent, the classification loss factor takes a larger value.
The key of the invention for realizing the active dialogue is that the proportion of the answer (i.e. the reference answer V in S203) in the corpus is adjusted in the step S103, so that the question sentence, the praise sentence and the like occupy a certain proportion, the generated dialogue model can adjust the self parameters in the training process by the classification loss factor lossB, and output a new answer sentence after adjustment, and recalculate the loss function until the loss function converges to a smaller value, and the value of lossB tends to 0, the classification probability distribution of the output answer sentence of the generated dialogue model is basically consistent with the classification probability of the reference answer sentence in the training dialogue sentence, and the generated dialogue model can be considered to realize the diversity of the output answer sentence, namely the active dialogue.
During the test, a trained model (the loss function has converged) is used as the model for the test.
Specifically, if the generated dialogue model should output the question sentence, but incorrectly output the statement sentence or other sentence patterns, the generated dialogue model is reflected on the loss factor, the wrong prediction type is restrained, model parameters are adjusted through a back propagation algorithm, and finally the generated dialogue model can fit the output sentence type in the corpus, so that active dialogue is realized.
In summary, the invention discloses a man-machine conversation method, which is used for preprocessing an acquired current conversation sentence to obtain a word vector sequence, inputting the word vector sequence into a sentence classification model to obtain a sentence function label of the current conversation sentence, inputting the word vector sequence and the sentence function label into a generated conversation model, and outputting an answer sentence of the current conversation sentence by adopting the generated conversation method. According to the invention, a certain proportion of question sentences are added in the sentence function labels of the training sentence classification model, so that the answer sentences output by the generated dialogue model comprise a certain proportion of question sentences, and in order to avoid the situation that the generated dialogue model outputs question sentences and wrongly outputs sentence patterns other than question sentences, a loss function is introduced, wherein the loss function represents the loss condition of the classification probability of the answer sentences output by the generated dialogue model relative to the classification probability of the reference answer sentences.
In practical application, the test of the generated dialogue model comprises the following three steps:
Step one, in an active dialogue of a generated dialogue model, testing the functional classification diversity of dialogue sentences:
And performing word segmentation and word embedding processing on dialogue data D=W 1,W2,.....,Wn-1 serving as test data to obtain a word vector sequence, inputting the word vector sequence into a pre-trained sentence classification model to obtain a sentence function tag of a current dialogue sentence, inputting the word vector sequence and the sentence function tag into a pre-trained generated dialogue model, and outputting an answer sentence of the current dialogue sentence by using a generated dialogue method.
Step two, automatic evaluation:
a confusion (Perplexity) index may be employed to evaluate the generative dialog model. The confusion (perplexity) is an index for measuring the quality of the language model. The method mainly estimates the probability of occurrence of a sentence according to each word, normalizes the sentence length, and has the expression shown in a formula (7), wherein the formula (7) is as follows:
In the formula, PPL (S) represents XXX, exp represents XXX, n represents XXX, k represents XXX, log (p (S|S ')) represents XXX, S represents a dialogue sentence composed of a word sequence of length n, S' represents a word sequence composed of 1 to k-1, and p (S|S ') represents the probability that the sequence S is generated under the S' condition.
Another criterion for automatic evaluation is classification entropy: the adopted method is to count the proportion of each sentence function type, list the proportion into a table, and verify the proportion by using the concept of the classification entropy: since the entropy of the sentence function probability is related to the diversity of the output of the generated dialogue model, and the diversity of the sentence function affects the initiative of the dialogue system, the classification entropy of the generated sentence is an important index for comparing the distinction between the basic model and the improved model.
The class entropy can be expressed as: wherein i represents a sentence function tag, n represents the number of types of sentence classification, four are preferable in the present invention, including: question sentences, statement sentences, imperative sentences and exclamatory sentences, and P (i) represents the probability that a sentence classification belongs to an ith classification label.
Step three, manually evaluating
The manual evaluation has the advantage of higher accuracy, and answers generated in the model test are evaluated by adopting a manual evaluation method, wherein the evaluation can be divided into a plurality of indexes:
1. fluency of the answer sentence;
2. Contextual relevance of answer sentences;
3. Initiative of answer sentences;
Each index is given a score of 0-5 by the user, and the user scores are collected and recorded after the evaluation is finished.
Assuming that sentence classification models and generated dialogue models have been obtained through training, the following illustrates the process of user testing:
inputting a current dialogue sentence X:
i work in the sports center.
Preprocessing according to the method in the step S102, and segmenting the current dialogue sentence into four words:
I/at/sports center/work
Then converting into a word vector sequence x 1,...,x4;
corresponding to step S201, the word vector sequence is encoded by using the encoder to obtain the hidden state, and the encoding is repeated until the hidden state h 4 is obtained.
Corresponding to step S202, the hidden state is decoded by using the decoder to obtain the output sequence y 1,...,yn, and decoding is stopped when y n is the stop symbol < EOS >. In this embodiment, y 6 = < EOS > is assumed.
The label D (Y) =2 of the output sentence obtained by classification in S103 represents a question sentence. The result is the same as the output classification result of the training corpus.
For the output sequence y 1,...yn, it is converted into an output sequence of natural language by looking up a word vector table.
The length of the output sequence obtained in this embodiment is assumed to be 5, specifically:
You/yes/what/department/
The sentence is a question sentence, and it can be seen that in this embodiment, the present invention outputs the question sentence, that is, an active dialogue is implemented. Results with statistical significance can be obtained by examining more examples.
Corresponding to the embodiment of the method, the invention also discloses a man-machine conversation system.
Referring to fig. 3, a schematic structural diagram of a family dialogue system according to an embodiment of the present invention is disclosed, where the system includes:
a dialogue sentence acquisition unit 301 configured to acquire a current dialogue sentence;
a preprocessing unit 302, configured to preprocess the current dialogue sentence to obtain a word vector sequence;
the preprocessing unit 302 may specifically be configured to:
word segmentation is carried out on the current dialogue sentence to obtain a word sequence;
And carrying out word embedding processing on each word in the word sequence, and converting each word into a word vector to obtain the word vector sequence.
It should be noted that, word segmentation is performed on the current dialogue sentence, that is, the dialogue sentence is segmented into a sequence taking a word as a unit, and the sequence is a word sequence.
In this embodiment, a word vector sequence may be obtained by performing word embedding processing on each word in the word sequence, where word embedding is a process of converting a word into a word vector of a preset length.
A function tag obtaining unit 303, configured to input the word vector sequence to a pre-trained sentence classification model, so as to obtain a sentence function tag of the current dialogue sentence;
wherein, sentence function tags include four types: question sentences, statement sentences, imperative sentences and exclamation sentences.
The sentence classification model is obtained by training a word vector sequence of dialogue sentences and sentence function labels as training samples, wherein the ratio of question sentences in the sentence function labels of the training sentence classification model is in a preset ratio interval, and the value of the preset ratio interval is determined according to actual needs, such as 50% -60%.
Sentence function is the purpose of a user when expressing a sentence, and is one of the important attributes in linguistics, namely, an important factor representing the interaction process between the user and the dialogue system. Sentence functions are divided into four types: question sentences, statement sentences, imperative sentences and exclamation sentences. Since sentence functions are related to the purpose of speaking, sentence functions are important factors in characterizing the interaction process of a user with a dialog system. Before classifying dialogue sentences according to functions, the dialogue sentences need to be preprocessed, and the preprocessing process comprises the following steps: and segmenting the dialogue sentence to obtain a word sequence, carrying out word embedding processing on each word in the word sequence, and converting each word into a word vector to obtain a word vector sequence.
The dialogue system in this embodiment may be formalized, specifically: the given dialog statement context set d=w 1,W2,.....,Wn-1,Wn,Wn generates a reply Y n. Where W n represents the nth dialog sentence in the dialog sentence context set. In the invention, the mere use of dialogue sentence context sets and replies as training data is insufficient, and because the generated dialogue model cannot directly acquire the current dialogue state information, a sentence function information is required as a supplement. Sentence function information is used to avoid the generated dialogue model outputting monotonous statement sentences each time a question is answered, but can transform multiple sentence types, including actively asking questions to the user.
Since the sentence classification process is supervised learning according to the sentence function, all the acquired history dialogue sentences need to be divided. In this embodiment, all the historical dialogue sentences are divided into a training set and a testing set, the training set may account for 10% -20% of all the historical dialogue sentences, and each historical dialogue sentence in the training set is labeled with a sentence function label. The specific method of labeling is not required, and can be manual labeling or manual correction after labeling by using other models. The sentence function labels are marked by sentence classification models obtained through training of historical dialogue sentences in the test set.
In addition to the historical dialogue sentences in the training set, a certain proportion of statement sentences and question sentences are required to be added, so that the active question capability of the dialogue system is improved. The training data is in the form of: in the preprocessing of the dialogue sentences, the input sentences X and the output sentences Y (X, Y) are subjected to sentence function classification according to the method in step S103, so as to obtain sentence function labels D (Y), wherein D (Y) is the question sentences, and the proportion of D (Y) is 50% -60%.
When the dialogue sentences are functionally classified, the sentence classification model is input as dialogue sentences U, consists of word vector sequences x 1,x2,.....,xn and is output as sentence functional labels L.
And an answer sentence acquisition unit 304, configured to input the word vector sequence and the sentence function tag into a pre-trained generated dialogue model, and output an answer sentence of the current dialogue sentence by using a generated dialogue method.
The generated dialogue model is learned by adopting a back propagation algorithm in the training process, and a loss function is converged to a minimum value, wherein the loss function is used for representing the loss condition of the classification probability of the answer sentence output by the generated dialogue model relative to the classification probability of the reference answer sentence.
In this embodiment, the generative dialogue model outputs answer sentences using the generative dialogue method. The generated dialogue is to train an end-to-end deep neural network to obtain a generated dialogue model to generate answer sentences of dialogue sentences, so the generated dialogue model can generate more flexible answer sentences for the dialogue sentences.
In the training process, the generated dialogue model provides the real word vector sequence in the dialogue sentence to the generated dialogue model, and the generated dialogue model can be learned through a back propagation algorithm. The back propagation algorithm is defined as an algorithm that trains some given feedforward neural network for a given input pattern given a known classification. The back propagation algorithm may converge the loss function to a minimum given the training set.
In summary, the invention discloses a man-machine dialogue system, which is used for preprocessing an acquired current dialogue sentence to obtain a word vector sequence, inputting the word vector sequence into a sentence classification model to obtain a sentence function label of the current dialogue sentence, inputting the word vector sequence and the sentence function label into a generated dialogue model, and outputting an answer sentence of the current dialogue sentence by adopting a generated dialogue method. According to the invention, a certain proportion of question sentences are added in the sentence function labels of the training sentence classification model, so that the answer sentences output by the generated dialogue model comprise a certain proportion of question sentences, and in order to avoid the situation that the generated dialogue model outputs question sentences and wrongly outputs sentence patterns other than question sentences, a loss function is introduced, wherein the loss function represents the loss condition of the classification probability of the answer sentences output by the generated dialogue model relative to the classification probability of the reference answer sentences.
To further optimize the above embodiment, the answer sentence acquisition unit 304 may specifically be configured to:
encoding the word vector sequence by using an encoder to obtain a hidden state;
Decoding the hidden state by using a decoder to obtain an output sequence, and stopping decoding when the last character of the output sequence is a stop symbol;
and inputting the decoded output sequence and the sentence function tag into the generated dialogue model, and outputting the answer sentence of the current dialogue sentence by adopting a generated dialogue method.
Wherein the expression of the hidden state is as follows:
ht＝f(xt,ht-1);c＝ht (1)；
wherein h t is the hidden state, f is an activation function, the activation function is a nonlinear function, x t is a word vector in the t-th word vector sequence, h t-1 is a context vector of the word vector sequence corresponding to the time t-1, and c is a context vector.
In this embodiment, the loss function is a weighted average of the output statement loss factor and the classification loss factor, and the expression of the loss function is as follows:
Wherein loss is the loss function, alpha and beta represent weights of weighted averages, alpha and beta are both values between 0 and 1, and alpha+beta=1;
Wherein, the expression of the output statement loss factor is as follows:
Wherein lossA represents the output sentence loss factor, n represents the total length of the current dialogue sentence, i represents the independent variable in the summation process, represents the current dialogue sentence length in the answer sentence generation process, P represents the probability from the 1 st word to the n-1 st word for generating the n-th word, y n represents the n-th word in the answer sentence, y 1 represents the 1 st word in the answer sentence, and y n-1 represents the n-1 st word in the answer sentence;
the expression of the classification loss factor is as follows:
in the formula, k represents a sentence function class, i represents a variable in the summation process, the value is 1-k, Y represents a classification label corresponding to the answer sentence, U represents a classification label corresponding to the reference answer V, P (Y|k) represents the probability of obtaining the classification label Y by the answer sentence, and P (U|k) represents the probability of obtaining the classification label U by the generated dialogue model output reference answer V.
It should be specifically noted that, in the system embodiment, the specific working principle of each component is referred to the corresponding portion of the method embodiment, and will not be described herein.
Finally, it is further noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
In the present specification, each embodiment is described in a progressive manner, and each embodiment is mainly described in a different point from other embodiments, and identical and similar parts between the embodiments are all enough to refer to each other.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (6)
1. A human-machine conversation method, comprising:
acquiring a current dialogue sentence;
preprocessing the current dialogue sentence to obtain a word vector sequence;
inputting the word vector sequence into a pre-trained sentence classification model to obtain a sentence function label of the current dialogue sentence, wherein the sentence classification model is obtained by training the word vector sequence of the dialogue sentence as a training sample and the sentence function label as a label, and the questionable sentence proportion in the sentence function label of the sentence classification model is trained in a preset proportion interval;
Inputting the word vector sequence and the sentence function label into a pre-trained generation type dialogue model, and outputting an answer sentence of the current dialogue sentence by adopting a generation type dialogue method, wherein the generation type dialogue model adopts a back propagation algorithm to learn in the training process, and a loss function converges to a minimum value, and the loss function is used for representing the loss condition of the classification probability of the answer sentence output by the generation type dialogue model relative to the classification probability of a reference answer sentence;
The step of inputting the word vector sequence and the sentence function label into a pre-trained generation type dialogue model, and outputting the answer sentence of the current dialogue sentence by adopting a generation type dialogue method, which comprises the following steps: encoding the word vector sequence by using an encoder to obtain a hidden state; decoding the hidden state by using a decoder to obtain an output sequence, and stopping decoding when the last character of the output sequence is a stop symbol; inputting the output sequence and the sentence function label obtained by decoding into the generated dialogue model, and outputting the answer sentence of the current dialogue sentence by adopting a generated dialogue method;
wherein the expression of the hidden state is as follows:
ht＝f(xt,ht-1);c＝ht；
wherein h t is the hidden state, f is an activation function, the activation function is a nonlinear function, x t is a word vector in the t-th word vector sequence, h t-1 is a context vector of the word vector sequence corresponding to the time t-1, and c is a context vector.
2. The human-machine conversation method of claim 1 wherein the preprocessing the current conversation sentence to obtain a word vector sequence specifically includes:
word segmentation is carried out on the current dialogue sentence to obtain a word sequence;
And carrying out word embedding processing on each word in the word sequence, and converting each word into a word vector to obtain the word vector sequence.
3. The human-machine conversation method of claim 1 wherein the penalty function is a weighted average of output statement penalty factors and class penalty factors, the penalty function being expressed as:
Wherein loss is the loss function, alpha and beta represent weights of weighted averages, alpha and beta are both values between 0 and 1, and alpha+beta=1;
Wherein, the expression of the output statement loss factor is as follows:
Wherein lossA represents the output sentence loss factor, n represents the total length of the current dialogue sentence, i represents the independent variable in the summation process, represents the current dialogue sentence length in the answer sentence generation process, P represents the probability from the 1 st word to the n-1 st word for generating the n-th word, y n represents the n-th word in the answer sentence, y 1 represents the 1 st word in the answer sentence, and y n-1 represents the n-1 st word in the answer sentence;
the expression of the classification loss factor is as follows:
in the formula, k represents a sentence function class, i represents a variable in the summation process, the value is 1-k, Y represents a classification label corresponding to the answer sentence, U represents a classification label corresponding to the reference answer V, P (Y|k) represents the probability of obtaining the classification label Y by the answer sentence, and P (U|k) represents the probability of obtaining the classification label U by the generated dialogue model output reference answer V.
4. A human-machine conversation system, comprising:
A dialogue sentence acquisition unit for acquiring a current dialogue sentence;
the preprocessing unit is used for preprocessing the current dialogue sentence to obtain a word vector sequence;
The function tag obtaining unit is used for inputting the word vector sequence into a pre-trained sentence classification model to obtain a sentence function tag of the current dialogue sentence, wherein the sentence classification model is obtained by training the word vector sequence of the dialogue sentence as a training sample and the sentence function tag as a tag, and the question mark proportion in the sentence function tag for training the sentence classification model is within a preset proportion interval;
The answer sentence acquisition unit is used for inputting the word vector sequence and the sentence function label into a pre-trained generation type dialogue model, and outputting an answer sentence of the current dialogue sentence by adopting a generation type dialogue method, wherein the generation type dialogue model adopts a back propagation algorithm to learn in the training process, and a loss function is converged to a minimum value, and the loss function is used for representing the loss condition of the classification probability of the answer sentence output by the generation type dialogue model relative to the classification probability of a reference answer sentence;
The answer sentence acquisition unit is specifically configured to: encoding the word vector sequence by using an encoder to obtain a hidden state; decoding the hidden state by using a decoder to obtain an output sequence, and stopping decoding when the last character of the output sequence is a stop symbol; inputting the output sequence and the sentence function label obtained by decoding into the generated dialogue model, and outputting the answer sentence of the current dialogue sentence by adopting a generated dialogue method;
wherein the expression of the hidden state is as follows:
ht＝f(xt,ht-1);c＝ht；
wherein h t is the hidden state, f is an activation function, the activation function is a nonlinear function, x t is a word vector in the t-th word vector sequence, h t-1 is a context vector of the word vector sequence corresponding to the time t-1, and c is a context vector.
5. The human-machine conversation system of claim 4 wherein the preprocessing unit is specifically configured to:
word segmentation is carried out on the current dialogue sentence to obtain a word sequence;
And carrying out word embedding processing on each word in the word sequence, and converting each word into a word vector to obtain the word vector sequence.
6. The human-machine conversation system of claim 4 wherein the penalty function is a weighted average of output sentence penalty factors and class penalty factors, the penalty function being expressed as:
Wherein loss is the loss function, alpha and beta represent weights of weighted averages, alpha and beta are both values between 0 and 1, and alpha+beta=1;
Wherein, the expression of the output statement loss factor is as follows:
Wherein lossA represents the output sentence loss factor, n represents the total length of the current dialogue sentence, i represents the independent variable in the summation process, represents the current dialogue sentence length in the answer sentence generation process, P represents the probability from the 1 st word to the n-1 st word for generating the n-th word, y n represents the n-th word in the answer sentence, y 1 represents the 1 st word in the answer sentence, and y n-1 represents the n-1 st word in the answer sentence;
the expression of the classification loss factor is as follows:
in the formula, k represents a sentence function class, i represents a variable in the summation process, the value is 1-k, Y represents a classification label corresponding to the answer sentence, U represents a classification label corresponding to the reference answer V, P (Y|k) represents the probability of obtaining the classification label Y by the answer sentence, and P (U|k) represents the probability of obtaining the classification label U by the generated dialogue model output reference answer V.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110200723.6A CN112883179B (en) | 2021-02-23 | 2021-02-23 | Man-machine conversation method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110200723.6A CN112883179B (en) | 2021-02-23 | 2021-02-23 | Man-machine conversation method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112883179A CN112883179A (en) | 2021-06-01 |
| CN112883179B true CN112883179B (en) | 2024-05-24 |
Family
ID=76053827
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110200723.6A Active CN112883179B (en) | 2021-02-23 | 2021-02-23 | Man-machine conversation method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112883179B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019198386A1 (en) * | 2018-04-13 | 2019-10-17 | 国立研究開発法人情報通信研究機構 | Request rephrasing system, method for training of request rephrasing model and of request determination model, and conversation system |
| CN110990543A (en) * | 2019-10-18 | 2020-04-10 | 平安科技（深圳）有限公司 | Intelligent conversation generation method and device, computer equipment and computer storage medium |
-
2021
- 2021-02-23 CN CN202110200723.6A patent/CN112883179B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019198386A1 (en) * | 2018-04-13 | 2019-10-17 | 国立研究開発法人情報通信研究機構 | Request rephrasing system, method for training of request rephrasing model and of request determination model, and conversation system |
| CN110990543A (en) * | 2019-10-18 | 2020-04-10 | 平安科技（深圳）有限公司 | Intelligent conversation generation method and device, computer equipment and computer storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 赵宇晴 ; 向阳 ; .基于分层编码的深度增强学习对话生成.计算机应用.2017,(10),全文. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112883179A (en) | 2021-06-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Zandie et al. | Emptransfo: A multi-head transformer architecture for creating empathetic dialog systems | |
| CN108595436B (en) | Method and system for generating emotional dialogue content and storage medium | |
| CN108829662A (en) | A kind of conversation activity recognition methods and system based on condition random field structuring attention network | |
| Merdivan et al. | Dialogue systems for intelligent human computer interactions | |
| CN110069612B (en) | Reply generation method and device | |
| CN112115247A (en) | Personalized dialogue generation method and system based on long-time and short-time memory information | |
| CN111241807A (en) | Machine reading understanding method based on knowledge-guided attention | |
| CN113360618B (en) | Intelligent robot dialogue method and system based on offline reinforcement learning | |
| CN112765333B (en) | Automatic dialogue generation method and system based on emotion and prompt word combination | |
| CN112559706B (en) | Training method of dialogue generating model, dialogue method, device and storage medium | |
| CN111782788A (en) | Automatic emotion reply generation method for open domain dialogue system | |
| CN110597968A (en) | Reply selection method and device | |
| Elshaer et al. | Transfer learning from sound representations for anger detection in speech | |
| CN109308316B (en) | Adaptive dialog generation system based on topic clustering | |
| Luo et al. | EmotionX-DLC: self-attentive BiLSTM for detecting sequential emotions in dialogue | |
| CN116049387A (en) | Short text classification method, device and medium based on graph convolution | |
| CN115935975A (en) | Controllable-emotion news comment generation method | |
| CN116561265A (en) | Personalized dialogue generation method, model training method and device | |
| Carcassi et al. | Monotone quantifiers emerge via iterated learning | |
| Bunescu et al. | Learning to Surprise: A Composer-Audience Architecture. | |
| CN114328866A (en) | Strong anthropomorphic intelligent dialogue robot with smooth and accurate response | |
| CN112417118B (en) | Dialog generation method based on marked text and neural network | |
| CN111046157B (en) | Universal English man-machine conversation generation method and system based on balanced distribution | |
| CN110851580B (en) | Personalized task type dialog system based on structured user attribute description | |
| CN112883179B (en) | Man-machine conversation method and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |