CN109522545A - A kind of appraisal procedure that more wheels are talked with coherent property amount - Google Patents
A kind of appraisal procedure that more wheels are talked with coherent property amount Download PDFInfo
- Publication number
- CN109522545A CN109522545A CN201811181214.8A CN201811181214A CN109522545A CN 109522545 A CN109522545 A CN 109522545A CN 201811181214 A CN201811181214 A CN 201811181214A CN 109522545 A CN109522545 A CN 109522545A
- Authority
- CN
- China
- Prior art keywords
- vector
- semantic
- dialogue
- language
- wheel
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/30—Semantic analysis
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Machine Translation (AREA)
Abstract
The invention discloses a kind of appraisal procedures that more wheels are talked with coherent property amount, its main feature is that take turns dialog text as input more, using layering attention mechanism, respectively in the whole level of single discourse level and more wheel dialogues, the semantic information and intent information of dialogue are merged, realizes that more wheels talk with the automatic assessment of coherent property amount.Training speed is fast compared with prior art by the present invention, it is high to analyze accuracy rate, it does not need to extract entity from text, avoid the propagation that entity extracts error, it is especially suitable for extensive and real-time corpus, in conjunction with semantic information and intent information that dialogue is contained, effectively assesses the coherent property amount of more wheel dialogues automatically, the more wheel dialog generation systems of guidance generate the dialog text of more high quality, and dialog generation system is promoted faster and better to develop.
Description
Technical field
It is especially a kind of that more wheels are talked with using deep learning the present invention relates to internet deep learning model technical field
The appraisal procedure of coherent property amount.
Background technique
In recent years, the chats such as interactive system, such as apple Siri, the small ice of Microsoft/customer service robot (Chatbot) is more
More to often have in daily life.Dialogue (Dialogue) is the essential information interactive mode in mankind's social activity, including
Television interview, question answer dialog, wechat chat etc., a key core technologies in human-computer dialogue are that more wheel dialogues generate
(Multi-turn Dialogue Generation), this is mark and the natural language processing of Artificial Intelligence Development level
The research hotspot in field, the concern by more and more scientific research personnel.The quality evaluation that more wheel dialogues generate generallys use following
Two ways: 1) it is objective to borrow BLEU and ROUGE in other text generation tasks (such as machine translation or autoabstract etc.) etc.
Assessment measurement is seen, word-based overlapping is measured in this objective automatic assessment, has ignored dialogue naturally with diversity and interactivity
Feature is poorly suited for use in the quality evaluation of dialogue；2) manual evaluation, although this assessment mode is with higher accurate
Property, but artificial and time cost is big, can not be applicable in the quality evaluation of extensive and real-time dialogue.Usually occur in view of talking with
Two sides or it is multi-party between, the content qualities height that more wheel dialogues generate depends on the Text Coherence (Text between the dialogue of front and back
Coherence), if lacking continuity, dialogue is often made to be easily trapped into logical miss, causes dialogue that can not continue.Therefore,
The continuity assessment of either objective automatic assessment mode or manual evaluation mode, more wheel dialogues is that conversational quality is assessed
One key index.
Text Coherence assessment is mostly used based on physical grid (Entity Grid) or entity sequence (Entity
Sequence method), this method extract the entities such as name, place name in text (Entity) and its grammatical roles (such as first
Subject, predicate etc.), and inserted in grid node (i.e. lattice point), the conversion of the substantial grammar role between adjacent sentence then at
Then line between lattice point passes through artificial extraction feature or utilization convolutional neural networks (Convolutional Neural
Network, CNN) method carry out text continuity assessment.
The example of the dialogue of wheel more than two of following table 1, is further elaborated problem of the existing technology:
Table takes turns the example of dialogue more 1 two
1 middle left and right two of table column are coherent and incoherent more wheel dialogues example (the wheel number of dialogue is all 6) respectively,
Middle H indicates that the mankind (Human) in dialogue, C indicate chat robots (Chatbot).The sentence of each H or C is exactly one
Language (Utterance), such as H1 and C1 are a language." I wants that the wife to me buys to the language H1 on the left side in table 1
Flower " shows that speaker wants to buy colored wish, and it is to recommend suitable flower that language C1, which is reasonably replied,.The intention of dialogue
(Intention) information is indicated usually using conversation activity (Dialog Act), and the conversation activity of left side H1 marks just here
It is instruction, there is reasonable dialogue in coherent more wheels dialogue and be intended to conversion, is i.e. conversation activity is converted, and guidance dialogue has logically
Around theme and it is intended to expansion.
Existing dialogue continuity quality assessment techniques have following deficiency: (1) having ignored dialogue naturally has language diversity
The characteristics of with more wheel interactivity, therefore the accuracy of conversational quality assessment is lower；(2) lack the whole semantic letter of more wheel dialogues
Breath；(3) lack the intent information contained in more wheel dialogues；(4) depth for lacking more wheel dialog semantics information and intent information is melted
It closes；(5) artificial and time cost is big, can not be applicable in the quality evaluation of extensive and real-time dialogue；(6) based on physical grid or
The method of sequence relies on the performance that entity extracts, and extracting error will have a direct impact on the performance of subsequent step.
Summary of the invention
The purpose of the present invention is design a kind of to more wheel dialogue continuity quality evaluations in view of the deficiencies of the prior art
Method, to take turns dialog text as input, using layering attention mechanism, respectively in single discourse level and more wheel dialogue more
Whole level merges the semantic information and intent information of dialogue, realizes that more wheels talk with the automatic assessment of coherent property amount.By right
More wheels talk with the automatic assessment of coherent property amount, can instruct to take turns the dialogue text that dialog generation system generates more high quality more
This, promotes dialog generation system faster and better to develop, does not need to extract entity from text, avoids entity and extracts error
It propagates, is not necessarily to artificial design features, save cost, training speed is fast, is especially suitable for extensive and real-time corpus, from individually right
The continuity of the more wheel dialogues of the semantic information and intent information that words and whole dialogue level contain respectively in connection with dialogue, effectively assessment
Quality substantially increases analysis accuracy rate.
The object of the present invention is achieved like this: a method of to more wheel dialogue continuity quality evaluations, its main feature is that
Layering semantic modeling is carried out to more wheel dialogues using deep learning, talks with word order after the semantic vector expression for obtaining each language
Column are modeled, and then obtain the whole semantic information for more taking turns dialogue, and continuity quality evaluation carries out in the steps below:
Step 1: using the term vector of pre-training, being initialized as term vector for each word in single language by tabling look-up,
And with the term vector sequence of single language to input, single language is carried out using deep learning model (such as LSTM model) semantic
Study, obtains the semantic vector of each language.
Step 2: the conversation activity type of single language, which is initialized as conversation activity vector, to be indicated, then using attention
Power mechanism merges the single language semantic vector of conversation activity vector sum of single language, obtains the semantic of single language and is intended to melt
Resultant vector.
Step 3: for more wheel dialogues, word order is arranged, and semantic using single language is intended to fusion vector, constructs more wheels
The semantic of dialogue is intended to fusion sequence vector, and is input with the semantic fusion sequence vector that is intended to of more wheel dialogues, utilizes depth
Learning model (such as LSTM model) learns more wheel dialogues, obtains the whole vector for more taking turns dialogue.
Step 4: by the language semantic vector sequence of more wheel dialogues, as the defeated of deep learning model (such as LSTM model)
Enter, semantic modeling is carried out to more wheel dialogues, obtains the whole semantic vector for more taking turns dialogue, and with the conversation activity sequence of more wheel dialogues
It is classified as input, the whole intention that more wheels are talked with is learnt using deep learning model (such as CNN model), take turns is obtained more and talks with
Whole be intended to vector.
Step 5: using attention mechanism, and the whole semantic vector sums for merging more wheel dialogues are integrally intended to vector, obtains pair
The whole semantic intention fusion vector of words.
Step 6: it is comprehensive that the dialogue vector progress that dialog semantics information and intent information obtain is merged in different level by respectively
It closes (such as simple concatenation), the overall coherence vector talked with, and is input with the overall coherence vector of dialogue, using beating
Divide function, obtain the continuity quality evaluation scores for more taking turns dialogue, then utilizes cross entropy loss function calculating training data
Continuity score error updates model parameter by backpropagation and carries out model training.
Step 7: more wheels that more wheels dialogue to be assessed and its conversation activity sequence inputting have been trained are talked with into coherent property
Automatic assessment models are measured, the continuity quality evaluation to engage in the dialogue and prediction score.
The present invention has training speed fast compared with prior art, and analysis accuracy rate is high, is not necessarily to artificial design features, saves
Cost does not need to extract entity from text, avoids the propagation that entity extracts error, is especially suitable for extensive and real-time language
Material, using layering attention mechanism, respectively in the whole level of single discourse level and more wheel dialogues, the language contained in conjunction with dialogue
Adopted information and intent information, effectively the coherent property amount of the more wheel dialogues of automatic assessment, instructs more wheel dialog generation systems to generate
Higher-quality dialog text promotes dialog generation system faster and better to develop.
Detailed description of the invention
Fig. 1 is operation of the present invention flow chart.
Specific embodiment
Below with the process of specific implementation, condition and experimental method, the present invention is described in further detail, wherein
Related technical term is defined as follows:
More wheel dialogues (Multi-turn Dialogue): more wheel dialogues are made of orderly dialogue more than two-wheeled or two-wheeled,
It include a language (Utterance), i.e., the content of text that single interlocutor's single is said in every wheel dialogue.As shown in table 2 below
More wheels dialogue altogether include 6 wheel dialogue, i.e. 6 language, " I wants that the wife to me buys a little flowers to H1." it is the first of more wheel dialogues
A language.
Conversation activity (Dialog Act): the intention of each dialogue is reacted in conversation activity, shares 42 classes, including is stated, doubted
It asks, indicate, advocate, explain.The conversation activity type of each language as shown in table 2 below, wherein " red rose wants how many to language H2
Money? " conversation activity type be query, express the doubt of interlocutor, and " one 5 yuan, ten 40 yuan of language C2." then pass through
The price for stating red rose, solves above-mentioned query.
Table 2 six takes turns the behavior type example of dialog text and every wheel dialogue
Term vector (Word Vector): each word in text is indicated using the vector of low-dimensional continuity numerical value.Term vector
It is obtained by the way of pre-training from corpus.
Deep learning model (Deep Learning Model): deep learning model is divided into three parts: input layer, hidden
Hide layer and output layer.Wherein, hidden layer can be expanded into multilayered structure, and the neuron between adjacent two layers is connected with each other, layer
Intrinsic nerve member does not connect.Common deep learning model has convolutional neural networks model (Convolutional Neural
Network, CNN), Recognition with Recurrent Neural Network model (Recurrent Neural Network, RNN), shot and long term remember nerve net
Network model (Long-Short Term Memory, LSTM) etc..
Refering to attached drawing 1, the input in the present invention is that the more wheels comprising N number of language u talk with D, is expressed as D={ u1,
u2,…,ui,…,uN, i=[1,2 ..., N], wherein uiRepresent i-th of language.Each language u includes n word, then language u table
It is shown as character string sequence u={ w1,w2,…,wj,…,wn, j=[1,2 ..., n].The whole semanteme of more wheel dialogues in order to obtain
Vector indicates, it is necessary first to which the semantic vector for obtaining single language indicates.
The present invention carries out layering semantic modeling to more wheel dialogues using deep learning, is obtaining the semantic vector of each language
Talk with word order column after expression to be modeled, and then obtain the whole semantic information for more taking turns dialogue, continuity quality evaluation is pressed
State step progress:
Step 1: in single language layer, using deep learning model (such as CNN, RNN or LSTM model), to more wheel dialogue D
In each language u carry out semantic modeling, the semantic vector for obtaining single language indicates that detailed process is described as follows:
Input: the more wheels comprising N number of language u talk with D, term vector dictionary, deep learning model and relevant parameter；
Output: the semantic vector of single language indicates；
Process: step a1: each language generally comprises multiple words, logical first in order to carry out semantic modeling to single language
It crosses in the term vector dictionary of pre-training and tables look-up, converting its corresponding term vector for each word in single language indicates.Words
Language u includes n word, i.e. character string sequence u={ w1,w2,…,wj,…,wn, j=[1,2 ..., n] passes through pre-training of tabling look-up
Term vector dictionary, by each word w in language ujIt is initialized as term vector, obtains the term vector sequence s={ x of language u1,
x2,…,xj,…,xn, j=[1,2 ..., n], wherein xjIndicate the term vector of j-th of word of language u.
Step a2: it is input with the term vector sequence s of language u, utilizes deep learning model (such as CNN, RNN or LSTM mould
Type) semantic modeling is carried out to language u, the semantic vector for obtaining single language indicates.By taking LSTM model as an example, process description is such as
Under:
(1), the term vector sequence s={ x of language is inputted1,x2,…,xj,…,xn}；
(2), to each term vector in term vector sequence, following formula successively are pressed using the replicated blocks in LSTM model
A~e is handled:
ft=σ (Wfxt+Ufht-1+bf) (a)
it=σ (Wixt+Uiht-1+bi) (b)
ot=σ (Woxt+Uoht-1+bo) (c)
ct=ft*ct-1+it*tanh(Wcxt+Ucht-1+bc) (d)
ht=ot*tanh(ct) (e)
Wherein, xtFor t-th of term vector (t=[1,2 ..., n]) in term vector sequence s, as t time step input to
Amount；ft, it, otRespectively indicate forgetting door, input gate and the out gate of t time step；Wf,Wi,Wo,WcAnd Uf,Ui,Uo,UcIt is all power
Weight parameter, bf,bi,bo,bcIt is all bias term；σ is S type curve activation primitive (sigmoid), and tanh is hyperbolic tangent function；*
Indicate corresponding element multiplication (Element-wise Multiplication) operation of two vectors；htFor hiding for t time step
Layer state.
(3) the hiding layer state h of the last one time step n is exportedn, as language u semantic vector indicate h.The first step
The rapid semantic information that single language is only obtained by the semantic modeling that deep learning model carries out each language, does not account for
To the intent information of language.
Step 2: in single language layer, the semantic vector of the single language obtained using step 1 is indicated, using attention
Mechanism merges the intent information of single language, and output obtains the semantic of single language and is intended to fusion vector, and detailed process describes such as
Under:
Input: the semantic vector expression for the single language that step 1 obtains, the corresponding conversation activity type of language, depth
Practise model and relevant parameter；
Output: the semantic of single language is intended to fusion vector；
Process: step b1: after the conversation activity type of language is initialized as the expression of conversation activity vector, using attention
Mechanism, the semantic vector for merging the single language that the conversation activity vector sum step 1 of single language obtains indicate, obtain single
The semantic of language is intended to fusion vector, and process description is as follows:
(1), it is obtained corresponding to each conversation activity type for 42 kinds of conversation activity types by random initializtion
Fixed dimension vector indicate (by 200 dimension for), constitute conversation activity vector dictionary E ∈ R42×200。
(2), its conversation activity type is initialized as by the conversation activity vector dictionary E that tables look-up for single language u
Vector vda。
(3), using attention mechanism, the conversation activity vector v of single language is mergeddaThe semantic vector obtained with step 1
It indicates h, obtains the semantic of language and be intended to fusion vector hda, by taking the transformation of 5 sublinears as an example, calculated by following formula i~k:
zi=Wih+bi(i=[1,2 ..., 5]) (i)
Wherein, WiFor weight, biFor bias term, ziFor the result of the i-th linear transformation to semantic vector h；αiFor scalar
Value indicates to utilize conversation activity vector vdaThe z being calculatediWeighted value；hdaI.e. the semantic of language is intended to fusion vector, is pair
Each ziWeighted sum as a result, having merged the semantic information and intent information of language u.
Step 3: for more wheel dialogues, word order is arranged, and the semantic of the language obtained using step 2 is intended to fusion vector,
The semantic of the more wheel dialogue entirety of building is intended to fusion sequence vector, using deep learning model (such as LSTM), obtains more wheel dialogues
Whole vector indicate.The coherent property amount of more wheel dialogues is to carry out continuity assessment, therefore, this hair to more the whole of wheel dialogue
For more wheel dialogues, word order arranges bright third step, and the semantic of the language obtained using step 2 is intended to fusion vector hda,
The semantic of the more wheel dialogue entirety of building is intended to fusion sequence vector, using deep learning model (by taking LSTM model as an example), to more
Word order column are modeled if wheel dialogue, and the whole vector for obtaining more taking turns dialogue indicates hd, detailed process is described as follows:
Input: the semantic of each language that step 2 obtains is intended to fusion vector hdaThe semantic of more wheels dialogue constituted is intended to
Merge sequence vector, deep learning model and relevant parameter；
Output: the whole vector of more wheel dialogues indicates；
Process: step c1: for more wheel dialogue D, word order arranges { u1,u2,…,ui,…,uN, it is obtained using step 2
The semantic of single language is intended to fusion vector hda, construct the semantic of more wheel dialogues and be intended to fusion sequence vector { hda1,hda2,...,
hdai,...,hdaN, i=[1,2 ..., N], wherein hdaiIndicate i-th of language u in dialogueiSemantic be intended to fusion vector.
Step c2: it is input with the semantic fusion sequence vector that is intended to of more wheel dialogues, using LSTM model to more wheel dialogues
It is modeled, the whole vector for obtaining more taking turns dialogue indicates hd, the process description of LSTM model is with reference to step a2。
Step 4: word order column and conversation activity sequences to more wheel dialogues, carry out respectively deep learning (such as LSTM or
CNN it) models, the whole semantic vector table for obtaining mostly wheel dialogue, which shows, and entirety is intended to vector indicates.Above-mentioned steps two and step 3
It is to merge semantic and intent information using attention mechanism in single discourse level, is then talked with using fused sequence
Whole indicate.In order to merge the semantic information of dialogue using attention mechanism in the whole level of more wheel dialogues and be intended to letter
Breath, it is necessary first to whole semantic modeling be carried out to more wheel dialogues and be intended to model, therefore, four steps of the invention uses deep
It spends learning model (such as CNN, RNN or LSTM model), to more wheel dialogues, word order column and conversation activity sequence model respectively, obtain
Whole semantic vector table to more wheel dialogues shows and the whole vector that is intended to indicates that detailed process is described as follows:
Input: the semantic vector for the single language that step 1 obtains constitutes the semantic vector sequence of more wheel dialogues, more wheels pair
The conversation activity sequence of words, deep learning model and relevant parameter；
Output: the whole semantic vector table of more wheel dialogues shows and the whole vector that is intended to indicates；
Process: step d1: for more wheel dialogue D, word order arranges { u1,u2,…,ui,…,uN, it is obtained often using step 1
The semantic vector of a language indicates h, constitutes the language semantic vector sequence s of Dh={ h1,h2,...,hi,...,hN, i=[1,
2 ..., N], as the input of LSTM model, to more wheel dialogues carry out semantic modelings obtain taking turns more dialogue it is whole it is semantic to
Amount indicates hsem, LSTM model detailed process is with reference to step a2；
Step d2: with it is more wheel dialogue conversation activity sequences be input, using deep learning model (such as CNN, RNN or
LSTM model), the whole of more wheel dialogues is intended to model, the whole vector that is intended to for obtaining more taking turns dialogue indicates that process is such as
Under:
(1), for more wheel dialogue D, word order arranges { u1,u2,…,ui,…,uNAnd each language conversation activity class
Type obtains the conversation activity sequence s of Dda={ da1,da2,...,dai,...,daN, i=[1,2 ..., N], wherein daiIt is i-th
A language uiConversation activity type.
(2), using deep learning model, conversation activity sequence is modeled, obtains the whole conversation activity table of dialogue
Show, by taking CNN model as an example, process description is as follows:
(I) it is directed to conversation activity sequence sda={ da1,da2,...,dai,...,daN, to each conversation activity dai(i
=[1,2 ..., N]), by tabling look-up in the conversation activity vector dictionary E that constructs in step 2, obtain its corresponding vector table
Show, then it is the sequence vector { v comprising N number of vector that conversation activity is Sequence Transformed1,v2,...,vi,...,vN, as CNN model
Input.
(II) vector v of input is carried out convolution operation by convolutional layer, is calculated and is carried out by following formula f:
cj=f (uTvj-k+1:j+b) (f)
Wherein, f is nonlinear function, as tanh activation primitive (hyperbolic tangent), S type curve activate
Function (sigmoid) etc., b are bias term, and u indicates that window size is the convolution filter of k, v(j-k+1:j)Indicate conversation activity sequence
The vector of jth-k+1 conversation activities to j-th of conversation activity (total k conversation activity) in column indicates.Filter quantity is
M obtains m characteristic value sequence C={ C in the case where size is the sliding window of k1,C2,…,Cm, wherein each characteristic value sequence Ci
=[c1,c2,…,cN-k+1], Ci∈RN-k+1, i=[1 ..., m].
(III) in order to extract important feature and control the consistency of output, pond layer carries out this m characteristic value sequence C
Pondization operation obtains the feature vector of m dimensionThe whole vector that is intended to as more wheel dialogues indicates vd。
Step 5: talking with whole level, indicated, obtained using two vectors obtained in attention mechanism fusion steps four
Whole semantic intention fusion vector must be talked with.For the semantic information talked in the whole level fusion of more wheel dialogues and it is intended to letter
Breath, the 5th step of the invention are using attention mechanism, fusion steps d1Obtained in whole semantic vector table show hsemAnd step
Rapid d2Obtained in be integrally intended to vector indicate vd, obtain talking with whole semantic intention fusion vectorDetailed process describes such as
Under:
Input: the whole semantic vector table of more wheel dialogues shows hsem, the entirety of more wheel dialogues, which is intended to vector, indicates vd, depth
Learning model and relevant parameter；
Output: the entirety of more wheel dialogues is semantic to be intended to fusion vector；
Process: step e1: attention mechanism is used, the whole semantic vector table for merging more wheel dialogues shows hsemWith whole meaning
Figure vector indicates vd, by taking the transformation of 5 sublinears as an example, specific calculate is carried out by following formula g, l and n:
gi=Wdihsem+bdi(i=[1,2 ..., 5]) (g)
Wherein, WdiFor weight, bdiFor bias term, giFor to semantic vector hsemI-th linear transformation result；βiFor
Scalar value indicates to indicate v using conversation activitydThe g being calculatediWeighted value,It is then to each giWeighted sum knot
Fruit has merged the whole semantic intention fusion vector of dialogue of more wheel dialog semantics information and intent information.
Step 6: above-mentioned these steps using layering attention mechanism, single language and more wheel dialogues it is whole this two
A level merges the semantic information and intent information of dialogue respectively, and two kinds of whole vectors for obtaining more taking turns dialogue indicate hdWith
6th step of the invention is that both comprehensive whole vectors indicate, is utilized scoring functions (such as softmax, sigmoid function)
The continuity quality evaluation score talked with, wherein the training that more wheels talk with the coherent automatic assessment models of property amount is to pass through
Cross entropy loss function and backpropagation (backpropagation) Lai Jinhang's, detailed process is described as follows:
Input: the whole vector for more wheels dialogue that step 3 obtains indicates hdThe entirety of the more wheels dialogue obtained with step 5
Semanteme is intended to fusion vectorDeep learning model and relevant parameter；
Output: the continuity quality evaluation score of more wheel dialogues；
Process: step f1: step 3 and step 5 are merged into dialog semantics information and intent information in different level respectively
Two obtained dialogue vectors indicate hdWithCarry out comprehensive (by taking simple concatenation as an example), the overall coherence talked with to
Amount:Wherein,Indicate concatenation.
Step f2: with vector hcMore wheels are talked with using scoring functions (such as softmax, sigmoid function) for input
Overall coherence property amount is given a mark, the continuity probability distribution h talked withs.By taking softmax function as an example, overall calculation
It is carried out by following formula m:
hs=softmax (Wshc+bs) (m)
Wherein, WsFor weight, bsFor bias term, for will more wheel dialogues overall coherence vector hcIt is mapped to two-dimentional mesh
Space is marked, bivector h is obtainedo, as the input of softmax function, softmax function specifically calculate by following formula p into
Row:
Wherein,Indicate vector hsJth dimension value, j=[0,1],WithReal number value between 0-1, Xiang Jiahe
It is 1, whereinIndicate the continuity probability of dialogue, that is, the continuity quality evaluation score talked with.
Step f3: the training of the automatic assessment models of coherent property amount of more wheel dialogues is that have the learning process of supervision,
To after the continuity quality evaluation score of more wheel dialogues, for training data T, following public affairs are pressed in the calculating of cross entropy loss function
Formula q is carried out:
Wherein, θ is the parameter sets for needing training to update in model, including weight involved in each step and bias term
The conversation activity vector dictionary E constructed in parameter and step 2, | T | indicate the dialogue number that training data is concentrated, yiIt represents
Training data concentrates the continuity score of the dialogue of wheel more than i-th, is 1 if more wheel dialogues are coherent dialogue, otherwise for
0。
In order to which the automatic assessment models of coherent property amount of more wheel dialogues are trained and are learnt, this step is to above-mentioned loss
Function carry out derivation, by continuity error carry out backpropagation, update model parameter, until loss (θ) be less than predetermined threshold τ,
Then stop updating, completes model training.
Step 7: more wheels that more wheels dialogue to be assessed and its conversation activity sequence inputting have been trained are talked with into coherent property
Automatic assessment models are measured, predict the continuity quality evaluation score of dialogue.7th step of the invention is by more wheels to be assessed
Dialogue and its conversation activity sequence inputting have trained obtained more wheels to talk with the coherent automatic assessment models of property amount into step 6
In, it obtains characterization and talks with the real number value (between [0-1]) of coherent degree, and export this real number value and talk with as more wheels
Continuity quality evaluation score.
Above only the present invention is further illustrated, and not to limit this patent, all is equivalence enforcement of the present invention,
It is intended to be limited solely by within the scope of the claims of this patent.
Claims (1)
1. a kind of appraisal procedure that more wheels are talked with coherent property amount, it is characterised in that carried out using deep learning to more wheel dialogues
Layering semantic modeling, dialogue word order column are modeled after the semantic vector expression for obtaining each language, and then obtain more wheels pair
The whole semantic information of words, continuity quality evaluation carry out in the steps below:
Step 1: using the term vector of pre-training, being initialized as term vector for each word in single language by tabling look-up, and with
The term vector sequence of single language is input, carries out semantic study to single language using deep learning model, obtains each words
The semantic vector of language；
Step 2: the conversation activity type of single language, which is initialized as conversation activity vector, to be indicated, attention machine is then used
System, merges the semantic vector of the single language of conversation activity vector sum of single language, obtains the semantic of single language and is intended to fusion
Vector；
Step 3: for more wheel dialogues, word order is arranged, and semantic using single language is intended to fusion vector, constructs more wheel dialogues
It is semantic be intended to fusion sequence vector, and be input with the semantic fusion sequence vectors that are intended to of more wheel dialogues, utilize deep learning
Model learns more wheel dialogues, obtains the whole vector for more taking turns dialogue；
Step 4: the language semantic vector sequence of more wheel dialogues carries out more wheel dialogues as the input of deep learning model
Semantic modeling obtains the whole semantic vector for more taking turns dialogue, and is input with the conversation activity sequence of more wheel dialogues, using depth
Learning model learns the whole intention of more wheel dialogues, obtains the whole intention vector for more taking turns dialogue；
Step 5: using attention mechanism, and the whole semantic vector sum for merging more wheel dialogues is integrally intended to vector, obtains talking with whole
Body semanteme is intended to fusion vector；
Step 6: integrating the dialogue vector that dialog semantics information and intent information obtain is merged in different level respectively,
The overall coherence vector talked with, and be input with the overall coherence vector of dialogue, using scoring functions, obtain take turns more
Then the continuity quality evaluation score of dialogue calculates the continuity score error of training data using cross entropy loss function,
Model parameter, which is updated, by backpropagation carries out model training；
Step 7: more wheels that more wheels dialogue to be assessed and its conversation activity sequence inputting have been trained are talked with into coherent property amount certainly
Dynamic assessment models, the continuity quality evaluation to engage in the dialogue and prediction score.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811181214.8A CN109522545B (en) | 2018-10-11 | 2018-10-11 | A kind of appraisal procedure that more wheels are talked with coherent property amount |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811181214.8A CN109522545B (en) | 2018-10-11 | 2018-10-11 | A kind of appraisal procedure that more wheels are talked with coherent property amount |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109522545A true CN109522545A (en) | 2019-03-26 |
| CN109522545B CN109522545B (en) | 2019-08-23 |
Family
ID=65770262
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811181214.8A Active CN109522545B (en) | 2018-10-11 | 2018-10-11 | A kind of appraisal procedure that more wheels are talked with coherent property amount |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109522545B (en) |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110162779A (en) * | 2019-04-04 | 2019-08-23 | 北京百度网讯科技有限公司 | Appraisal procedure, device and the equipment of quality of case history |
| CN110188190A (en) * | 2019-04-03 | 2019-08-30 | 阿里巴巴集团控股有限公司 | Talk with analytic method, device, server and readable storage medium storing program for executing |
| CN111159356A (en) * | 2019-12-31 | 2020-05-15 | 重庆和贯科技有限公司 | Knowledge graph construction method based on teaching content |
| CN111241263A (en) * | 2020-04-24 | 2020-06-05 | 支付宝(杭州)信息技术有限公司 | Text generation method and device and electronic equipment |
| CN111428470A (en) * | 2020-03-23 | 2020-07-17 | 北京世纪好未来教育科技有限公司 | Text continuity judgment method, text continuity judgment model training method, electronic device and readable medium |
| CN111460115A (en) * | 2020-03-17 | 2020-07-28 | 深圳市优必选科技股份有限公司 | Intelligent man-machine conversation model training method, model training device and electronic equipment |
| US20200327582A1 (en) * | 2019-04-15 | 2020-10-15 | Yandex Europe Ag | Method and system for determining result for task executed in crowd-sourced environment |
| CN112417112A (en) * | 2020-11-10 | 2021-02-26 | 中山大学 | Open domain dialogue system evaluation method based on graph characterization enhancement |
| CN112487158A (en) * | 2020-11-06 | 2021-03-12 | 泰康保险集团股份有限公司 | Problem positioning method and device for multi-turn conversation |
| US11727329B2 (en) | 2020-02-14 | 2023-08-15 | Yandex Europe Ag | Method and system for receiving label for digital task executed within crowd-sourced environment |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9348816B2 (en) * | 2008-10-14 | 2016-05-24 | Honda Motor Co., Ltd. | Dialog coherence using semantic features |
| CN106599196A (en) * | 2016-12-14 | 2017-04-26 | 竹间智能科技（上海）有限公司 | Artificial intelligence conversation method and system |
-
2018
- 2018-10-11 CN CN201811181214.8A patent/CN109522545B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9348816B2 (en) * | 2008-10-14 | 2016-05-24 | Honda Motor Co., Ltd. | Dialog coherence using semantic features |
| CN106599196A (en) * | 2016-12-14 | 2017-04-26 | 竹间智能科技（上海）有限公司 | Artificial intelligence conversation method and system |
Non-Patent Citations (2)
| Title |
|---|
| AMRUTA PURANDARE ET AL: "Analyzing Dialog Coherence Using Transition Patterns in Lexical and Semantic Features", 《PROCEEDINGS OF THE TWENTY-FIRST INTERNATIONAL FLAIRS CONFERENCE》 * |
| 孙立茹 等: "基于深度学习的生成式聊天机器人算法综述", 《电脑知识与技术》 * |
Cited By (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110188190A (en) * | 2019-04-03 | 2019-08-30 | 阿里巴巴集团控股有限公司 | Talk with analytic method, device, server and readable storage medium storing program for executing |
| CN110162779A (en) * | 2019-04-04 | 2019-08-23 | 北京百度网讯科技有限公司 | Appraisal procedure, device and the equipment of quality of case history |
| US11727336B2 (en) * | 2019-04-15 | 2023-08-15 | Yandex Europe Ag | Method and system for determining result for task executed in crowd-sourced environment |
| US20200327582A1 (en) * | 2019-04-15 | 2020-10-15 | Yandex Europe Ag | Method and system for determining result for task executed in crowd-sourced environment |
| CN111159356A (en) * | 2019-12-31 | 2020-05-15 | 重庆和贯科技有限公司 | Knowledge graph construction method based on teaching content |
| CN111159356B (en) * | 2019-12-31 | 2023-06-09 | 重庆和贯科技有限公司 | Knowledge graph construction method based on teaching content |
| US11727329B2 (en) | 2020-02-14 | 2023-08-15 | Yandex Europe Ag | Method and system for receiving label for digital task executed within crowd-sourced environment |
| CN111460115A (en) * | 2020-03-17 | 2020-07-28 | 深圳市优必选科技股份有限公司 | Intelligent man-machine conversation model training method, model training device and electronic equipment |
| CN111460115B (en) * | 2020-03-17 | 2023-05-26 | 深圳市优必选科技股份有限公司 | Intelligent man-machine conversation model training method, model training device and electronic equipment |
| CN111428470B (en) * | 2020-03-23 | 2022-04-22 | 北京世纪好未来教育科技有限公司 | Text continuity judgment method, text continuity judgment model training method, electronic device and readable medium |
| CN111428470A (en) * | 2020-03-23 | 2020-07-17 | 北京世纪好未来教育科技有限公司 | Text continuity judgment method, text continuity judgment model training method, electronic device and readable medium |
| CN111241263A (en) * | 2020-04-24 | 2020-06-05 | 支付宝(杭州)信息技术有限公司 | Text generation method and device and electronic equipment |
| CN112487158B (en) * | 2020-11-06 | 2023-05-05 | 泰康保险集团股份有限公司 | Multi-round dialogue problem positioning method and device |
| CN112487158A (en) * | 2020-11-06 | 2021-03-12 | 泰康保险集团股份有限公司 | Problem positioning method and device for multi-turn conversation |
| CN112417112A (en) * | 2020-11-10 | 2021-02-26 | 中山大学 | Open domain dialogue system evaluation method based on graph characterization enhancement |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109522545B (en) | 2019-08-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109522545B (en) | A kind of appraisal procedure that more wheels are talked with coherent property amount | |
| Liu et al. | Knowledge diffusion for neural dialogue generation | |
| CN107133224B (en) | Language generation method based on subject word | |
| CN110222163B (en) | Intelligent question-answering method and system integrating CNN and bidirectional LSTM | |
| CN107832400A (en) | A kind of method that location-based LSTM and CNN conjunctive models carry out relation classification | |
| CN110717334A (en) | Text emotion analysis method based on BERT model and double-channel attention | |
| CN108363695B (en) | User comment attribute extraction method based on bidirectional dependency syntax tree representation | |
| CN109635109A (en) | Sentence classification method based on LSTM and combination part of speech and more attention mechanism | |
| CN107578106A (en) | A kind of neutral net natural language inference method for merging semanteme of word knowledge | |
| CN106372058A (en) | Short text emotion factor extraction method and device based on deep learning | |
| Wen et al. | Dynamic interactive multiview memory network for emotion recognition in conversation | |
| CN112667818B (en) | GCN and multi-granularity attention fused user comment sentiment analysis method and system | |
| CN111274398A (en) | Method and system for analyzing comment emotion of aspect-level user product | |
| CN110083710A (en) | It is a kind of that generation method is defined based on Recognition with Recurrent Neural Network and the word of latent variable structure | |
| CN113535904B (en) | Aspect level emotion analysis method based on graph neural network | |
| CN107679225B (en) | Reply generation method based on keywords | |
| CN110427616A (en) | A kind of text emotion analysis method based on deep learning | |
| CN112115242A (en) | Intelligent customer service question-answering system based on naive Bayes classification algorithm | |
| CN112364148B (en) | Deep learning method-based generative chat robot | |
| CN112287106A (en) | Online comment emotion classification method based on dual-channel hybrid neural network | |
| CN113656564A (en) | Power grid service dialogue data emotion detection method based on graph neural network | |
| CN116028604A (en) | Answer selection method and system based on knowledge enhancement graph convolution network | |
| Mai et al. | A unimodal representation learning and recurrent decomposition fusion structure for utterance-level multimodal embedding learning | |
| CN114328866A (en) | Strong anthropomorphic intelligent dialogue robot with smooth and accurate response | |
| CN110297894A (en) | A kind of Intelligent dialogue generation method based on auxiliary network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |