CN115495552A - Multi-round dialogue reply generation method and terminal equipment based on dual-channel semantic enhancement - Google Patents

Abstract

The invention discloses a multi-round dialogue reply generation method based on dual-channel semantic enhancement and a terminal device. The method includes acquiring the initial word vector of the dialogue text; acquiring the sequential semantic representation of the initial word vector, including acquiring the discourse-level sentence of the initial word vector Semantic vector, determine the dialogue-level sentence semantic vector of the initial word vector according to the discourse-level sentence semantic vector, record the dialogue-level sentence semantic vector as a sequential semantic representation; obtain the graph domain semantic representation of the initial word vector on the graph domain; according to the sequential semantic representation And graph domain semantic representation, carry out semantic enhancement on the dialogue text, obtain the enhanced semantic representation; generate reply text according to the enhanced semantic representation. The invention aims to integrate the semantic advantages in different structural modeling to obtain information association and semantic reasoning with a larger span. The model of the invention performs well on the benchmark model, and alleviates the problem of long-distance semantic dependence.

Description

Multi-round dialogue reply generation method and terminal equipment based on dual-channel semantic enhancement

technical field

The invention belongs to the technical field of artificial intelligence, and in particular relates to a method for generating multi-round dialogue replies based on dual-channel semantic enhancement and a terminal device.

Background technique

With the rise of the Internet of Everything and human-computer interaction, dialogue systems, as a widely used communication medium, have vertically penetrated into many scenarios such as smart customer service, AI speakers, and smart cockpits. At the same time, due to its great advantages of improving information service experience and assisting voice command interaction, dialogue systems have great research value and application value, among which multi-turn dialogue systems are the most prominent. Multi-round dialogue reply generation is a kind of generative dialogue that focuses on continuous dialogue and complex semantic interaction. It can provide meaningful and diverse fluent responses to users based on the interaction text between users and agents within a certain period of time. Over the years, it has been gradually investigated and paid attention to by researchers from various countries.

Most of today's intelligent dialogue systems are developed based on end-to-end deep neural network technology. With the popularity of application scenarios, the responses of dialogue systems cannot keep pace with the times. The form is relatively single, and the content lacks scene value. Among them, although the continuous interactive multi-round dialogue system research improves the response quality by introducing common sense or fixed sentence patterns, the main challenge is to effectively model the context and obtain accurate semantic representation.

In the process of extracting semantic information in the multi-round dialogue reply generation in the prior art, due to the limitation of the model structure and the long sequence structure of the conversation history, it is difficult to obtain accurate query information, and it is easy to introduce semantic noise to interfere with the generated reply, resulting in the generation of abnormal Ideal response with less robustness.

Contents of the invention

The present invention provides a multi-round dialog reply generation method based on dual-channel semantic enhancement and a terminal device, which solves the technical problems of poor reply quality and poor robustness existing in the multi-round dialog reply method in the prior art.

The first aspect of the content of the present invention discloses a multi-round dialogue reply generation method based on dual-channel semantic enhancement, including:

Obtain the initial word vector of the dialogue text;

Obtaining the sequential semantic representation of the initial word vector includes obtaining the utterance-level sentence semantic vector of the initial word vector, determining the dialogue-level sentence semantic vector of the initial word vector according to the utterance-level sentence semantic vector, and converting the dialogue Sentence-level semantic vectors are recorded as sequential semantic representations;

Obtaining the graph domain semantic representation of the initial word vector on the graph domain;

performing semantic enhancement on the dialog text according to the sequential semantic representation and the graph domain semantic representation, to obtain an enhanced semantic representation;

A reply text is generated according to the enhanced semantic representation.

Preferably, obtaining the utterance-level sentence semantic vector of the initial word vector specifically includes:

The initial word vector is input into the sentence layer encoder and the word attention module in turn to obtain the discourse-level sentence semantic vector;

Determine the dialogue-level sentence semantic vector of the initial word vector according to the utterance-level sentence semantic vector, specifically including:

The discourse-level sentence semantic vector is input into the context encoder and the sentence attention module in sequence to obtain the dialogue-level sentence semantic vector;

Both the sentence layer encoder and the context encoder are bidirectional gated neural networks;

The mechanisms used in the word attention module and the sentence attention module are attention mechanisms.

Preferably, obtaining the graph domain semantic representation of the initial word vector on the graph domain, specifically includes:

Obtain the topic keywords of the initial word vector, determine the nodes of the heterogeneous cognitive map according to the topic keywords, and the nodes include "topic-sentence cluster" nodes, dialogue query nodes, common nodes (dialogues that do not contain topics) sentence cluster);

According to the nodes of the heterogeneous cognitive graph, determine the edges of the heterogeneous cognitive graph and the weight of each edge, and the weight is determined according to the degree of topic overlap between sentences in the dialogue text corresponding to the initial word vector;

A graph neural network is used to learn the vector representation of the nodes in the heterogeneous cognitive graph, and obtain the graph domain semantic representation of the initial word vector on the graph domain.

Preferably, performing semantic enhancement on the dialog text according to the sequential semantic representation and the graph domain semantic representation specifically includes:

The dialogue text is semantically enhanced according to a first formula, and the first formula is:

为所述顺序语义表示，

为所述图域语义表示，δ为所述顺序语义表示中的语义数量，(1-δ)为所述图域语义表示中的语义数量。In the formula, c _final is the enhanced semantic representation,

For the sequential semantic representation,

is the graph-domain semantic representation, δ is the semantic quantity in the sequential semantic representation, and (1-δ) is the semantic quantity in the graph-domain semantic representation.

Preferably, the reply text is generated according to the enhanced semantic representation, which specifically includes:

Input the enhanced semantic representation into the one-way gated neural network to obtain the hidden state of each word in the generated reply text;

According to the hidden state, the generation probability of each word is determined, and the reply text is determined according to the generation probability.

Preferably, the enhanced semantic representation is input into a one-way gated neural network to obtain the hidden state of each word in the generated reply text, specifically including:

Generate the hidden state of each word in the reply text according to the second formula, the second formula is:

为y_i的隐藏状态，GRU(·)表示将其中的参数输入至门控神经网络中，

为y_i-1的隐藏状态，c_final为所述增强后的语义表示。In the formula, y _i is the i-th word in the reply text generated in the training phase, and y _i-1 is the i-1th word in the reply text generated in the training phase,

is the hidden state of y _i , and GRU( ) means that the parameters in it are input into the gated neural network,

is the hidden state of y _i-1 , and c _final is the enhanced semantic representation.

Preferably, according to the hidden state, determining the generation probability of each word specifically includes:

Determine the generation probability of each word according to the third formula, the third formula is:

为预测阶段生成回复文本中的第i个词，

为

的生成概率，

和

别为预测阶段主题关键词词表和回复文本词表中第i个词

的生成概率。In the formula,

generate the i-th word in the reply text for the prediction phase,

for

The generation probability of

with

Don’t be the i-th word in the topic keyword vocabulary and reply text vocabulary in the prediction stage

the generation probability of .

根据第四公式确定，所述第四公式为：Preferably, the

Determined according to the fourth formula, the fourth formula is:

为训练阶段生成回复文本中第i个词y_i的隐藏状态，y_i-1为训练阶段生成回复文本中第i-1个词，c_final为所述增强后的语义表示，vocab表示变量i。In the formula, η(·) is a non-linear function tanh, V is a reply text vocabulary, K is a subject keyword vocabulary,

Generate the hidden state of the i-th word y _i in the reply text for the training phase, y _i-1 generates the i-1th word in the reply text for the training phase, c _final is the semantic representation after the enhancement, and vocab represents the variable i .

根据第五公式确定，所述第五公式为：Preferably, the

Determined according to the fifth formula, the fifth formula is:

为训练阶段生成回复文本中第i个词y_i的隐藏状态，y_i-1为训练阶段生成回复文本中第i-1个词，c_final为所述增强后的语义表示，vocab表示变量i。In the formula, η(·) is a non-linear function tanh, V is a reply text vocabulary, K is a subject keyword vocabulary,

Generate the hidden state of the i-th word y _i in the reply text for the training phase, y _i-1 generates the i-1th word in the reply text for the training phase, c _final is the semantic representation after the enhancement, and vocab represents the variable i .

The second aspect of the content of the present invention discloses a terminal device, including a memory, a processor, and a computer program stored in the memory and operable on the processor, when the processor executes the computer program, the steps of the method described above.

Compared with the prior art, the present invention has the following beneficial effects:

The invention is a sequential and graph-domain dual-channel cooperative semantic modeling and reasoning method, which aims to integrate semantic advantages in different structural modeling and obtain information association and semantic reasoning with a larger span. In detail, on the one hand, the present invention constructs a dialogue-level heterogeneous cognitive graph. The graph nodes are the integration of topic semantics and sentence cluster semantics. The edges in the graph represent the degree of topic overlap between sentences. Learning, to obtain the semantic representation of the dialogue context on the graph domain; on the other hand, embedding a hierarchical attention mechanism in the preserved sequential channel obtains the sequential semantic representation of the dialogue context. Finally, the information contributions of the two semantic representations are reconciled for prediction. The model of the present invention performs well on the benchmark model, and alleviates the problem of long-distance semantic dependence.

The present invention helps to promote the further development of multi-round dialogue generation, helps the system to better understand the high-level semantic information of the context, and can also regain new cognition from the reconstructed heterogeneous cognitive map structure, helping to generate diversification, Valuable information with good robustness improves users' satisfaction and efficiency in using information services.

Description of drawings

FIG. 1 is a schematic flowchart of a multi-round dialog reply generation method based on dual-channel semantic enhancement according to an embodiment of the present invention;

2 is a detailed flow chart of a method for generating multi-round dialogue replies based on dual-channel semantic enhancement according to an embodiment of the present invention;

Fig. 3 is the structural representation of graph neural network of the present invention;

FIG. 4 is a schematic diagram of node semantic representation encoding according to an embodiment of the present invention;

Fig. 5 is the aggregation strategy of the semantic feature of the specific embodiment of the present invention;

Fig. 6 is a performance graph of the SGDC model and the baseline model of the specific embodiment of the present invention on test samples with different context lengths. In Figure 6, (a) is the PPL value of the dataset DailyDialog, (b) is the PPL value of the dataset MuTual, (c) is the Dist-2 value of the dataset DailyDialog, and (d) is the Dist-2 of the dataset MuTual Value, (e) is the EA value of the dataset DailyDialog, (f) is the EA value of the dataset MuTual.

detailed description

The technical solutions of the present invention will be further described in detail below in conjunction with the accompanying drawings and specific implementation cases. It should be understood that the following examples are only for illustrating and explaining the present invention, and should not be construed as limiting the protection scope of the present invention. All technologies realized based on the above contents of the present invention are covered within the scope of protection intended by the present invention.

Multi-round dialogue response generation is essentially a problem of predicting sequences based on sequences. The present invention performs task-based modeling of multi-round dialogue reply generation, which belongs to the task of natural language text generation. A multi-round dialog sequence Diag contains M>2 rounds of utterances, m∈(1,M] represents the mth round, and the dialog sequence can be used Diag={U ₁ ,...U _m-1 ,U _M } , where the Dialogue History utterance sequence (U ₁ , ...... U _m-2 ) represents the contextual information of the entire dialogue context, and the Dialogue Query (Dialogue Query) utterance U _m-1 represents The current dialogue progress state, U _M is the target speech (Target Response) to be generated by the multi-round dialogue reply generation task of the present invention.

Most of today's intelligent dialogue systems are developed based on end-to-end deep neural network technology. With the popularity of application scenarios, the responses of dialogue systems cannot keep pace with the times. The form is relatively single, and the content lacks scene value. Among them, although the continuous interactive multi-round dialogue system research improves the response quality by introducing common sense or fixed sentence patterns, the main challenge is to effectively model the context and obtain accurate semantic representation. The current most commonly used hierarchical encoder-decoder framework ignores the fact that dialogue is produced in a coherent process, where any two utterances are semantically related and complement or reduce each other. When encoding each utterance individually without considering their interrelationships, hierarchical models may fail to capture utterance coherence in context and end up producing non-ideal responses. Therefore, the encoder-decoder framework based on the hierarchical model still needs to identify the contribution of different sentences and encode them differently in the context semantic modeling of the dialogue history.

In the multi-round dialogue reply generation, the topic information is the high-level semantic feature extracted from the dialogue history, and the model improves the information volume and relevance of the response reply by integrating the topic semantics. However, the existing research only introduces or selects topics to alleviate semantic sparsity. The vector representation of topics does not consider the semantic interaction with the specific dialogue context where the topics are located. This context-free approach may be due to the inherent diversity of natural language This leads to inaccurate topic representation and utterance sentence representation, which in turn impairs the effect of response generation.

The vector representation learning of multi-round dialogue text input has gradually stabilized from the disordered bag-of-words structure to the sequence structure, and the semantic modeling method has also progressed from machine learning methods to deep learning methods represented by recurrent neural networks and attention mechanisms. Due to the neural network learning mode of sequence structure, semantic modeling is still difficult to solve the problem of long-distance dependence. With the widespread application of graph neural networks in various NLP sub-tasks, the task of multi-round dialogue response generation also urgently needs to break the shackles of non-Euclidean space, deeply explore the graph structure of its own input, and model accordingly Sequence Structure Modeling in Context Aided Status Quo.

The first aspect of the present invention provides a multi-round dialogue reply generation method based on dual-channel semantic enhancement, as shown in Figures 1 and 2, including:

Step 1. Obtain the initial word vector of the dialogue text.

Step 2. Obtain the sequential semantic representation of the initial word vector, including obtaining the discourse-level sentence semantic vector of the initial word vector, determine the dialogue-level sentence semantic vector of the initial word vector according to the discourse-level sentence semantic vector, and record the dialogue-level sentence semantic vector as the sequence Semantic representation.

The main purpose of this step is to model the context of the dialogue text in the temporal development through the semantic analysis of the sequential channel, and form a semantic vector representation of the dialogue context in the sequential channel. In order to reduce the loss of important semantics in the process of multi-round dialogue flow, this step adds layered attention to the important semantics of different granularities in the encoder of the layered architecture. The hierarchical architecture encoder includes sentence-level encoder and context encoder, and the hierarchical attention includes word attention and sentence attention.

Step 2 specifically includes:

Step 21. Input the initial word vector into the sentence layer encoder and word attention module in sequence to obtain the sentence-level sentence semantic vector. The sentence layer encoder is a two-way gated neural network, and the mechanism used by the word attention module is the attention mechanism.

The sentence-level encoder and word attention are based on the initial word vector representation of the dialogue context to perform sentence-level sentence semantic vector learning, such as formula (1):

为双向门控神经网络(Bidirectional GatedRecurrentUnit，

)，Num_i为句子U_i的词汇总数，

为双向门控神经网络在学习词汇x_j,i时的相邻隐藏状态输出，w_j,i为句子U_i第j个位置上词汇x_j,i的初始向量，

为句子U_i第j个位置上词汇x_j,i最新的向量表示，也代表双向门控神经网络在学习词汇x_j,i后的隐藏状态输出。In the formula, taking the utterance-level sentence semantic vector learning of U _i as an example,

For the bidirectional gated neural network (Bidirectional GatedRecurrentUnit,

), Num _i is the total vocabulary of sentence U _i ,

is the adjacent hidden state output of the bidirectional gated neural network when learning the vocabulary x _j,i , w _j,i is the initial vector of the vocabulary x _j,i at the jth position of the sentence U _i ,

is the latest vector representation of the vocabulary x _j,i at the jth position of the sentence U _i , and also represents the hidden state output of the bidirectional gated neural network after learning the vocabulary x _j,i .

作为句子U_i的语义向量，而是使用解码步骤中的s_t-1和隐藏状态序列

进行相似度计算，确定附加在每个隐藏状态上的各自权重{α_1,i,α_2,i...α_j,i}，从而加权求和得到U_i的语义向量

如公式(2)和公式(3)所示：Different from the encoder of ordinary layered architecture, the present invention does not use the last hidden state

as the semantic vector of the sentence U _i , but instead use s _t-1 and the hidden state sequence in the decoding step

Carry out similarity calculations, determine the respective weights {α _1,i ,α _2,i ...α _j,i } attached to each hidden state, and thus obtain the semantic vector of U _i by weighted summation

As shown in formula (2) and formula (3):

In the formula, η( ) represents the Relu function, which can save calculations and alleviate the problems of overfitting and gradient disappearance. So far, the present invention can obtain the sentence semantic vector sequence of discourse level

Step 22. Input the utterance-level sentence semantic vector into the context encoder and the sentence attention module in turn to obtain the dialogue-level sentence semantic vector. The context encoder is a bidirectional gated neural network, and the mechanism used by the sentence attention module is the attention mechanism.

Context encoder and sentence attention are based on utterance-level sentence semantic vectors for dialogue-level sentence semantic vector learning. Similar to the calculation above:

为双向门控神经网络(BidirectionalGatedRecurrentUnit，

)，

为双向门控神经网络在学习词汇x_t,i时的相邻隐藏状态输出，β_i,t为句子U_i第t个位置上词汇x_t,i的权重，

为句子U_i第t个位置上词汇x_t,i最新的向量表示，也代表双向门控神经网络在学习词汇x_t,i后的隐藏状态输出,

是上下文编码器获得的输出层的隐藏状态向量，

是经过句子注意力加权计算后获得的对话级的句子语义向量表示。

是计算了解码状态和上下文编码器的每个隐藏状态的关联权重后加权聚合得到的，能够基于文本的实时申请状况来调整整个对话上下文的最终语义表示。本发明一般将

称作

为对话级句子语义向量，这个对话上下文的最终语义向量代表着顺序通道中经过复杂语义交互学习后的语境，是解码器的重要参照及输入。Among them, η( ) represents the Relu function,

For the bidirectional gated neural network (BidirectionalGatedRecurrentUnit,

),

is the adjacent hidden state output of the bidirectional gated neural network when learning the vocabulary x _t,i , β _i,t is the weight of the vocabulary x _t,i at the tth position of the sentence U _i ,

is the latest vector representation of the vocabulary x _t,i at the tth position of the sentence U _i , and also represents the hidden state output of the bidirectional gated neural network after learning the vocabulary x _t,i ,

is the hidden state vector of the output layer obtained by the context encoder,

It is a dialogue-level sentence semantic vector representation obtained after sentence attention weighted calculation.

It is obtained by weighted aggregation after calculating the associated weights of the decoding state and each hidden state of the context encoder, and can adjust the final semantic representation of the entire dialogue context based on the real-time application status of the text. The present invention will generally

called

It is a dialogue-level sentence semantic vector. The final semantic vector of this dialogue context represents the context after complex semantic interaction learning in the sequential channel, and is an important reference and input for the decoder.

Step 3. Obtain the graph domain semantic representation of the initial word vector on the graph domain.

The main purpose of the present invention is to model the medium and long-distance semantic associations of dialogue texts across time series through the explicit and implicit semantic analysis of the graph-domain channel, and to learn the semantic vector representation of the dialogue context in the graph-domain channel. First, the graph is constructed according to the explicit and implicit relations in the dialogue context, then a new graph neural network is designed to learn the vector representation of the nodes, and finally the final semantic vector representation is obtained through pooling calculation. In order to reduce the loss of important semantics in the process of multiple rounds of dialogue flow, the present invention designs a double-gated filtering mechanism in the traditional graph neural network layer. The double-gated mechanism can reduce semantic noise when node information is updated.

Compared with the simplest fully connected neural network (MLP), graph neural network technology updates node information on the graph domain structure. In addition to the weight matrix, an additional adjacency matrix A is added for aggregation calculation during calculation, as shown in Figure 3 shown. In the research, there are three major categories of common graph neural networks, namely:

Graph Convolution Networks (GCN)

Graph Convolutional Neural Network (GCN) is divided into GCN based on spectral domain and GCN based on space. The latter is the most widely used in GCN. Therefore, this invention focuses on GCN based on space (hereinafter referred to as GCN). Similar to the convolution calculation of traditional CNN on European data, GCN performs convolution according to the node relationship in the graph domain data, and aggregates the representation of the central node and its neighbors along the edge to update the vector of the central node Indicates that it can adapt to different positions and structures, and can also share weights during node calculations. Information transfer calculation between nodes:

代表v节点在第k层的节点向量，u∈N_v代表v节点的邻居节点集合。where M _k (·) and U _k (·) are functions with learnable parameters, generally using a fully connected neural network (MLP).

Represents the node vector of node v at layer k, u∈N _v represents the set of neighbor nodes of v node.

Graph Attention Networks (GAT)

Graph Attention Network (GAT) supports amplifying the influence of the most important parts of neighbor nodes. It uses an attention mechanism to determine the weights of neighborhood nodes during the aggregation process, controls multiple neighbor node representation vectors to input the semantic information of the center node, and generates important target-oriented random walk representations. Information transfer calculation between nodes:

对v节点语义信息贡献的注意力函数。where W _k (·) and U _k (·) are functions with learnable parameters, generally using a fully connected neural network (MLP). α( ) is a node that can adaptively control neighbor nodes

An attention function that contributes to the semantic information of v-nodes.

Graph Spatial-Temporal Networks (GSTN)

Graph spatio-temporal network (GSTN) performs well in spatio-temporal correlation, such as node prediction of traffic network and other application scenarios. GSTN can predict future node values or labels, as well as predict space-time graph labels. It follows two methods based on RNN and CNN in construction. Taking the RNN-based method as an example, adding a graph convolution unit can capture the spatio-temporal dependencies, and the node update is calculated as:

Among them, Gconv( ) is the graph convolution unit, A _v is the adjacency matrix of the central node at layer k, which represents the association between the neighbor node and the central node; RNN( ) is a classic recurrent neural network calculation, detailed introduction:

h _t ＝σ(U·x _t +W·h _t-1 ) (9)

Based on this, the above step 3 specifically includes:

Step 31. Obtain the topic keywords of the initial word vector, and determine the nodes of the heterogeneous cognitive map according to the topic keywords. The nodes include topic nodes, non-topic nodes and query nodes.

The present invention constructs a heterogeneous cognitive map by extracting subject keywords and connecting the segmented dialogue contents explicitly and implicitly. First, the entire dialogue context {U ₁ , U ₂ ... U _m-1 , U _m } is divided into three parts {U, Q, R}, which are the dialogue history sentences {U ₁ , U ₂ ... U _m-2 }, dialogue query sentence {U _m-1 } and reply text {U _m }, where the dialogue history sentence is far from the reply text, representing the global historical information of the dialogue, and the dialogue query sentence is close to the reply text, representing the dialogue The short-term intention information is coarse-grained semantic information. In addition, unlike long text analysis, there is often content in the dialogue context that is irrelevant to the direction of the entire dialogue flow, such as "Yes, I understand", so the present invention extracts topic keywords to better understand the dialogue context. Topic keywords are special named entities, which are important entities distributed in the entire dialogue context. They are highly recognizable and represent fine-grained semantic information, which can be used to model the semantic flow association of dialogues.

The present invention uses a Term Frequency-Inverse Document Frequency algorithm (Term Frequency-Inverse Document Frequency, TF-IDF algorithm) to extract topic keywords, and topic keywords are high-frequency words in the dialogue text that can represent the dialogue context.

Step 32. According to the nodes of the heterogeneous cognitive graph, determine the edges of the heterogeneous cognitive graph and the weight of each edge. The weight is determined according to the topic overlap degree between sentences in the dialogue text corresponding to the initial word vector.

The implementation algorithms of step 31 and step 32 are shown in Table 1.

Table 1 Heterogeneous Cognitive Map Construction Algorithm

Algorithm 1 expounds the process of constructing heterogeneous cognitive maps from dialogue texts. The heterogeneous cognitive graph is established to support cognitive reasoning based on graph-domain channels. Specifically, it can use dialogue queries, dialogue history, and collaborative information in topic keywords to perform multi-hop reasoning to obtain stronger semantic interactions. Among them, the present invention uses Stanford CoreNLP (https://stanfordnlp.github.io/CoreNLP) to perform data processing such as word segmentation and part-of-speech tagging, but it is not enough to represent the semantics of dialogue sentences, so the TF-IDF algorithm is used to extract subject keywords, Topic keywords are high-frequency words in the dialogue text that can represent the context of the dialogue.

After the subject keyword set K is obtained in the present invention, the present invention constructs graph nodes through the status of subject keywords in dialogue sentences. The set of sentences containing a topic k and the topic k constitute the first type of important nodes in the heterogeneous cognitive graph, called v _k . It can be noticed that a sentence may contain several topic keywords, which indicates that the current sentence has rich semantic information and can establish a connection channel for information interaction with other similar sentences. When there is a sentence that does not contain the topic keyword, the present invention considers that the current sentence has little semantic effect on the entire dialogue, and it will be summarized into a special node v _empty . At the same time, since the dialogue query sentence is closest to the reply text, the present invention considers that the semantic role is the most important, and attributes it to another special node v _Q .

When establishing connections between nodes in the graph, an edge set E={e _i,j } is constructed from the above three types of nodes {v _K ,v _empty ,v _Q } according to explicit and implicit relationships. v _k is a "topic-sentence cluster" node, v _empty is an ordinary node (a dialog sentence cluster not containing a topic), and v _Q is a dialog query node. It should be noted here that the present invention takes heterogeneous node characteristics into consideration in the subsequent node representation, and the establishment of edges only needs to consider the weight of connections. In steps 13-17 of the algorithm, the present invention can see that when node v _i and node v _j share sentences, the present invention adds an edge e _i,j , the more sentences they share, the relationship between the two nodes The tighter it is, the greater the weight. In addition, due to the association of two types of special nodes, the present invention directly connects two special nodes to construct a special edge e _Q,E , which can learn important relevant information in semantic noise from the heuristic of query sentences.

Step 33. Use the graph neural network to learn the vector representation of the nodes in the heterogeneous cognitive graph, and obtain the graph domain semantic representation of the initial word vector on the graph domain.

Node Semantic Representation Encoding

The reasoning on the heterogeneous cognitive graph is based on updating and learning graph node representations. The initial representation of graph nodes can bring correct guidance to subsequent graph neural network learning. Taking Figure 4 as an example, the present invention calculates the initial vector representations of three kinds of nodes:

① For the node v _empty , the present invention performs average pooling on the initial semantic vectors of the sentences belonging to the node v _empty to obtain the node vector v _e , and when the node v _empty has no sentence set, the present invention performs average pooling on all dialogue history sentences :

作为节点向量v_Q：②For node v _Q , the present invention directly expresses the initial vector of the dialogue query sentence

As node vector v _Q :

③ For node v _k , the present invention performs cascading operations on the topic vector and sentence vector belonging to the node, and performs dimension transformation through a single-layer fully connected network. Take the node where the topic k ₁ is located as an example:

After obtaining the initialization vector representations of the three types of nodes, in order to facilitate subsequent graph neural network calculations, the present invention refers to each node vector representation {v _Q , v _e , v _K } as {v ₁ , v ₂ ...v _m } , where m=K+2.

Transmission and update of node information

Message passing between graph nodes is achieved through two steps: information aggregation and information combination, and this process can be performed multiple times (often referred to as layers or hops). Information aggregation is to gather the semantic interaction information of adjacent nodes in the same layer; information combination is to update and combine the information of the same node in different layers.

Information aggregation focuses on how a certain node collects semantic information of adjacent nodes. The present invention concerns that when the dialogue lasts for several rounds, the number of adjacent nodes of a certain node is large and not all of them can bring equivalent semantic information to the central node. Some even introduce semantic noise. Therefore, the present invention is different from the common graph neural network, and selects the GRU unit to filter the information content of the adjacent node clusters when updating the nodes of the l layer, so as to alleviate the semantic noise. Specifically, the reset gate R _t in the gating mechanism will control the information flow from the neighbor node v _j to v _i :

为边缘类型为r的节点v_i的邻居簇，

为某一邻接节点v_j在第l层中的节点表示。|·|表示邻接节点簇的大小。GRU单元定义了聚合邻接信息的转换过程。相邻节点表示的转换，可以通过多层感知机(MLP)实现。

表示节点v_i在第l层的聚合信息，出于图域结构连接复杂、节点多的考虑，本发明又增加了一个残差连接，避免梯度消失的同时保留自身重要语义：where R is the set of all types of edges,

is the neighbor cluster of node v _i whose edge type is r,

It is the node representation of a certain adjacent node v _j in layer l. |·| represents the size of the adjacent node cluster. The GRU unit defines the transformation process for aggregating adjacency information. The transformation of the representation of adjacent nodes can be realized by multi-layer perceptron (MLP).

Indicates the aggregation information of node v _i at layer l. Considering the complex connection of the graph domain structure and the large number of nodes, the present invention adds a residual connection to avoid gradient disappearance while retaining its own important semantics:

Among them, f _s is realized by multi-layer perceptron (MLP).

Information combination focuses on updating and combining representations of the same node in different layers to obtain information content after multi-hop cognition. However, existing studies have shown that graph neural networks are prone to smoothing problems in reasoning between layers. Smoothing problems will lead to similar node representations, thus losing the ability to distinguish information. In order to solve this problem, the present invention will control the size of the information flow of node _v from layer l to layer l+1 from different sources. This is to add a Gate weight in the information combination:

具体来讲，

是决定了信息组合时来自原始节点表示和更新节点表示中的信息数量，类似于灵活的残差机制。η(·)为非线性激活函数Leaky ReLU，⊙表示逐元素乘法，f_s、f_g均采用单层MLP实现。经过多层消息传递后，所有节点都将拥有它们最终更新后的节点表示。Among them, sigmoid( ) is to determine the weight by quantifying the contribution of different information sources of the same node to the update of inter-layer information

Specifically,

is to determine the amount of information from the original node representation and the updated node representation when the information is combined, similar to the flexible residual mechanism. η( ) is the non-linear activation function Leaky ReLU, ⊙ means element-wise multiplication, and f _s and f _g are implemented by single-layer MLP. After multiple layers of message passing, all nodes will have their final updated node representations.

Step 4. Perform semantic enhancement on the dialogue text according to the sequence semantic representation and the graph domain semantic representation, and obtain the enhanced semantic representation.

图域通道通过本发明构建的异构认知图和设计的双门控GNN，能够在图上进行对话意图和语义的多跳推理，建立中远距离的多个语义关联表示

两个通道的语义结果相辅相成，通过信息协同可以达到整个对话语境的高级语义认知。The sequential channel can obtain semantic representation on sequential dialogue data through hierarchical attention and progressive modeling of encoder

Through the heterogeneous cognitive map constructed by the present invention and the designed double-gated GNN, the graph-domain channel can perform multi-hop reasoning on dialogue intent and semantics on the graph, and establish multiple semantic association representations at medium and long distances

The semantic results of the two channels complement each other, and the high-level semantic cognition of the entire dialogue context can be achieved through information collaboration.

后，为了解码器中方便和顺序通道语义进行协同，使用权重得分score_i进行诸节点的语义信息管理：In the graph-domain channel, the semantic representation of each node is obtained through multi-hop reasoning. This semantic representation is the aggregation of long-distance information transmission between layers. The present invention obtains

Finally, for the convenience of the decoder to cooperate with the sequential channel semantics, use the weight score score _i to manage the semantic information of the nodes:

是未进入双通道的对话查询句子语义表示，代表最初的对话意图，对生成的回复有着很好的引导作用，所以本发明用来计算节点信息的信息管理权重；NumL是语义节点的总数量。in,

It is the semantic representation of the dialogue query sentence that has not entered the dual channel, represents the original dialogue intention, and has a good guiding effect on the generated reply, so the present invention is used to calculate the information management weight of node information; NumL is the total number of semantic nodes.

In the dual-channel information coordination module, the present invention also uses a Gate mechanism to control the influence of the semantic information of the two channels on the generated reply decoding process:

输送到解码器的语义数量，1-δ代表图域通道

输送到解码器的语义数量。两部分语义通过相加，组合成最终的语义增强后的语义表示c_final。c_final是学习了序列语义发展和图域语义关联后的集成语义，在信息集成上侧重了对话查询句子的对话方向，能够辅助解码器精准的解码生成新的词汇。where δ is the sequence channel

The number of semantics delivered to the decoder, 1-δ represents the image domain channel

The amount of semantics fed to the decoder. The two parts of semantics are added together to form the final semantically enhanced semantic representation c _final . c _final is the integrated semantics after learning the development of sequence semantics and the association of graph-domain semantics. In terms of information integration, it focuses on the dialogue direction of dialogue query sentences, which can assist the decoder to accurately decode and generate new vocabulary.

Step 5. Generate a reply text according to the enhanced semantic representation, specifically including:

Step 51. Input the enhanced semantic representation into the one-way gated neural network to obtain the hidden state of each word in the generated reply text.

In the decoder module part, the present invention uses a one-way GRU to decode and generate the latest hidden state to update the semantic vector of the entire decoding layer, thereby obtaining the hidden state to obtain the probability distribution of the decoding vocabulary:

是在生成回复文本的第i个词y_i时的解码层隐藏状态，c_final是双通道信息协同之后的语义表示，代表着对话思路清晰后的语义启发，y_i-1在训练时是本发明回复文本的第i-1个词的向量表示，在预测时用预测文本的第i-1个词的向量表示

代替，可以保证回复文本的一致性。in,

is the hidden state of the decoding layer when generating the i-th word y _i of the reply text, c _final is the semantic representation after the two-channel information collaboration, representing the semantic inspiration after the dialogue is clear, y _i-1 is the original Invent the vector representation of the i-1th word of the reply text, and use the vector representation of the i-1th word of the predicted text during prediction

Instead, the consistency of the reply text can be guaranteed.

Step 52: Determine the generation probability of each word according to the hidden state, and determine the reply text according to the generation probability.

When decoding the generated text, the present invention believes that the generated reply tends to extend the reply from the topic keywords, so it is different from the previous encoder and adds a topic bias probability. The model is forced to constrain the topic development, and the corresponding generation probability is calculated. for:

Among them, K and V represent subject keyword vocabulary and reply text vocabulary respectively. Correspondingly, the probability values of p _V and p _K are uniformized by Softmax:

Among them, η(·) is a nonlinear function tanh. In the process of training, the present invention defines θ as a trainable parameter, divides the training text into batches for training, and obtains the best model effect by optimizing the cross-entropy loss function based on negative log likelihood, and the learning parameter is The process of gradient reverse conduction and update descent, in which:

The present invention proposes for the first time a neural network model of a fine-grained information interaction method based on topic-enhanced dialogue history understanding. On the one hand, the model of the present invention uses topic semantics and each sentence to perform fine-grained semantic interaction to obtain enhanced semantic representations of dialogue history sentences; on the other hand, it uses dialogue query sentences to guide topic matrix fusion to obtain dialogue intent semantic representations. The two operations aim to use topic semantics to enhance the understanding of context, thereby breaking through the disadvantages of indiscriminate use of topic information in the past;

For the first time, the invention breaks the rigid thinking of sequence structure modeling of dialogue history context content, and proposes a model based on sequential and graph-domain dual-channel collaborative semantic modeling and reasoning methods. Drawing on the "Twin Towers" model in the recommendation system, our dual-channel model can understand and train the entire dialogue context from the perspective of the graph domain. At the same time, the dual-channel collaborative semantic modeling can maximize the semantic value and broaden the The research idea of round dialogue system.

The multi-round dialogue reply generation method based on dual-channel semantic enhancement of the present invention can be applied to the intelligent customer service system of the e-commerce platform, the voice interaction module of the bionic AI robot, and the new retrieval of the portal website. In addition, it can also be embedded in military information services to achieve intelligent planning and analysis of battle situation texts and auxiliary command and decision-making scenarios to improve the efficiency and interactive experience of information services.

The method of the present invention improves the fluency, diversity and rationality of generated responses in multiple rounds of dialogue, improves the robustness of generative dialogue, and optimizes the ability and effect of text semantic modeling.

Below, the present application will be described in detail with more specific embodiments.

Experiment preparation:

1. Research questions

In the sequential and graph dual-channel collaborative semantic modeling and reasoning model (Sequential and Graph Dual-channel Collaborative, SGDC) proposed by the present invention, this embodiment proposes the following three research questions to guide subsequent experiments:

RQ1: Does the SGDC model of this application perform better than other baseline models in terms of fluency, relevance and diversity?

RQ2: How does the length of the entire dialogue context (number of turns) affect the performance of our SGDC model on multi-turn dialogue reply generation?

RQ3: When the model decodes and predicts the reply text, does the synergy of sequential and graph-domain dual channels affect the overall performance of our SGDC model?

2. Dataset

In this embodiment, the DailyDialog data set and the MuTual data set are selected for experimentation.

The DailyDialog dataset is collected from daily life by scholars in the field, with a total of 13,118 conversations, covering various topics such as education, travel, weather, and shopping, and can reflect most of the communication between us humans. The semantic structure of DailyDialog is more formal, more topical, and the number of speakers is reasonable. The dialogue rounds are not redundant, and it has more research and application value.

The MuTual dataset is a high-quality manually annotated multi-turn dialogue inference dataset, which contains a total of 8,860 manually annotated dialogues based on Chinese students' English listening comprehension tests. Compared with previous dialogue benchmark datasets, MuTual is more challenging for inference. The conditions of the above two datasets are described in Table 2.

Table 2 Dataset information

DailyDialog MuTual number of conversations 13118 8860 Average number of rounds of dialogue 7.9 4.73 average number of words in a sentence 14.6 19.57

3. Experimental comparison benchmark model

In this embodiment, five related multi-round dialogue generation models are selected as the baseline algorithm model to compare the overall performance with the algorithm model of this application, and to discuss and analyze the experimental results. The introduction of the baseline algorithm model is as follows:

S2S-Att: The most popular encoder-decoder framework, the encoder encodes the input sequence into an intermediate state, and then uses the decoder to decode and generate, both the encoder and the decoder use the gated neural network GRU, and at the same time at each time The Attention mechanism is added to the decoder input of the step to ensure that each predicted word is the most relevant to the input text.

HRED: The first hierarchical context modeling approach for response generation, which encodes each sentence using an utterance-level GRU and converts utterance vectors into a dialogue-level vector representation using a dialogue-level GRU. Compared with the ordinary S2S framework, the three-level semantic progression of "vocabulary-discourse-dialogue" is considered, which can help information aggregation and dissemination at each level, thereby realizing multi-round dialogue history modeling.

THRED: It is the first model to introduce topic awareness in the field of multi-round dialogue generation. To expand, THRED introduces topic awareness on the basis of the HRED model, and uses the topic-context joint attention mechanism to decode the guidance reply.

ReCoSa: Use the Self-Attention mechanism to associate the dialogue context most closely related to the reply text. It is an improved hybrid model of Transformer and HRED. An attention mechanism is embedded in both word-level and utterance-level encoders for hierarchical modeling. It is currently the state-of-the-art performance in the field of multi-turn dialogue generation.

In addition, in order to explore the impact of the synergy of dual-channel semantics on model performance, we constructed three baseline models based on common semantic feature aggregation techniques, which are briefly introduced as follows:

-Avg: A two-channel model using the mean aggregation strategy, focusing on background semantics;

-Max: A dual-channel model using the maximum aggregation strategy, focusing on foreground semantics;

-Concat: A two-channel model using the mean aggregation strategy, focusing on global semantics.

The aggregation strategy of semantic features is an important strategy adopted when a number of semantic vectors are aggregated or when a semantic matrix is converted into a fixed-length vector representation. As shown in Figure 5, when semantic features are aggregated, information redundancy can be reduced and the focus semantics can be reduced. And to prevent training overfitting, which is similar to the role of the pooling layer in the convolutional neural network. Commonly used aggregation strategies are maximum aggregation, mean aggregation, join aggregation, and gated aggregation.

Maximum Aggregation Strategy

The operation of the maximum value aggregation strategy is to take the maximum value of the vector element value of each dimension as the vector element value of the same dimension represented by the new semantic vector. According to the flow of the maximum value aggregation strategy, other element values of the same dimension will not be passed to the next layer of semantic vectors. Such aggregation enables the model to focus on the most shallow and prominent parts of the semantics. This aggregation strategy can be used for text representations with a single vocabulary, simple syntax, and shallow semantics. The maximum pooling layer can be used for aggregation operations. The maximum aggregation formula is:

h _max = max{h ₁ ,h ₂ ... h _i } (26)

mean aggregation strategy

The operation of the mean aggregation strategy is to average the vector element values of each dimension as the same-latitude element vector value represented by the new semantic vector. The mean aggregation strategy considers that the elements of each dimension contain equal semantic value, and the mean can be used to neutralize the semantics and reduce the deviation of the semantic direction, which is a strategy that can improve the robustness of the model. The further back the position in the deep neural network, the richer and more balanced the semantic information, the more suitable for the mean aggregation strategy. The mean aggregation formula is:

Join Aggregation Strategy

The join aggregation strategy is an aggregation strategy that takes into account both the advantages of mean aggregation and maximum aggregation. Taking the two feature vectors v ₁ ∈ R ⁿ and v ₂ ∈ R ^m as an example, the common join aggregation strategy operation is to splice the vector elements, the most obvious is to increase the dimension to v _concat ∈ R ^n+m , the join aggregation is Violently attaching semantic information and retaining semantic information to the greatest extent is a direct but very effective strategy. After splicing, a linear mapping is often used to convert to the required number of dimensions. The formula for join aggregation is:

Gated Aggregation Strategy

The gated aggregation strategy supports a flexible information collaboration method, and controls the input degree of each dimension or vector semantics through the calculation of Gate. The specific operation of the gated aggregation strategy will learn the value of the Gate through the activation function sigmoid and gradient update in deep learning, and sequentially combine the Gate value with each state through the Hadamard product (Hadamard), so as to aggregate to obtain the final vector representation . The gated aggregation strategy supports the training update of deep learning, and can confirm the important weight of aggregation of each dimension according to the neural network, which is suitable for complex semantic and reasoning task models. The formula for gated aggregation is:

4. Experimental evaluation criteria and indicators

The automatic evaluation of multi-round dialogue response generation mainly considers the fluency, diversity and relevance of the generated text. The fluency of the text represents the reasonable compliance of the grammatical and semantic associations of words, that is, the deep learning model maximizes the probability of part-of-speech sequences and lexical collocation distances, which meets the requirements of human-readable and comprehensible texts, and there is no ambiguous understanding. The diversity of texts means that the dialogue context is rich in semantics and extensible. It is not a meaningless reply of "Idon't Know", but a necessary requirement for a chat-type dialogue system. Relevance to the topic refers to whether the reply has practical significance, and it is an important direction for judging the reply generation task.

In terms of specific implementation, in order to better evaluate the pros and cons of the baseline algorithm and the research work of this application, according to the literature (completion), the perplexity PPL, dist-1 and dist-2 and the sentence correlation index based on Embedding are used to evaluate and generate responses respectively fluency, variety, and relevance.

PPL: According to the references, we use the language model perplexity (PPL) to evaluate the fluency of the generated text. The lower the PPL value, the higher the probability of the generated reply text, the more reasonable the vocabulary arrangement and collocation, and the easier it is to understand and fluency. Its formula is:

Distinct: This paper uses automated evaluation metrics Distinct-1 and Distinct-2 to evaluate the content diversity of generated text. The higher the score value of Distinct-n, the higher the proportion of n-tuples in the sentence, which means that the generated text is richer in content and the reply effect is better. The formula is calculated as follows:

Embedding: Different from the ngram method to calculate the degree of overlap or co-occurrence between the prediction and the real, the Embedding-based evaluation method converts the text into a low-dimensional semantic representation, and then measures the degree of correlation through text similarity. In this embodiment, Greedy Matching (GM), Embedding Average (EA) and Vector Extrema (VE) are used for evaluation. The larger the value of the three evaluation indicators, the closer the semantic correlation between the predicted text and the real text, and the more relevant the reply is.

Greedy Matching (GM), embedded greedy value measurement uses greedy search to generate vocabulary or semantics similar to keywords in real text as much as possible, considers word-level alignment in a more granular manner, and evaluates long texts more accurately. The formula is calculated as follows:

Embedding Average (EA), the embedded average metric is widely used to measure text similarity. Cosine similarity is used to measure the semantic vectors of predicted and real texts, where semantic vectors are a method of calculating the meaning of a phrase by averaging the vector representations of its constituent words, calculated as follows:

Vector Extrema (VE), the embedding extreme value measurement uses the extreme value of each dimension of the word vector when calculating the text vector, and uses the cosine similarity to measure and compare as above. It is worth noting that this evaluation index focuses on the information extreme value, that is, the topic information, and thus can be used to measure relevance to the topic, the formula is calculated as follows:

Parameter setting and implementation environment

In order to fairly compare the baseline algorithm and the algorithm model of the present invention, the embodiments of the present invention all use the Adam optimizer and the Pytorch framework, and during the training period, the word embedding vectors are randomly initialized and model updated, the dimension is 512 dimensions, and all recurrent neural networks (GRU and BIGRU) The input and output hidden dimensions of the unit are also 512 dimensions, the model learning rate is set to 0.0001 for gradient clipping, and the number of samples (batch size) participating in training in each iteration is set to 64, and all are in the workstation of NVIDIA RTX GPU Optimizing training and validating predictions on .

In addition, the topics in the model of this embodiment are extracted through TF-IDF. In order to speed up the training process and prevent the accumulation of errors caused by large model errors in the early training process, a teacher forcing mechanism is introduced to force the input of the decoder to be modified to the target token, thereby reducing the error transmission in the model and ensuring that the parameters can be updated normally .

The hardware and software implementation configuration of this work is shown in Table 3 below:

Table 3 Software and hardware implementation configuration

Analysis and discussion of experimental results:

1. Overall performance compared to the baseline model

In order to explore the RQ1 problem, this example compares the performance of the SGDC model and the baseline model of the DialogueRNN framework on the two data sets of MuTual and DailyDialog. The evaluation results of fluency, relevance and diversity are shown in Table 4. Among them, the best-performing baseline model indicator value is underlined, and the best evaluation indicator value is bolded.

Table 4 Performance comparison between the SGDC model and the baseline model of the DialogueRNN framework

It can be seen from the experimental results in Table 4 that:

1 Obviously, the SGDC model of the dual-channel semantic modeling of this embodiment outperforms other baseline models in most evaluations on the two datasets. The significantly winning model performance on the two datasets demonstrates the effectiveness of the sequential and graph-domain dual-channel semantic collaborative modeling approach, which can obtain semantic associations and inference effects that cannot be mined by the DialogueRNN framework.

2 In the evaluation of the correlation dimension, we found an interesting phenomenon: Taking the MuTual dataset as an example, the SGDC model scored higher in the three indicators evaluated in the correlation. Compared with the best baseline model, VE and The EA score is about 2% higher, the GM score is 3.5% higher, and VE, EA, and GM scores are all 5% higher than all other baseline models. Similarly, the same is true on the DailyDialog dataset, but the improvement is not as large as that of the MuTual dataset. In order to explain this capability gap in correlation, it is important to understand the nature of these models and datasets. The MuTual dataset is a more rigorously labeled dataset that focuses on multi-round dialogue reasoning. The SGDC model mines semantic connections from different structural perspectives. , the sequential channel can capture general progressive semantics, and the graph-domain channel can get long-distance information dependence through edge connections across distance barriers in multiple rounds of dialogue. The advantages of this dual-channel collaborative modeling are obviously better than other DialogueRNN frameworks baseline model.

3 The score of this embodiment is significantly higher than that of ReCoSa, which may be because the graph-domain channel of the present invention is constructed based on the relationship between "topic-sentence cluster" nodes, and the semantic reasoning on heterogeneous graphs is more than that of ReCoSa, which only focuses on the prior mechanism of question-answering It is more effective, which also shows that the model of the present invention can accurately perceive the topic direction of the dialogue, so it can obtain a closer similarity score with the real reply text, and keep the dialogue track on the topic.

4 Surprisingly, in terms of diversity Dist-1 and Dist-2 scores, there is not much difference between the model of the present invention and other benchmark models, which may be because the semantic modeling of the present invention pursues semantic connections too much, and also uses topic Bias probability, so a certain amount of text diversity is lost. This is a point that is difficult for dialogue generation models to balance. However, from the perspective of scenario applications, it is worthwhile to exchange more closely related replies for controllable loss of diversity.

Impact of dialog context length on performance

In order to explore the RQ2 problem, this embodiment analyzes the performance of the SGDC model and the baseline model on test samples with different context lengths, that is, the number of rounds of multi-round dialogue is different. In this embodiment, the sample test set is manually divided into three groups according to the length of the dialogue context, which are short dialogues (dialogue turns less than 5), medium dialogues (dialogue turns between 6 and 10 rounds) and long dialogues ( Dialogue turns are greater than 10 rounds). Some objective evaluation indicators are used to evaluate the performance of each model, and the results are drawn in Figure 6. The three histograms in each model in the figure refer to short, medium and long stories from left to right.

It can be seen from the experimental results that:

1 Whether it is the SGDC model or the baseline model, as the length of the dialogue increases, the perplexity score increases monotonously to varying degrees, which shows that the longer the dialogue, the more complex the semantic modeling of the dialogue and the more difficult it is to capture information associations, and The model is vulnerable to irrelevant semantic noise, resulting in a decline in predictive power.

2 The SGDC model of the present invention performs better than the baseline model no matter in the test set of short, medium or long dialogues, which proves the robustness of the model of the present invention. In addition, compared with the best benchmark model, SGDC has the largest improvement in long-form dialogues. It can be seen that the graph-domain channel of SGDC has indeed played a role in capturing long-distance semantic dependencies, and has played a unique role in semantic modeling of long-form dialogues. Some ability advantages.

The impact of the cooperative mode of dual-channel semantics on performance

In order to explore the RQ3 problem, this embodiment changes the two-channel information collaboration method in the SGDC model, and designs three variant models to explore the best aggregation strategy for the semantic information of the two channels. The three variant models are:

SGDC _Avg : The SGDC _Avg model chooses the average aggregation strategy. This collaborative method assumes that the semantic information in the two channels is equal, so the semantic representation of the entire context is obtained by averaging pooling for the two semantic vectors;

SGDC _Max : The SGDC _Max model focuses on selecting the most important semantic features, and adopts the maximum aggregation strategy. This collaborative method assumes that the maximum value in the semantic vector representation can reflect important semantics. Therefore, the two semantic vectors are pooled through the maximum to obtain a semantic representation of the entire context;

·SGDC _Concat : The SGDC _Concat model believes that the semantic information of the two channels is equally important and cannot be compromised, so the semantic representation of the entire context is obtained through the direct combination of semantic vectors;

Table 5 The results of the impact of collaborative methods on performance

For the convenience of comparison, we refer to the model used in the above experiment as SGDC _Gate . Table 5 records the generation effects of SGDC _Gate and its three variants on the two data sets of MuTual and DailyDialog, among which this example chooses The evaluation index (GM\EA\VE) of semantic relevance can be found through the difference of evaluation index in Table 5:

①SGDC _Gate is significantly better than SGDC _Avg and SGDC _Max in the three indicators of semantic relevance. This shows that the semantic information of the two channels has its own uniqueness and importance. Only by finding the optimal aggregation strategy through the Gate mechanism can the maximum benefits of semantic information aggregation be guaranteed, and the average aggregation strategy of SGDC _Avg may cause respective losses. Losing important semantics, SGDC _Max 's maximization aggregation strategy focuses too much on important semantics, which belongs to local correlation, and it is difficult to capture the balance of semantic associations, thus losing the overall semantic relevance of the reply text.

②SGDC _Gate is not much different from SGDC _Concat in the evaluation of EA and VE, but it is obviously better than SGDC _Concat in GM. This curious phenomenon can be explained from the specific details of the evaluation index. EA and VE respectively measure the average level and extreme value level of the word embedding similarity between the predicted text and the real text. SGDC _Gate finds the balance point of aggregation through the Gate mechanism , SGDC _Concat accepts everything as it is ordered. The semantic enhancement of the graph-domain channel itself captures long-distance dependencies. Therefore, the two are precise solution and brute force solution respectively. big difference. The GM value not only considers the similarity of word embedding, but also considers the alignment between words, so it is a more fine-grained evaluation item. At this time, the advantages of the Gate mechanism are reflected. SGDC _Gate can filter more finely than SGDC _Concat Granular Semantic Noise.

The present invention draws on the rich research ideas of graph neural network technology, proposes a collaborative semantic modeling and reasoning method based on dual channels of sequence and graph domain, and designs a dual-gated graph neural network based on nodes on the graph. Moreover, the model of the present invention has been verified experimentally on both the open domain data set and the dialogue reasoning data set. The experimental results show the advantages of the dual-channel collaborative semantic modeling and reasoning method in various evaluation items. With the increase of rounds, the model of the present invention still has good robustness.

The present invention starts from the analysis and reuse of content features and dual-channel enhancement of structural features, and the proposed model scheme has relatively good practical application value:

(1) The generative dialogue method only considers the progressive semantics of the sequence structure, ignoring the long-distance contextual strong interaction. How to effectively and comprehensively utilize the context information, the present invention adopts the most direct method, that is, breaks the sequence structure, learns from the thinking mode of repeated thinking in human dialogue, and uses the context information to design a heterogeneous graph that can be cognitively reasoned. The idea of "and then stand" can expand the validity of the research.

(2) During the information transmission process of graph neural network update node representation, the present invention designs a double-gated GNN, which performs filtering and screening of information transmission through the GRU unit of information aggregation and the Gate mechanism of information combination. This design can effectively filter semantic noise and capture key semantic information, which is an attempt in the field of dialogue generation.

A second aspect of the present invention provides a terminal device, including a memory, a processor, and a computer program stored in the memory and operable on the processor. When the processor executes the computer program, the steps of the above method are implemented.

The invention proposes a sequential and graph-domain dual-channel collaborative semantic modeling and reasoning method, aiming to integrate semantic advantages in different structural modeling and obtain information association and semantic reasoning with a larger span. In detail, on the one hand, the present invention constructs a dialogue-level heterogeneous cognitive graph. The graph nodes are the integration of topic semantics and sentence cluster semantics. The edges in the graph represent the degree of topic overlap between sentences. Learning, to obtain the semantic representation of the dialogue context on the graph domain; on the other hand, embedding a hierarchical attention mechanism in the preserved sequential channel obtains the sequential semantic representation of the dialogue context. Finally, the information contributions of the two semantic representations are reconciled for prediction. The model of the present invention performs well on the benchmark model, and alleviates the problem of long-distance semantic dependence.

The above are only preferred embodiments of the present invention, and do not limit the present invention in any form. Although the present invention has been disclosed above with preferred embodiments, it is not intended to limit the present invention. Any person familiar with the art, without departing from the scope of the technical solution of the present invention, can use the technical content disclosed above to make many possible changes and modifications to the technical solution of the present invention, or modify it into an equivalent implementation of equivalent changes example. Therefore, any simple modifications, equivalent changes and modifications made to the above embodiments according to the technical essence of the present invention shall fall within the protection scope of the technical solution of the present invention.

Claims (10)

1. A multi-round dialogue reply generation method based on dual-channel semantic enhancement, characterized in that it comprises: Obtain the initial word vector of the dialogue text; Obtaining the sequential semantic representation of the initial word vector includes obtaining the utterance-level sentence semantic vector of the initial word vector, determining the dialogue-level sentence semantic vector of the initial word vector according to the utterance-level sentence semantic vector, and converting the dialogue Sentence-level semantic vectors are recorded as sequential semantic representations; Obtaining the graph domain semantic representation of the initial word vector on the graph domain; performing semantic enhancement on the dialog text according to the sequential semantic representation and the graph domain semantic representation, to obtain an enhanced semantic representation; A reply text is generated according to the enhanced semantic representation. 2. The method according to claim 1, wherein obtaining the discourse-level sentence semantic vector of the initial word vector specifically comprises: The initial word vector is input into the sentence layer encoder and the word attention module in turn to obtain the discourse-level sentence semantic vector; Determine the dialogue-level sentence semantic vector of the initial word vector according to the utterance-level sentence semantic vector, specifically including: The discourse-level sentence semantic vector is input into the context encoder and the sentence attention module in sequence to obtain the dialogue-level sentence semantic vector; Both the sentence layer encoder and the context encoder are bidirectional gated neural networks; The mechanisms used in the word attention module and the sentence attention module are attention mechanisms. 3. The method according to claim 1, wherein obtaining the semantic representation of the initial word vector on the graph domain includes: Obtain the topic keywords of the initial word vector, determine the nodes of the heterogeneous cognitive map according to the topic keywords, and the nodes include topic-sentence cluster nodes, dialogue query nodes and common nodes; According to the nodes of the heterogeneous cognitive graph, determine the edges of the heterogeneous cognitive graph and the weight of each edge, and the weight is determined according to the degree of topic overlap between sentences in the dialogue text corresponding to the initial word vector; A graph neural network is used to learn the vector representation of the nodes in the heterogeneous cognitive graph, and obtain the graph domain semantic representation of the initial word vector on the graph domain. 4. The method according to claim 1, wherein, according to the sequential semantic representation and the graph domain semantic representation, performing semantic enhancement on the dialog text, specifically comprising: The dialogue text is semantically enhanced according to a first formula, and the first formula is:

In the formula, c _final is the enhanced semantic representation,

For the sequential semantic representation,

is the graph-domain semantic representation, δ is the semantic quantity in the sequential semantic representation, and (1-δ) is the semantic quantity in the graph-domain semantic representation. 5. The method according to any one of claims 1-4, wherein generating a reply text according to the enhanced semantic representation specifically includes: Input the enhanced semantic representation into the one-way gated neural network to obtain the hidden state of each word in the generated reply text; According to the hidden state, the generation probability of each word is determined, and the reply text is determined according to the generation probability. 6. The method according to claim 5, wherein the enhanced semantic representation is input into a one-way gated neural network to obtain the hidden state of each word in the generated reply text, specifically comprising: Generate the hidden state of each word in the reply text according to the second formula, the second formula is:

In the formula, y _i is the i-th word in the reply text generated in the training phase, and y _i-1 is the i-1th word in the reply text generated in the training phase,

is the hidden state of y _i , and GRU( ) means that the parameters in it are input into the gated neural network,

is the hidden state of y _i-1 , and c _final is the enhanced semantic representation. 7. The method according to claim 5, wherein, according to the hidden state, determining the generation probability of each word specifically comprises: Determine the generation probability of each word according to the third formula, the third formula is:

In the formula,

generate the i-th word in the reply text for the prediction phase,

for

The generation probability of

with

Respectively, the i-th word in the topic keyword vocabulary and reply text vocabulary in the prediction stage

the generation probability of . 8. The method of claim 7, wherein the

Determined according to the fourth formula, the fourth formula is:

In the formula, η(·) is a non-linear function tanh, V is a reply text vocabulary, K is a subject keyword vocabulary,

Generate the hidden state of the i-th word y _i in the reply text for the training phase, y _i-1 generates the i-1th word in the reply text for the training phase, c _final is the semantic representation after the enhancement, and vocab represents the variable i . 9. The method of claim 7, wherein the

Determined according to the fifth formula, the fifth formula is:

In the formula, η(·) is a non-linear function tanh, V is a reply text vocabulary, K is a subject keyword vocabulary,

Generate the hidden state of the i-th word y _i in the reply text for the training phase, y _i-1 generates the i-1th word in the reply text for the training phase, c _final is the semantic representation after the enhancement, and vocab represents the variable i . 10. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and operable on the processor, characterized in that, when the processor executes the computer program, the computer program according to claim The step of any one of 1 to 9 methods.

Images