CN113436752B - Semi-supervised multi-round medical dialogue reply generation method and system - Google Patents

Abstract

The invention belongs to the field of conversational information processing, and provides a semi-supervised multi-round medical conversation reply generation method and system. The method comprises the steps of inputting the problems of a patient in a first round of dialogue into a semi-supervised medical dialogue model to obtain replies of the first round of dialogue; in the second round and the later conversations, inputting the problems of the current round of patients and the replies of the previous round of conversations into a semi-supervised medical conversation model to obtain replies of the corresponding round of conversations until the patients have no new problem input; the semi-supervised medical dialogue model comprises a context encoder, a priori state tracker, an inference strategy state tracker, a priori strategy network, an inference strategy network and a reply generator, wherein the context encoder is used for encoding received information and inputting the information into the priori state tracker and the priori strategy network, the priori state tracker is used for continuously tracking the physical state of a user, the priori strategy network is used for generating a doctor action, and the reply generator is used for generating a corresponding reply according to the physical state and the doctor action.

Description

A semi-supervised multi-round medical dialogue response generation method and system

Technical Field

The present invention belongs to the field of conversational information processing, and in particular relates to a semi-supervised multi-round medical conversation response generation method and system.

Background Art

The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

In order to address both the information needs of open domains and the professional needs of highly vertical domains, the conversational paradigm is used to connect people with information. Existing dialogue systems can be divided into two categories: task-oriented and open-domain dialogue systems. Task-oriented dialogue systems are designed to help people complete specific tasks. For example, scheduling, booking restaurants, and checking the weather. Open-domain dialogue systems are mainly used to chat with people to meet people's needs for information and entertainment. Unlike medical Q&A, conversations in real medical scenarios are more likely to contain multiple rounds of interactions. Because the patient needs to express his/her symptoms, the medicines he/she is taking, and his/her medical history through the context of the conversation. This feature makes explicit state tracking indispensable, which provides more indicative and interpretable information than hidden state representation. Considering the particularity of medical dialogue, medical reasoning capabilities (such as whether to prescribe medicine, what medicine to prescribe to treat the disease, and what symptoms to ask) are also indispensable features in medical diagnosis.

Existing medical dialogue methods are based on the task-oriented dialogue paradigm, following the paradigm that the patient expresses symptoms and the dialogue system returns the diagnosis result (i.e., determines what disease the patient has). It has achieved good results. However, these methods only focus on the single field of diagnosis, which cannot meet the various needs of patients in actual applications, and require a large number of manually labeled states and actions. This is impossible when the dialogue data is highly confidential or the data scale is huge, and these works are limited by the impact of the training data scale, and even generative methods cannot be used to generate replies, and replies can only be composed through templates. Some task-based dialogue methods can be applied to state tracking in medical dialogues, but they still cannot cope with scenarios without sufficient labeled data. In order to reduce the demand for data labeling in task-oriented dialogue systems, Jin et al. and Zhang et al. both used semi-supervised learning methods for state tracking, but ignored the reasoning ability of the dialogue subject, that is, the actions of the physician were not modeled. Liang et al. proposed a method to use incompletely labeled data to train specific modules in task-oriented dialogue systems, but could not infer unlabeled labels at the time of training, resulting in limited improvement in medical dialogue systems without both state and action labeling. The inventors found that these methods did not consider retrieval from large-scale medical knowledge, failed to generate knowledge-rich responses, and performed poorly in scenarios such as medical dialogues that have strong demands for reasoning ability.

Summary of the invention

In order to solve the technical problems existing in the above-mentioned background technology, the present invention provides a semi-supervised multi-round medical dialogue response generation method and system, which takes into account both the patient status and the doctor's actions, so that the dialogue system has the ability to model the user's physical status and medical reasoning at the same time.

In order to achieve the above object, the present invention adopts the following technical solution:

A first aspect of the present invention provides a semi-supervised method for generating responses to multi-round medical conversations.

A semi-supervised multi-round medical dialogue response generation method, comprising:

Input the patient's questions in the first round of dialogue into the semi-supervised medical dialogue model to obtain the responses in the first round of dialogue;

In the second and subsequent rounds of dialogue, the patient's questions in the current round and the responses in the previous round are input into the semi-supervised medical dialogue model to obtain responses for the corresponding round of dialogue until the patient has no new questions input;

Among them, the semi-supervised medical dialogue model includes a context encoder, a priori state tracker, an inference state tracker, a priori policy network, an inference policy network and a reply generator. The context encoder is used to encode the received information and input it into the priori state tracker and the priori policy network. The prior state tracker is used to continuously track the user's physical state. The priori policy network is used to generate the doctor's corresponding actions. The reply generator is used to generate corresponding replies according to the physical state and the doctor's actions.

The inference state tracker is used to infer the user's physical state, and the inference strategy network is used to infer the doctor's actions; the inference state tracker and the inference strategy network are only executed during the training phase of the semi-supervised medical dialogue model.

A second aspect of the present invention provides a semi-supervised multi-turn medical dialogue response generation system.

A semi-supervised multi-turn medical dialogue response generation system, comprising:

A first-round dialogue response generation module, which is used to input the patient's questions in the first-round dialogue into the semi-supervised medical dialogue model to obtain the responses to the first-round dialogue;

The second and subsequent dialogue response generation module is used to input the patient's questions in the current round and the responses in the previous round of dialogue into the semi-supervised medical dialogue model in the second and subsequent dialogues, and obtain the responses in the corresponding round of dialogues until the patient has no new questions input;

Among them, the semi-supervised medical dialogue model includes a context encoder, a priori state tracker, an inference state tracker, a priori policy network, an inference policy network and a reply generator. The context encoder is used to encode the received information and input it into the priori state tracker and the priori policy network. The prior state tracker is used to continuously track the user's physical state. The priori policy network is used to generate the doctor's corresponding actions. The reply generator is used to generate corresponding replies according to the physical state and the doctor's actions.

The inference state tracker is used to infer the user's physical state, and the inference strategy network is used to infer the doctor's actions; the inference state tracker and the inference strategy network are only executed during the training phase of the semi-supervised medical dialogue model.

A third aspect of the present invention provides a computer-readable storage medium.

A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps in the semi-supervised multi-round medical dialogue response generation method as described above.

A fourth aspect of the present invention provides a computer device.

A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps in the semi-supervised multi-round medical dialogue response generation method as described above are implemented.

Compared with the prior art, the present invention has the following beneficial effects:

(1) In the second and subsequent rounds of dialogue, the present invention inputs the patient's questions in the current round and the replies in the previous round of dialogue into the semi-supervised medical dialogue model to obtain replies for the corresponding round of dialogue until the patient has no new questions to input. The user's physical state and the doctor's actions are explicitly modeled and represented by text spans, thereby improving the model's ability to model the patient's physiological state and conduct medical reasoning.

(2) At the model level, the present invention treats the user's physical state and the doctor's actions as latent variables, and proposes a model training method with and without intermediate annotations (i.e., supervision). This method greatly reduces the dependence of the dialogue model on labeled data.

(3) The present invention proposes to use the tracked patient status to retrieve from a large-scale medical knowledge graph during the process of policy network learning. The explicit status, action and reasoning path in the medical knowledge graph improves the interpretability of the responses generated by the dialogue system.

(4) In model training, the present invention proposes a two-stage cascading reasoning method to improve the stability when there is less supervised training data.

Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

FIG. 1( a ) is a diagram of supervised data training according to an embodiment of the present invention;

FIG1( b ) is an unsupervised data training of an embodiment of the present invention;

FIG1( c ) is a module used in the test phase of an embodiment of the present invention;

FIG2 is a specific implementation method of the medical dialogue system according to an embodiment of the present invention;

FIG. 3 is a schematic diagram of a model during a training process according to an embodiment of the present invention.

DETAILED DESCRIPTION

The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

It should be noted that the following detailed descriptions are all illustrative and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and/or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and/or combinations thereof.

Terminology explanation:

Encoder-Decoder: A neural network structure that encodes a word sequence and then decodes it into another word sequence. It is mainly used in machine translation, dialogue systems, etc.

Encoding: Representing a word sequence as a continuous vector.

Decoding: Represent a continuous vector as a target sequence.

Expectation: The sum of each possible outcome in an experiment multiplied by its outcome probability, which is expressed in the form of E· in the present invention.

KL Divergence: It is an asymmetric measure of the difference between two probability distributions. The present invention uses the form of KL(·||·), and its calculation formula is as follows:

Where q and p represent two discrete distributions, and q(i) and p(i) represent the probability values of the i-th item of distribution q and p respectively.

Latent variable: latent variable, or hidden variable, latent variable, in statistics represents unobservable random variable, as opposed to observed variable.

Training phase (Train): The training phase of the neural network model receives training data as input and continuously adjusts the parameters in the neural network model through training samples.

Test phase: After the neural network model is trained, the trained neural network model parameters are used to output the labels and other information corresponding to the input data in the test phase. We will also call it the deployment phase.

Embodiment 1

This embodiment provides a semi-supervised multi-round medical dialogue response generation method, which includes:

Input the patient's questions in the first round of dialogue into the semi-supervised medical dialogue model to obtain the responses in the first round of dialogue;

In the second and subsequent rounds of dialogue, the patient's questions in the current round and the responses in the previous round are input into the semi-supervised medical dialogue model to obtain responses for the corresponding round of dialogue until the patient has no new questions input;

Among them, the semi-supervised medical dialogue model includes a context encoder, a priori state tracker, an inference state tracker, a priori policy network, an inference policy network and a reply generator. The context encoder is used to encode the received information and input it into the priori state tracker and the priori policy network. The prior state tracker is used to continuously track the user's physical state. The priori policy network is used to generate the doctor's corresponding actions. The reply generator is used to generate corresponding replies according to the physical state and the doctor's actions.

The inference state tracker is used to infer the user's physical state, and the inference strategy network is used to infer the doctor's actions; the inference state tracker and the inference strategy network are only executed during the training phase of the semi-supervised medical dialogue model.

The context encoder is used to encode the received information; the patient's questions in the first round of dialogue are directly encoded; for the patient's questions in the second and subsequent rounds of dialogue and the corresponding replies in the previous round of dialogue, the context information is encoded and input into the five modules of prior state tracker, reasoning state tracker, prior strategy network, reasoning strategy network and reply generator.

采样的到的状态实例

输出概率分布

The input signal of the prior state tracker is: the output probability distribution of the inference state tracker in the previous round of dialogue

Sampled state instance

Output probability distribution

采样的到的状态实例

和当前轮的医师回复R_t，输出概率分布

The input signal of the reasoning state tracker is: the output probability distribution of the reasoning state tracker in the previous round of dialogue

Sampled state instance

And the current round of physician responses R _t , output probability distribution

采样的到的状态实例

以及外部医疗知识图谱G，输出概率分布

The input signal of the prior policy network is: the output probability distribution of the inference state tracker in the current round of dialogue

Sampled state instance

And the external medical knowledge graph G, output probability distribution

采样的到的状态实例

和当前轮的医师回复R_t，输出概率分布

The input signal of the reasoning policy network is: the output probability distribution of the reasoning state tracker in the current round of dialogue

Sampled state instance

And the current round of physician responses R _t , output probability distribution

以及从概率分布

中采样得到的实例

作为输入；测试阶段则接收

以及从概率分布

中采样得到的实例

作为输入，输出对话回复信息R_t。The input signal of the reply generator is: The reply generator input is divided into two situations: training phase and test phase (that is, when deployed). In the training phase, it receives

And from the probability distribution

The sampled examples

As input; the test phase receives

And from the probability distribution

The sampled examples

As input, the dialogue reply information R _t is output.

In the actual deployment stage, in each dialogue round, given the patient's statement, the medical dialogue system uses the prior state tracker to continuously track the user's physical state, and uses the prior policy network to generate the doctor's corresponding actions. Finally, the reply generator combines the state and actions sampled from the prior state tracker and the prior policy network to generate the corresponding reply, corresponding to the process in Figure 1(c). The dialogue process continues until the patient has no new questions to input, that is, the patient actively ends the current dialogue.

Medical dialogue systems have two key features: patient status (symptoms, medications, etc.) and physician actions (treatment, diagnosis, etc.). These two features make medical dialogue systems more complex than other knowledge-intensive dialogue scenarios. Similar to task-oriented dialogue systems, the medical dialogue generation process is divided into the following three stages:

(1) Patient state tracking: For a given dialogue history, the dialogue system tracks the physical state of the patient.

(2) Physician strategy learning: Given the patient status and dialogue history, the dialogue system gives the current physician’s action;

(3) Medical response generation: Given the conversation history, tracked states, and predicted actions, give fluent and accurate natural language responses.

For scenarios with labeled data, in the tth round of the conversation, the patient asks a question or describes his symptoms U _t , and then the medical dialogue system receives the previous round's reply R _t-1 , the current round's question U _t and the state _St-1 tracked in the previous round, and then outputs the current round's state St _t , and then uses R _t-1 U _t St _t to output the action _At that the physician should take in the current round, and finally generates a natural language reply R _t to feed back to the patient. However, in medical dialogue systems, in many cases, the patient's physiological state and the physician's action are not labeled. Therefore, we regard both state and action as hidden variables, and considering that state runs through the entire conversation process, we use a sequence of words to represent it; the same is true for physician actions, that is, the physician's reply may contain multiple keywords. In actual operation, the length of state and action is set to a fixed length of |S| and |A| respectively. And the state has an initial value, which is "<pad><pad>...<pad>", where "<pad>" represents a filler word. The design details of State and Action are as follows:

State design: state is used to record the information about the user's physical state obtained by the dialogue system during the entire dialogue process. It is represented by a sequence of words, such as "cold, fever, cough, night sweats...", and is initialized to "<pad><pad>......<pad>".

Action design: Action is used to express the summary of the doctor's reply, which is also expressed as a sequence, such as "999 Ganmao Ling Granules Jizhi Syrup..."

The semi-supervised medical dialogue model consists of six modules: context encoder, prior state tracker, inference state tracker, prior policy network, inference state tracker, and response generator. In a whole medical dialogue, there are often multiple interactions, and the following process goes through multiple rounds until the dialogue ends.

The prior state tracker and the reasoning state tracker are used for patient state tracking, and the reasoning state tracker is only executed in the training phase; the prior policy network and the reasoning policy network are used for physician policy learning, and the reasoning policy network is only executed in the training phase; the response generator is used for medical response generation. The following mainly describes the input and output of each module from an unsupervised perspective, that is, using unsupervised data ^Du , corresponding to Figure 1(b).

来表示对话上下文。In round t, the context encoder is a GRU-based (or LSTM-based, Transformer-based, Bert-based) encoder that receives the previous round’s response R _t-1 and the current round’s patient’s question U _t as input, and outputs a continuous space vector

To represent the conversation context.

来表示对话上下文。其中M和N分别表示R_t-1和U_t序列长度。In the tth round, given the previous round reply Rt _-1 and the current round patient's question _Ut as input, the context encoder first uses the bidirectional GRU Encoder to encode a sequence word granularity representation _Ht = {ht _,1 , _ht,2 ,…, _ht,M+N }, and outputs a vector

to represent the conversation context. Where M and N represent the length of R _t-1 and U _t sequences respectively.

表示R_t-1第i个词的词嵌入(embedding)，这个BiGRU编码器采用了上一个时刻的上下文表示

初始化，attn[17]表示的是attention操作。in

Represents the word embedding of the i-th word in R _t-1 . This BiGRU encoder uses the context representation of the previous moment.

Initialization, attn[17] represents the attention operation.

作为输入，然后采用一个基于GRU的解码器来输出一个序列的词，即

推理状态追踪器采用了先验状态追踪器相近的结构，但是额外接受了当前轮的回复R_t作为输入，其输出一个词序列，即

我们使用

和

分别表示先验状态追踪器和推理状态追踪器，生成概率分布简写为

和

The prior state tracker receives the context encoder output and the state at the previous moment

As input, a GRU-based decoder is then used to output a sequence of words, namely

The reasoning state tracker uses a structure similar to the prior state tracker, but additionally accepts the current round response _Rt as input, and outputs a word sequence, namely

We use

and

They represent the prior state tracker and the inference state tracker respectively, and the generated probability distribution is abbreviated as

and

采样得到

送入先验状态追踪器和推理状态追踪器中。Both the prior state tracker and the reasoning state tracker are encoder-decoder structures. In the absence of supervision information, the states of all dialogue rounds are unknown, and the state of the next round needs to depend on the state of the previous round as input, so we start from

Sampling

Feed it into the prior state tracker and the inference state tracker.

编码为

使用

初始化先验状态追踪器的解码器，其中

为训练参数。在第i个解码时刻，输出

然后序列解码得到S_t的先验分布为:The prior state tracker first converts the sampled

Encoded as

use

Initialize the decoder of the prior state tracker, where

is the training parameter. At the i-th decoding moment, the output

Then the prior distribution of _St obtained by sequence decoding is:

Where MLP stands for Multi-Layer Perceptron. |S| is the length of the state text span.

编码为

另外编码R_t为

其使用

初始化解码器，其中

为训练参数。在第i个解码时刻，输出

然后序列解码得到S_t的近似后验分布：The inference state tracker has a similar structure to the prior state tracker, and it also uses the GRU Encoder to

Encoded as

In addition, R _t is encoded as

Its use

Initialize the decoder, where

is the training parameter. At the i-th decoding moment, the output

Then the sequence is decoded to obtain the approximate posterior distribution of _St :

推理策略网络结构相近，其接收

S_t并且额外接收当前轮回复R_t作为输出，后输出一个序列的词，即

我们使用

和

分别表示先验策略网络和推理策略网络，简写为

和

The prior policy network receives the context encoder output, the current round _St and the external medical knowledge G as input, and then uses a GRU-based decoder to output a sequence of words, i.e.

The reasoning strategy network structure is similar, and its receiving

S _t and additionally receives the current round reply R _t as output, and then outputs a sequence of words, i.e.

We use

and

Represent the prior strategy network and the reasoning strategy network, respectively, abbreviated as

and

采样得到

推理策略网络从

采样得到

The prior strategy network and the reasoning strategy network are also encoder-decoder structures.

Sampling

The reasoning strategy network is

Sampling

表示G_n编码后的节点表示，其中|G_n|为G_n中的节点数量。Before introducing the two policy networks, we first introduce a knowledge graph retrieval operation qsub and a knowledge graph encoding operation RGAT[15]. qsub retrieves a subgraph _Gn from the medical knowledge graph G using the state obtained by tracing from G. Starting from state, it extracts all nodes and edges that can be reached by n-step jumps, and connects all nodes that appear in state to ensure that the graph _Gn is fully connected. RGAT is a graph encoding method that combines the type of edges and obtains the embedding representation of the node after multiple propagations, that is, a vector representation in a continuous space. We use

represents the node representation after _Gn encoding, where | _Gn | is the number of nodes in _Gn .

编码为

后来使用

来初始化解码器，在第i个解码时刻，输出

解码过程包含两个部分，一种从词表中生成，另一种从检索得到的知识图谱G_n中进行拷贝。The prior policy network uses GRU Encoder to

Encoded as

Later use

To initialize the decoder, at the i-th decoding moment, output

The decoding process consists of two parts, one is generated from the vocabulary, and the other is copied from the retrieved knowledge graph _Gn .

中第j个节点的embedding。Z_A为生成Where e _j represents the jth node in G _n , g _j represents

The embedding of the jth node in _{Z A} is generated

Regularization term of the copy. In the case of e _j = A _t,i, I(e _j , A _t,i ) = 1, otherwise I(e _j , A _t,i ) = 0.

Then the prior distribution of _At can be expressed as:

编码为

编码R_t为

后来使用

初始化解码器，在第i个解码时刻输出

为了强化R_t对于结果的影响，对于A_t近似后验分布，我们只考虑直接的生成概率。The reasoning policy network is encoded using GRU Encoder

Encoded as

The code R _t is

Later use

Initialize the decoder and output at the i-th decoding time

In order to strengthen the impact of R _t on the results, we only consider the direct generation probability for the approximate posterior distribution of A _t .

S_t和A_t作为输入，然后输出医疗回复R_t。使用

表示回复生成器，简写为

The reply generator is a GRU-based decoder that receives the context encoder output

S _t and A _t are used as input, and then the medical response R _t is output.

Represents a reply generator, abbreviated as

和

分别采样得到

和

将其编码为

和

后来初始化回复生成器的解码器为

在第i个解码时刻输出

则得到R_t的输出概率为：The reply generator uses only the output of the inference state tracker and the inference policy network during the unsupervised training phase.

and

Sampled separately

and

Encode it as

and

Later the decoder of the reply generator is initialized as

Output at the i-th decoding time

Then the output probability of R _t is:

表示从词表中生成的概率，

表示从

R_t-1和U_t中拷贝的概率，|R|为回复的长度。in

represents the probability generated from the vocabulary,

Indicates from

R _t-1 and the probability of copying in U _t , |R| is the length of the reply.

The training loss functions for supervised data and unsupervised data are L ^sup and L ^un respectively, where L ^un is:

Where E· represents expectation, and KL(·||·) represents KL divergence (Kullback-Leibler divergence).

再同时优化剩余模块，以提高训练过程中的稳定性。L^un被拆分为L^s和L^a两个训练目标：Considering the instability in training when the proportion of supervised data is small, that is, the prior policy network is easily misled by the wrong state sampled from the prior state tracker, the present invention proposes a two-stage cascade reasoning training method, which divides ^Lun into multiple training parts. Since the policy network depends on the output of the state tracker, it is first optimized.

The remaining modules are then optimized simultaneously to improve the stability during training. L ^un is split into two training objectives, L ^s and L ^a :

In the first training phase, minimizing L ^s improves the model state tracking performance, and in the second phase, minimizing L ^s + L ^a maintains the state tracking effect and the policy learning ability of the training model. We name it the two-stage cascade reasoning training method.

Figure 3 is a schematic diagram of the model during training. global_step is an integer used to record the number of training rounds.

In the semi-supervised scenario, the conversation data used for model training consists of supervised and unsupervised data. Below we introduce the training methods for supervised data ^Da and unsupervised data ^Du respectively.

(a) For the supervised data ^Da

The training samples are sampled from ^Da to form the mini-batch required for training, and the data Rt _-1 , _Ut , St _-1 , _St , _At , and _Rt are obtained. The corresponding input data is sent to the above 6 modules, corresponding to (a) in Figure 1. Negative LogLikelihood (NLL) Loss is used for training. The actual training loss function is:

(b) For the unsupervised data ^Du

采样得到

后将

送入到

和

中。再后来，从

和

中分别采样得到

和

分别作为

和

的输入。再然后从

采样得到

最后

结合R_t-1,U_t生成回复R_t，以上过程对应图1中的(b)。训练loss为L^un(亦可选用L^s+L^a作为训练loss，以提高训练稳定性)。We sample training samples from ^Du to form the mini-batch required for training, and obtain the data R _t - ₁ , U _t , R _t . The labeled data in the middle, St _-1 , St _t , At _, are missing because they are not labeled.

Sampling

Later

Send to

and

Later, from

and

The samples were obtained from

and

As

and

Then from

Sampling

at last

Combine R _t-1 and U _t to generate the reply R _t . The above process corresponds to (b) in Figure 1. The training loss is L ^un (L ^s +L ^a can also be used as the training loss to improve the training stability).

For the entire training data set D = {D ^a , ^Du }, the specific training steps are as follows:

Step 1: Assume that the proportion of the supervised data ^Da to the total training data D is α (0≤α≤1), select a random number between 0 and 1. If it is less than α, go to Step 2; if it is greater than α, go to Step 3.

Step 2: Use supervised data to train the model, corresponding to method (a), with the training loss L ^sup . Update the parameters by gradient descent and then go to Step 4.

Step 3: Use supervised data to train the model, corresponding to method (b), the training loss is ^Lun , and the parameters are updated by gradient descent and then go to Step 4.

Step 4: Determine whether the model converges. If so, go to Step 5, otherwise go to Step 1.

Step 5: Save the model weights and end the training, as shown in Figure 3.

The semi-supervised medical dialogue model is trained using the currently public medical dialogue datasets in the industry and academia. The sampled supervised data and unsupervised data are fed into the model, and the corresponding loss function is calculated and then gradient descent is performed to optimize the model parameters.

和先验策略网络的输出

作为输入)，最后生成回复返回给用户。对话系统持续和病人交互，先验状态追踪器在每个对话轮中先使用前一时刻的状态作为输入，后更新追踪到的病人身体状态，等待一段时间过后未接收到病人新的问题，结束当前会话。After the model training is completed, the model parameters are all fixed, and we can discard the inference state tracker and inference policy network. At this point, the model can be applied to actual conversation scenarios. As shown in Figure 2, given a patient question input model, the context encoder, prior state tracker, prior policy network, and response generator work in sequence (at this moment, the response generator only uses the prior state tracker output

and the output of the prior policy network

As input), the system generates a reply and returns it to the user. The dialogue system continues to interact with the patient. In each dialogue round, the prior state tracker first uses the state of the previous moment as input, then updates the tracked patient's physical state, and ends the current session after a period of time has passed without receiving any new questions from the patient.

Embodiment 2

A semi-supervised multi-turn medical dialogue response generation system, comprising:

A first-round dialogue response generation module, which is used to input the patient's questions in the first-round dialogue into the semi-supervised medical dialogue model to obtain the responses to the first-round dialogue;

The second and subsequent dialogue response generation module is used to input the patient's questions in the current round and the responses in the previous round of dialogue into the semi-supervised medical dialogue model in the second and subsequent dialogues, and obtain the responses in the corresponding round of dialogues until the patient has no new questions input;

Among them, the semi-supervised medical dialogue model includes a context encoder, a priori state tracker, an inference strategy state tracker, a priori strategy network, an inference strategy network and a reply generator. The context encoder is used to encode the received information and input it into the priori state tracker and the priori strategy network. The priori state tracker is used to continuously track the user's physical state. The priori strategy network is used to generate the doctor's corresponding actions. The reply generator is used to generate corresponding replies according to the physical state and the doctor's actions.

The inference state tracker is used to infer the user's physical state, and the inference strategy network is used to infer the doctor's actions; the inference state tracker and the inference strategy network are only executed during the training phase of the semi-supervised medical dialogue model.

Each module in this embodiment corresponds to each step in the first embodiment one by one, and the specific implementation process is the same, which will not be repeated here.

Embodiment 3

This embodiment provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the semi-supervised multi-round medical dialogue response generation method as described above.

Embodiment 4

This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps in the semi-supervised multi-round medical dialogue response generation method as described above are implemented.

Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) containing computer-usable program codes.

The present invention is described with reference to the flowchart and/or block diagram of the method, device (system), and computer program product according to the embodiment of the present invention. It should be understood that each process and/or box in the flowchart and/or block diagram, as well as the combination of the process and/or box in the flowchart and/or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for implementing the functions specified in one or more processes in the flowchart and/or one or more boxes in the block diagram.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device that implements the functions specified in one or more processes in the flowchart and/or one or more boxes in the block diagram.

These computer program instructions may also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and/or one or more boxes in the block diagram.

A person skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims (8)

1. A semi-supervised multi-round medical dialogue response generation method, characterized by comprising: Input the patient's questions in the first round of dialogue into the semi-supervised medical dialogue model to obtain the responses in the first round of dialogue; In the second and subsequent rounds of dialogue, the patient's questions in the current round and the responses in the previous round are input into the semi-supervised medical dialogue model to obtain responses for the corresponding round of dialogue until the patient has no new questions input; Among them, the semi-supervised medical dialogue model includes a context encoder, a priori state tracker, an inference strategy state tracker, a priori strategy network, an inference strategy network and a response generator. The context encoder is used to encode the received information and input it into the priori state tracker and the priori strategy network. The priori state tracker is used to continuously track the user's physical state. The input signal of the priori strategy network is: the state instance sampled from the output probability distribution of the inference state tracker in the current round of dialogue, and the external medical knowledge graph G; The decoding process of the prior policy network consists of two parts, one is generated from the vocabulary, and the other is copied from the retrieved knowledge graph _Gn :

in,

is a continuous space vector representing the conversation context; the prior strategy network uses the GRU autoencoder to

Encoded as

represents the output probability distribution of the inference state tracker in the current round of dialogue

The sampled state instance; MLP represents the multi-layer perception mechanism,

represents the output of the prior strategy network at the i-th decoding time; e _j represents the j-th node in G _n , and g _j represents

The word embedding of the jth node in ; Z _A is the regularization term for generating copies; in the case of e _j = At _,i , I(e _j ,A _t,i ) = 1, otherwise I(e _j ,A _t,i ) = 0; The prior policy network is used to generate physician actions and output probability distribution

Among them, |A| represents the length of the action; The response generator is used to generate corresponding responses according to the physical state and the physician's actions; The inference state tracker is used to infer the user's physical state, and the inference strategy network is used to infer the doctor's actions. The inference state tracker and the inference strategy network are only executed during the training phase of the semi-supervised medical dialogue model. In the unsupervised training process,

and

Sampled separately

and

Encode it as

and

Then, we initialize the decoder of the reply generator as

Output at the i-th decoding time

Then the output probability of R _t is:

in,

represents the probability generated from the vocabulary,

Indicates from

The probability of copying in R _t-1 and U _t , |R| is the length of the reply;

Represents the probability distribution

The examples sampled in ; R _t represents the doctor’s response in the current round; R _t-1 represents the doctor’s response in the previous round; U _t represents the question in the current round;

represents the output probability distribution of the reasoning state tracker in the current round of dialogue;

Represents the output probability distribution of the reasoning strategy network in the current round of dialogue; According to the two-stage cascade inference training method, the training loss function L ^un of unsupervised data is split into two training targets L ^s and L ^a . Since the policy network depends on the output of the state tracker, the inference state tracker and the inference policy network are optimized first, and then the remaining modules are optimized at the same time. in,

Where E· represents expectation, KL(·||·) represents KL divergence (Kullback-Leibler divergence); A _t represents the action that the doctor should take in the current round; S _t represents the output state of the current round; S _t-1 represents the state tracked in the previous round;

represents the output probability distribution of the reasoning state tracker in the previous round of dialogue;

represents the output probability distribution of the reasoning state tracker in the current round of dialogue;

represents the prior distribution of _At ;

represents a reply generator; In the first stage, L ^s is minimized to improve the model state tracking performance, and in the second stage, L ^s + L ^a is minimized to maintain the state tracking effect and the strategy learning ability of the training model. 2. The semi-supervised multi-round medical dialogue response generation method as described in claim 1 is characterized in that the reasoning state tracker and the reasoning strategy network are both encoder-decoder structures. 3. The semi-supervised multi-round medical dialogue response generation method as described in claim 1 is characterized in that the prior state tracker and the prior strategy network are both encoder-decoder structures. 4. The semi-supervised multi-round medical dialogue response generation method as described in claim 1 is characterized in that the response generator is a GRU-based decoder. 5. A semi-supervised multi-round medical dialogue response generation system, characterized by comprising: A first-round dialogue response generation module, which is used to input the patient's questions in the first-round dialogue into the semi-supervised medical dialogue model to obtain the responses to the first-round dialogue; The second and subsequent dialogue response generation module is used to input the patient's questions in the current round and the responses in the previous round of dialogue into the semi-supervised medical dialogue model in the second and subsequent dialogues, and obtain the responses in the corresponding round of dialogues until the patient has no new questions input; Among them, the semi-supervised medical dialogue model includes a context encoder, a priori state tracker, an inference strategy state tracker, a priori strategy network, an inference strategy network and a response generator. The context encoder is used to encode the received information and input it into the priori state tracker and the priori strategy network. The priori state tracker is used to continuously track the user's physical state. The input signal of the priori strategy network is: the state instance sampled from the output probability distribution of the inference state tracker in the current round of dialogue, and the external medical knowledge graph G; The decoding process of the prior policy network consists of two parts, one is generated from the vocabulary, and the other is copied from the retrieved knowledge graph _Gn :

in,

is a continuous space vector representing the conversation context; the prior strategy network uses the GRU autoencoder to

Encoded as

represents the output probability distribution of the inference state tracker in the current round of dialogue

The sampled state instance; MLP represents the multi-layer perception mechanism,

represents the output of the prior strategy network at the i-th decoding time; e _j represents the j-th node in G _n , and g _j represents

The word embedding of the jth node in ; Z _A is the regularization term for generating copies; in the case of e _j = At _,i , I(e _j ,A _t,i ) = 1, otherwise I(e _j ,A _t,i ) = 0; The prior policy network is used to generate physician actions and output probability distribution

Among them, |A| represents the length of the action; The response generator is used to generate corresponding responses according to the physical state and the physician's actions; The inference state tracker is used to infer the user's physical state, and the inference strategy network is used to infer the doctor's actions. The inference state tracker and the inference strategy network are only executed during the training phase of the semi-supervised medical dialogue model. In the unsupervised training process,

and

Sampled separately

and

Encode it as

and

Then, we initialize the decoder of the reply generator as

Output at the i-th decoding time

Then the output probability of R _t is:

in,

represents the probability generated from the vocabulary,

Indicates from

The probability of copying in R _t-1 and U _t , |R| is the length of the reply;

Represents the probability distribution

The examples sampled in ; R _t represents the doctor’s response in the current round; R _t-1 represents the doctor’s response in the previous round; U _t represents the question in the current round;

represents the output probability distribution of the reasoning state tracker in the current round of dialogue;

Represents the output probability distribution of the reasoning strategy network in the current round of dialogue; According to the two-stage cascade inference training method, the training loss function L ^un of unsupervised data is split into two training targets L ^s and L ^a . Since the policy network depends on the output of the state tracker, the inference state tracker and the inference policy network are optimized first, and then the remaining modules are optimized at the same time. in,

Where E· represents expectation, KL(·||·) represents KL divergence (Kullback-Leibler divergence); A _t represents the action that the doctor should take in the current round; S _t represents the output state of the current round; S _t-1 represents the state tracked in the previous round;

represents the output probability distribution of the reasoning state tracker in the previous round of dialogue;

represents the output probability distribution of the reasoning state tracker in the current round of dialogue;

represents the prior distribution of _At ;

represents a reply generator; In the first stage, L ^s is minimized to improve the model state tracking performance, and in the second stage, L ^s + L ^a is minimized to maintain the state tracking effect and the strategy learning ability of the training model. 6. The semi-supervised multi-round medical dialogue response generation system as described in claim 5 is characterized in that the reasoning state tracker and the reasoning strategy network are both encoder-decoder structures; the prior state tracker and the prior strategy network are both encoder-decoder structures. 7. A computer-readable storage medium having a computer program stored thereon, characterized in that when the program is executed by a processor, the steps in the semi-supervised multi-round medical dialogue response generation method as described in any one of claims 1 to 4 are implemented. 8. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps in the semi-supervised multi-round medical dialogue response generation method as described in any one of claims 1 to 4 are implemented.

Images