CN108491382A - A kind of semi-supervised biomedical text semantic disambiguation method - Google Patents

Abstract

The invention is a semantic disambiguation method for polysemous words in biomedical texts. It mainly includes: using Word2Vec to perform vectorized representation of words in biomedical texts, constructing a vectorized representation of contextual sentences based on the word vector language model based on the bidirectional LSTM model, and then using the relationship between the sentence vector space similarity and combining the label transfer method to convert the existing The labels of the annotated medical data are passed to the most similar unlabeled data according to the probability, and finally the semantic disambiguation of the biomedical text is combined with all the labeled data. Because biomedical data has the characteristics of strong professionalism and many terms, manual processing of medical data is time-consuming, labor-intensive and error-prone. Using the present invention can greatly reduce the cost of manual labeling. At the same time, compared with traditional machine learning methods, it can Effectively improve the accuracy of semantic disambiguation.

Description

A Semi-supervised Method for Semantic Disambiguation of Biomedical Text

technical field

The invention belongs to the field of natural language processing semantic disambiguation, and is a method and system based on semi-supervised biomedical text semantic disambiguation. Specifically, it refers to the semantic disambiguation of polysemous words in medical texts by using the bidirectional long short-term memory model Bi-LSTM based on the label transfer method.

Background technique

In recent years, with the explosive growth of digital information, it has become easier for medical staff to obtain medical electronic data. In the field of biomedicine, text data contains a large amount of knowledge and information in professional fields, and how to extract useful information from digital text information is becoming more and more important. Compared with general text data, the difficulty of medical text data lies in its strong professionalism and difficult data labeling. Therefore, understanding the semantic information of biomedical texts and automatically labeling medical data has become a research hotspot.

Traditional semantic disambiguation methods for biomedical texts include supervised learning methods, unsupervised learning methods and knowledge base-based learning methods. Supervised learning methods use labeled data to learn a latent classifier, and then use this classifier to make predictions for the latent semantics of unseen data. This method usually requires a large amount of labeled data to ensure the high accuracy of the classifier, and its manual labeling process is time-consuming and labor-intensive, so it is not the best choice when the amount of data is not large in some fields of biomedicine. Unsupervised learning methods do not require labeled data, the data is only classified based on potential similarities. The unsupervised learning method greatly simplifies the engineering of manual labeling data. However, the accuracy of this method still needs to be further improved, and it is not suitable for fields with low error tolerance rates such as the medical field. The knowledge base-based method uses the constructed and open-source medical knowledge base as training samples. The advantage of this method is high data reliability, but the disadvantage is that the knowledge base construction has poor scalability and is difficult to maintain.

Semantic disambiguation of biomedical texts often uses the word vector model to vectorize each word in the text, and the semantic information of words is stored in a low-dimensional space in the form of vectors, and similar semantic words have similar word vector representations. Common word vector conversion technologies include the Word2Vec model, which includes the Skip-gram model and the CBOW model. Among them, the Skip-gram model uses the target word to predict the word vector of the adjacent window word, and the CBOW model uses the adjacent window word to predict the word vector of the target word. Similarly, the sentence vector uses the word vector feature of each word in the sentence to represent the semantic information of the sentence. Common traditional fusion methods include cascading, averaging, and weighted summation. The cascading method is to directly splice the word vectors of each word in the sentence in order; the averaging method is to combine all the word vectors in the sentence The vectors are averaged to obtain sentence vectors; the weighted summation method is to assign different weights to each word according to the importance of semantic information, and then add and sum according to the weights to obtain sentence vectors. Sentence vectors are often used as features to initialize language models to facilitate subsequent natural language processing tasks.

Recurrent neural network (RNN) is a common neural network model for processing text information. It is characterized by being able to connect information from previous moments to tasks at the current moment, and has a certain degree of memory. However, when dealing with long sentences, theoretically RNNs can handle long-term dependencies. But in practice, Bengio.et al (1994) and others conducted in-depth research on this problem and found that RNN cannot successfully learn this knowledge. When the distance between words is far away, RNN may cause gradient explosion or gradient disappearance, resulting in Backpropagation fails and text information cannot be effectively preserved. In order to overcome this shortcoming, an improved model of RNN - long short-term memory model (LSTM) is proposed. The model adds three "gate" structures based on the internal structure of the RNN. The "forget gate" determines how much information is retained at the previous moment, the "input gate" determines how much information is retained at the current moment, and the "output gate" determines the output at the current moment. How much information. LSTM selectively utilizes the information of the previous moment and the information of the current moment through this special gate structure, effectively avoiding the problem of long-term dependence of RNN.

In recent years, semi-supervised learning has been successfully applied to semantic disambiguation tasks, and the bootstrapping algorithm can achieve better accuracy. A low-recall classifier can learn from a small set of labeled examples, and then expand the labeled set with these sentences, using this classifier to label unlabeled corpus with high confidence. In recent years, a label propagation algorithm for word sense disambiguation has been proposed. And compared with bootstrapping and support vector machine (SVM) supervised classifiers. Label propagation can achieve better performance because it assigns labels by optimizing the global objective, while traditional algorithms such as bootstrapping propagate labels based on instance local similarities.

Contents of the invention

The invention provides a method and system for semantic disambiguation of biomedical texts based on semi-supervised learning and deep learning. To a certain extent, it solves the problems of traditional disambiguation methods such as poor globalization, difficulty and high cost of manual labeling, and improves the accuracy of semantic disambiguation of biomedical texts and general texts.

The present invention consists of two parts: 1. Based on the two-way long short-term memory network LSTM model, the word vector is fused to form a sentence vector, and the semantic features of the sentence are generated. 2. A semi-supervised semantic disambiguation model based on the label transfer method, which uses the similarity of labeled data to automatically label unlabeled data and eliminate semantic ambiguity at the same time.

The technical scheme that the present invention adopts comprises the steps:

(1) Based on the two-way long short-term memory network LSTM model to form a sentence vector, and generate the semantic features of the sentence

The bidirectional long short-term memory network LSTM model includes: an output layer, a backward hidden layer, a forward hidden layer, and an input layer. Among them, at each time step, six unique weights are recycled, and the six weights correspond to the following: input layer to forward and backward hidden layers (w1, w3), hidden layer to hidden layer (w2, w5), forward and backward hidden layers to the output layer (w4, w6)

The hidden layer is an LSTM model, and the LSTM model consists of three gates (forget gage, input gate, output gate) and a memory unit (cell)

The word vector of each word is used as the input of the bidirectional cyclic neural network LSTM, and together with the output of the previous moment, the current output is obtained. The process is divided into three stages

The first stage: the forget gate layer selectively filters the information of the previous moment through the sigmoid function

in, For the previous moment output, is the current input, that is, the current word vector, A value from 0 to 1, used to filter the information learned at the last moment

Phase 2: Generate new information that needs to be updated

First, the input gate layer decides which values to update through sigmoid

Then a tanh layer is used to generate new candidate values

Candidate values for new information to refresh

The third stage: the output of the model

Get an initial output through the sigmoid layer

Then by the tanh function will Scale and multiply the two to get the output of the model

(2). Semi-supervised semantic disambiguation model based on label transfer method

The label transfer method uses the similarity between sample data to transfer the label of the labeled data to the unlabeled data according to the probability. First build a graph model for all samples, where each sample is a node, and the node and The similarity calculation method of is:

in is a hyperparameter. Each node propagates the label according to the probability according to the similarity with the surrounding nodes. The probability calculation method is:

n represents the number of edges.

Description of drawings

Fig. 1 is a schematic diagram of the system of the present invention.

Fig. 2 is a diagram of the internal structure of the LSTM of the present invention.

Detailed ways

(1) The user inputs biomedical text and generates sentence vector features

First, divide the biomedical text into phrases, then use the Word2Vec model to generate word vectors for the phrases, and then input the word vectors in each sentence into the two-way long-term and short-term memory model, and the model will output two sentence vectors, respectively and , form a new sentence vector by cascading :

The new sentence vector is then input into the multi-layer perceptron get the final sentence vector :

.

(2) Use the label transfer method to automatically label unlabeled data and disambiguate ambiguous words

Use the sentence vector feature obtained in (1) as a vector graph node, calculate the similarity of each node, automatically propagate the most similar label for unlabeled data according to the label transfer method, and for ambiguous words, also transfer the most suitable sentence vector according to the similarity semantic information.

(3) Experimental results

According to step (1) and step (2), the international general medical text MSH WSD dataset and NLM WSD dataset are used. Among them, the MSH WSD dataset contains 203 medical ambiguous entities, with a total of 37,888 ambiguous sentences, of which 37,090 samples are manually labeled; the NLM WSD dataset contains 50 ambiguous entities, including 552,153 common sentences, and each ambiguous entity is manually labeled 100 sample. This experiment adopts the ratio of 20: 1, randomly increases the unmarked data of one-twentieth of the original marked data from other medical corpus, tests according to the semi-supervised model based on the label transfer method proposed by the present invention, and its experimental results are compared as follows:

Table 1. Experimental results of MSH WSD dataset.

Table 2. Experimental results of NLM WSD dataset.

Among them, SVM means using support vector machine as a model, LSTM means using a one-way long-term short-term memory model, Bi-LSTM means using a two-way long-term short-term memory model; WE(Con) means using concatenated word vectors as sentence semantic features, WE(Avg) Represents the use of the average word vector method as the sentence semantic feature, WE (Wsum) represents the adoption of the weighted sum word vector method as the sentence semantic feature, Con represents the model used in the present invention as the sentence semantic feature; LP represents the use of the label transfer proposed by the present invention Law. According to the experimental results, it can be seen that the language model proposed in the present invention does not need manual labeling after adding unlabeled data, which reduces the cost of manual labeling by medical personnel, and also achieves the best accuracy in medical text semantic disambiguation. degree, it is proved that the invention is indeed feasible and effective.

Claims (5)

1. A semi-supervised biomedical text semantic disambiguation method is characterized by comprising the following steps:

(1) vectorization representation of words of medical text based on Word2Vec language model

(2) Performing vectorization representation on medical text sentences based on bidirectional long-term and short-term model Bi-LSTM on the basis of word vectors

(3) And automatically labeling the label-free data based on a label transfer method by using sentence vector space similarity, and performing semantic disambiguation on the polysemous words.

2. The Word2 vectored language model-based vectorized representation of words of medical text according to claim 1, wherein: the words may include both medical specific terms and general text words.

3. The Bi-directional long-short term memory model Bi-LSTM based vectorized representation of medical text sentences as claimed in claim 1 wherein: the bidirectional long-short term memory model Bi-LSTM inputs the word vector representation of each word in the sentence, and outputs the vectorized representation of the sentence.

4. The sentence vector spatial similarity of claim 1, wherein: and calculating the geometric distance between the sentence vectors by using an Euclidean distance formula, and calculating the similarity of the sentence vectors by using the reciprocal of the geometric distance.

5. The label delivery-based automated tagging of unlabeled data and semantic disambiguation of ambiguous words of claim 1, wherein: and (4) transferring the data label to the unlabeled data according to probability by using the similarity between the sentence vectors, and automatically carrying out semantic disambiguation on the medical text data.