CN113641888B - Event-related news filtering learning method based on fusion topic information enhanced PU learning - Google Patents

Abstract

The invention relates to a learning method for event-related news filtering based on fused topic information enhanced PU learning. The present invention extracts topic information from marked and unlabeled event-related news data sets through unsupervised pre-training, and then adds the extracted topic information to the initial training and subsequent iterative training process of PU learning to ensure that In the case of fewer initial event-related news samples, more sample information can be utilized, and topic enhancement is carried out in the subsequent iterative training process, so that the classifier trained in each iteration can be obtained from unlabeled data. Really reliable positive and negative sample data to improve the performance of the final event-related news classifier. Compared with the baseline model of PU learning, the present invention improves the F1 value by 1.8%, and leads more in the case of low initial samples and high iterations. The method of the present invention for enhancing PU learning by using subject information can effectively solve the problem of lack of training data in case-related news filtering tasks.

Description

Learning of Event-Related News Filtering Based on Fusion Topic Information Enhanced PU Learning method

technical field

The invention relates to a learning method for event-related news filtering based on fused topic information enhanced PU learning, and belongs to the technical field of natural language processing.

Background technique

The event-related news filtering task can usually be regarded as a binary classification problem, and commonly used methods can be divided into two categories: keyword retrieval and machine learning methods. Early researchers matched news texts with field-related keyword sets, such as algorithms such as KMP and Sunday. Currently, machine learning algorithms are an effective solution to filter event-related news. Researchers use statistical methods to make assumptions about data distribution to infer event-related news categories, such as SVM and decision trees. Some researchers also use deep learning algorithms to filter news, and use deep networks to extract hidden features of text and use them for classification. Due to the complex and changeable event-related news scenarios, it is difficult to construct a complete set of keywords, so keyword retrieval cannot be used to filter event-related news. It is difficult to cover all case situations and scenarios, and a large number of unlabeled event-related news is hidden in historical news. This lack of training data will make text filtering based on machine learning difficult. method is difficult to achieve the desired effect. Therefore, how to achieve better filtering performance with only a small number of event-related news samples is the focus of the invention.

The present invention mainly considers using topic information to enhance PU learning to classify event-related news. Therefore, on the basis of the PU learning methods proposed by Yu et al., Liu et al., Ren et al., Li et al. A method for classifying relevant news texts.

Contents of the invention

The present invention provides a learning method for event-related news filtering based on fused topic information enhanced PU learning to make full use of hidden topic information in news to improve the accuracy of event-related news filtering. At the same time, it achieves better results in event-related news filtering tasks than other baseline methods.

The technical solution of the present invention is: a learning method for filtering event-related news based on fusion topic information enhanced PU learning, and the specific steps of the method are as follows:

Step1, train the classifier, and add the unsupervised topic model VAE to enhance;

Step2. Predict the unlabeled data through the trained classifier model, and then sort the prediction results of the unlabeled news from high to low in probability;

Step3. After the initial training and prediction process is completed, perform PU learning iterations, that is, retrain the classifier on the newly obtained training set and repeat the entire prediction and training process;

Step4. Put all the samples into the classifier for training to obtain the event-related news classification model required in this article, and then filter out the required event-related news more accurately.

As a preferred solution of the present invention, the specific steps of the step Step1 are:

Step1.1. Use the improved version of the I-DNF algorithm to extract non-event-related news data, and obtain counterexamples of the same scale as the initial event-related news.

Step1.2, using variational autoencoder (VAE) as the topic model, the purpose is to extract potential features from the word vector space of the document, and the present invention is understood as topic features. The present invention refers to the work of predecessors and the principle of VAE, realizes this VAE structure and uses the entire event-related news data set for unsupervised pre-training. to train the initial classifier.

Step1.3, using the network structure of Embedding and bidirectional long short-term memory network (BiLSTM) as a classifier.

As a preferred solution of the present invention, the specific steps of the step Step1.1 are:

Step1.1.1. The frequency of a text feature in the positive set is greater than 90%, while the frequency of its appearance in the unlabeled set is only 10%, and such a feature is regarded as a feature of the positive example;

Step1.1.2. Establish a positive example feature set by using the different frequencies of features in the positive example set and the unlabeled set;

Step1.1.3. If the sample document in the unlabeled set U does not contain any features in the positive feature set, it is extracted from the unlabeled set U and marked as a negative example.

As a preferred solution of the present invention, said step Step1.2 includes:

Step1.2.1, Variational Autoencoder (VAE) architecture is an encoder-decoder architecture. In the encoder, the input is compressed into a latent distribution, and the decoder reconstructs the input signal by sampling according to the distribution in the data latent space;

Step1.2.2. Usually, the VAE model assumes that the posterior probability of the potential distribution of the input data approximately satisfies the Gaussian distribution, and then reconstructs it through the decoding network;

Step 1.2.3, the present invention implements the decoding network Decode using a fully connected network (MLP).

As a preferred solution of the present invention, the specific steps of the step Step1.3 are:

Step1.3.1, first use the Embedding network layer to embedding the input text to get the word embedding vector. In addition, the input text is passed through the VAE topic model to obtain the topic vector of the news text and obtain two kinds of encoding information;

Step1.3.2. Use the news topic vector to guide the word embedding vector; the new matrix formed is the news encoding vector integrated into the topic vector;

Step1.3.3. The news encoding vector after integrating the topic information is used to model its context relationship through the bidirectional long-term short-term memory network layer (BiLSTM), and obtain the news semantic representation vector.

As a preferred solution of the present invention, the specific steps of the step Step2 are:

Step2.1. Use the remaining unlabeled data samples in the dataset to predict the probability of categories through the classifier and topic model. The predicted result is the probability value that the news is news related to the event.

Step2.2, and then sort the prediction results of unmarked news from high to low in probability, and each prediction will obtain data with high probability according to a certain iteration step as reliable event-related news samples and low-probability data The data are used as reliable negative samples, and these samples are removed from unlabeled samples and added to the training data for subsequent iterative training process.

As a preferred solution of the present invention, the specific steps of the step Step3 are:

Step3.1 After completing the initial training and prediction process, retrain the classifier on the newly obtained training set and repeat the entire prediction and training process.

Step3.2. After each iteration is completed, the number of unlabeled data will decrease and the number of training sets will increase. When the unlabeled data is completely predicted as a reliable sample, the entire iterative process is completed.

The beneficial effects of the present invention are:

The invention applies the PU learning method to the task of filtering event-related news, effectively solving the problem of filtering event-related news in the case of a small amount of manual labeling.

The invention adopts an unsupervised pre-training method to extract the topic information of event-related news data, and uses it to enhance the training process of PU learning, which improves the accuracy rate compared with ordinary PU learning.

An event-related news data set was constructed and experimented with the method of the present invention. The experimental results show that the method proposed by the present invention achieves better results in the experiment than the PU learning method without topic enhancement.

Description of drawings

Fig. 1 is a general model diagram among the present invention;

Fig. 2 is the PU learning training process figure among the present invention;

Fig. 3 is the experimental result figure on the verification set in the present invention;

Fig. 4 is the experimental result figure on the unmarked data set among the present invention;

Fig. 5 is the different scale initial data contrast experiment result figure among the present invention;

Fig. 6 is a graph showing the experimental results of the iterative stride comparison in the present invention.

Detailed ways

Embodiment 1: as shown in Figure 1-5, based on the learning method of the event-related news filtering of PU learning based on fusion topic information, the specific steps of the learning method of event-related news filtering based on fusion topic information strengthening PU learning are as follows:

Step1, train the classifier, and add the unsupervised topic model VAE to enhance it.

Step2. Predict the unlabeled data through the trained classifier model, and then sort the prediction results of the unlabeled news from high to low.

Step3. After the initial training and prediction process is completed, the iteration of PU learning is performed, that is, the classifier is retrained on the newly obtained training set and the entire prediction and training process is repeated.

Step4. Put all the samples into the classifier for training to obtain the event-related news classification model required in this article, and then filter out the required event-related news more accurately.

The concrete steps of described step Step1 are:

Step1.1. Use the improved version of the I-DNF algorithm to extract non-event-related news data, and obtain counterexamples of the same scale as the initial event-related news.

Step1.2, using variational autoencoder (VAE) as the topic model, the purpose is to extract potential features from the word vector space of the document, and the present invention is understood as topic features. The present invention refers to the work of predecessors and the principle of VAE, realizes this VAE structure and uses the entire event-related news data set for unsupervised pre-training. to train the initial classifier.

Step1.3, using the network structure of Embedding and bidirectional long short-term memory network (BiLSTM) as a classifier.

Embodiment 2: As shown in Figures 1-5, the learning method of event-related news filtering based on fusion topic information enhanced PU learning, this embodiment is the same as Embodiment 1, wherein:

As a preferred solution of the present invention, the specific steps of the step Step1.1 are:

Step1.1.1. The frequency of a text feature in the positive set is greater than 90%, but its frequency in the unlabeled set is only 10%, and such a feature is regarded as a feature of the positive example.

Step1.1.2. Establish a positive example feature set based on the frequency of features appearing in the positive example set and the unmarked set.

Step1.1.3. If the sample document in the unlabeled set U does not contain any features in the positive feature set, it is extracted from the unlabeled set U and marked as a negative example.

As a preferred solution of the present invention, said step Step1.2 includes:

Step1.2.1, Variational Autoencoder (VAE) architecture is an encoder-decoder architecture. In the encoder, the input is compressed into a latent distribution Z, while the decoder reconstructs the input signal D by sampling from the distribution of Z in the latent space of the data.

Where Z represents the potential distribution, at this time, P(D|Z) describes the probability that D is generated by Z.

Step1.2.2, usually, the VAE model assumes that the posterior probability of the potential distribution Z of the input data D approximately satisfies the Gaussian distribution, that is

logP(Z|d ⁽ⁱ⁾ )＝logN(z; μ ⁽ⁱ⁾ ,δ ²⁽ⁱ⁾ I) (2)

Where d ⁽ⁱ⁾ represents a certain real sample in D, and each μ and δ ² are generated by d ⁽ⁱ⁾ through the neural network. Through the obtained μ ⁽ⁱ⁾ and δ ²⁽ⁱ⁾ , the distribution P(Z ⁽ⁱ⁾ |d ⁽ⁱ⁾ ) corresponding to each d ⁽ⁱ⁾ can be obtained, and then through the decoding network Refactor out />

Step1.2.3, the present invention realizes the generation of μ and δ ² and the realization of the decoding network Decode using a fully connected network (MLP).

Among them, m represents the preset number of potential topics. After the above calculations, the potential topic distribution of event-related news required by the present invention can be expressed as In order to make the reconstructed data as close as possible to the original data, the final optimization goal of VAE is to maximize the generation probability P(d ^{(i )} ) of d (i) while using the KL divergence to make the posterior obtained from the data The probability P(Z ⁽ⁱ⁾ |d ⁽ⁱ⁾ ) is as close as possible to its theoretical variational probability, that is, N(0,I), so the final expression of the optimization objective is as follows.

As a preferred solution of the present invention, the specific steps of the step Step1.3 are:

Step1.3.1, first use the Embedding network layer to embedding the input text to get the word embedding vector Among them, n represents the length of the news text, and v is the word vector dimension. In addition, the input text is passed through the VAE topic model to obtain the topic vector of the news text /> Where m is the number of preset themes. Two kinds of encoded information are obtained.

Step1.3.2. Use news topic vectors To guide the word embedding vector X. Since the topic vector obtained by the topic model involved in the case is a vector with a shape of 1*m, this paper copies n copies of it and stitches them into the word embedding vector X respectively, and the new matrix X′ formed is the news encoding vector integrated into the topic vector.

Step1.3.3. The news encoding vector after integrating the topic information is used to model its context relationship through the bidirectional long-term short-term memory network layer (BiLSTM), and obtain the news semantic representation vector. The specific formula is as follows.

Where H is the sentence vector encoded by BiLSTM, q is the hidden layer dimension of BiLSTM, and y represents the final probability output.

As a preferred solution of the present invention, the specific steps of the step Step2 are:

Step2.1. Use the remaining unlabeled data samples in the dataset to predict the probability of categories through the classifier and topic model. The predicted result is the probability value that the news is news related to the event.

Step2.2, and then sort the prediction results of unmarked news from high to low in probability, and each prediction will obtain data with high probability according to a certain iteration step as reliable event-related news samples and low-probability data The data are used as reliable negative samples, and these samples are removed from unlabeled samples and added to the training data for subsequent iterative training process.

As a preferred solution of the present invention, the specific steps of the step Step3 are:

Step3.1 After completing the initial training and prediction process, retrain the classifier on the newly obtained training set and repeat the entire prediction and training process.

Step3.2. After each iteration is completed, the number of unlabeled data will decrease and the number of training sets will increase. When the unlabeled data is completely predicted as a reliable sample, the entire iterative process is completed.

The present invention constructs an event-related news data set for experimentation, and combines the methods in this paper to do three types of experiments, one is to conduct comparative experiments with the performance of the PU classification algorithm without themes, and analyze the performance of the two in iterative training In addition, the comparative experiments of the initial test data sets of different scales were carried out, and finally the iterative stride comparison experiment was carried out, which verified the effectiveness of the method of the present invention compared with the unsubjected PU classification algorithm in the case of different strides . The experimental results verify the effectiveness of our method for the task of event-related news correlation analysis, and also illustrate the effect of using topic information to enhance the iterative process of PU learning to improve model performance.

The selection of experimental parameters directly affects the final experimental results. Since the length of the news text in the data set is about 100 to 250 characters, in order to facilitate the experimental verification effect, the present invention manually labels all the data, including 10,000 event-related news and 20,000 non-event-related news. The present invention sets the maximum length of the text to 200 characters. The Adam algorithm is used as the optimizer; the learning rate is set to 0.001; the dropout of the single-layer Bi-LSTM is set to 0.2; the batch size is set to 128; the number of training rounds is set to 20; The ratio of the number of positive and negative samples extracted each time. The evaluation indicators in this paper mainly use accuracy rate (Acc.), precision rate (P), recall (R) and F1 value.

The present invention compares the inventive method with the traditional PU learning method through these three types of experiments, and finds that the inventive method is superior to the traditional PU method in each iteration when the initial data size and iteration stride are fixed. When the initial data size is small or the iteration step is large, the performance of the method in this paper will be improved more and more stable.

In the present invention, the comparative experiment carried out with the performance of the PU classification algorithm without subject is mainly to verify the effectiveness of the method of the present invention on the event-related news filtering problem with only a small amount of event-related news samples, and to verify that topic information is important for PU Learning the reinforcement effect of an iterative process. Two sets of experiments were set up: one set was to use the reserved verification set to evaluate the generalization performance of the classifier trained by the iterative process, and the experimental results are shown in Figure 3; the other set was to evaluate the classifier trained by each iteration The prediction results on the remaining unlabeled samples and the experimental results are shown in Figure 4. Through the analysis of Figure 3, it can be known that the upper limit of the F1 value of the data set and classification model used in the present invention can reach 83.4% in the case of "supervised", while the F1 value in the case of "PU learning" is only 73.9%. The difference between the two is 13.7%, while the F1 value of the method of the present invention can reach 75.7% under the same experimental setting, which is 1.8% higher than that of "PU learning". Effectiveness on the event-related news filtering problem of news samples and the enhancement of topic models for PU learning.

From the analysis of Figure 4, it can be seen that the performance of the method of the present invention when predicting unlabeled data is ahead of the traditional PU learning scheme, and as the number of iterations increases, the gap between the two becomes larger. It also shows that the method of the present invention is effective for improving the PU learning method.

In the comparison experiment of initial data of different scales, the performance of the method of the present invention on the reserved verification set is improved compared with the PU learning method without themes. The experimental results are shown in Figure 5 below. Analyzing Figure 5, we can see that when there is only 500 initial labeled data, the traditional PU learning method has almost failed. This phenomenon is because PU learning depends on the size of the initial labeled data. When the initial data scale is too small, the training The accuracy of the resulting classifier is too low, and the error in the reliable positive and negative samples obtained in the subsequent prediction process is too large. As the iterative process progresses, this error will be accumulated and amplified and eventually lead to PU learning failure. As the size of the initial data increases, the error of each iteration is getting smaller and smaller, and the final training results are getting better and better. This is a common phenomenon in PU learning. The method in this paper also follows this common phenomenon. The difference is that our method shows better adaptability when the initial dataset is small. When the initial data size is only 750, the F1 value gap between the effect of this method and the traditional PU learning reaches 9.4%. With the increase of the initial data size, the gap between the two gradually narrows. This phenomenon is because the initial data When the scale is small, the unsupervised topic model will bring more information to a small amount of training data involved in the case, which will not reduce the performance of the initial classifier too much, and finally alleviate this "error accumulation" The phenomenon.

In the iterative stride comparison experiment, the performance of the method of the present invention on the reserved verification set is improved compared with the PU learning method without topics. The experimental results are shown in Figure 6 below. Analyzing Figure 6, it can be seen that when the iteration stride is 300 and 500, the performance of the present invention and the traditional PU learning can maintain a good level, and when the iterative stride is further expanded, the performance of the traditional PU learning begins to appear larger. decline, while the method of the present invention still remains at a better level. But when the iteration stride reaches 1500, both the present invention and the traditional PU learning are ineffective. This is because when the initial data size is only 1000, the accuracy of the classification model trained by PU learning is limited. Even if the subject information is added for enhancement, The required accuracy cannot be achieved either.

It can thus be proved that the learning method of the present invention can better filter event-related news, effectively solve the problem of lack of training data in the event-related news filtering task, and improve the accuracy of event-related news filtering results.

The specific implementation of the present invention has been described in detail above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned implementation, and can also be made without departing from the gist of the present invention within the scope of knowledge possessed by those of ordinary skill in the art. Variations.

Claims (3)

1. The learning method based on the event-related news filtering of subject information enhanced PU learning is characterized in that: the specific steps of the method are as follows: Step1, train the classifier, and add an unsupervised topic model to enhance; Step2. Predict the unlabeled data through the trained classifier model, and then sort the prediction results of the unlabeled news from high to low in probability; The concrete steps of described step Step2 are: Step2.1. Use the remaining unlabeled data samples in the data set to predict the probability of categories through the classifier and topic model; the prediction result is the probability value that the news belongs to the news related to the event; Step2.2. The prediction results of unmarked news will be sorted from high to low in probability. Each prediction will obtain data with high probability according to a certain iteration step as reliable event-related news samples and data with low probability. As reliable negative samples, these samples are removed from unlabeled samples and added to the training data for subsequent iterative training process; Step3. After the initial training and prediction process is completed, perform PU learning iterations, that is, retrain the classifier on the newly obtained training set and repeat the entire prediction and training process; Step4. Put all the samples into the classifier for training to obtain the required event-related news classification model, and filter out the required event-related news based on the event-related news classification model; Described step Step1 comprises: Use the I-DNF algorithm to extract non-event-related news data, obtain counterexamples of the same scale as the initial event-related news, to train the initial classifier, and add unsupervised topic model VAE for enhancement; Among them, the network structure of Embedding and bidirectional long short-term memory network BiLSTM is used as the classifier; First use the Embedding network layer to embedding the input text to get the word embedding vector Where n represents the length of the news text, and v is the word vector dimension; in addition, the input text is passed through the VAE topic model to obtain the topic vector of the news text/> Among them, m is the number of preset topics, and two kinds of encoding information are obtained; Topic vectors using news text To guide the word embedding vector X; since the topic vector obtained by the topic model is a vector with a shape of 1*m, copy n copies of it, and splicing them into the word embedding vector X respectively, the new matrix X' formed is to integrate the topic Vector of news encoding vectors: The news encoding vector after integrating topic information is used to model its context relationship through the bidirectional long-short-term memory network layer BiLSTM, and obtain the news semantic representation vector. The specific formula is as follows: Where H is the sentence vector encoded by BiLSTM, q is the hidden layer dimension of BiLSTM, and y represents the final probability output. 2. The learning method for filtering event-related news based on incorporating topical information to enhance PU learning according to claim 1, wherein the specific steps of obtaining a negative example of the same scale as the initial event-related news are as follows: Step1.1.1. The frequency of a text feature in the positive set is greater than 90%, while the frequency of its appearance in the unlabeled set is only 10%, and such a feature is regarded as a feature of the positive example; Step1.1.2. Establish a positive example feature set by using the different frequencies of features in the positive example set and the unlabeled set; Step1.1.3. If the sample document in the unlabeled set U does not contain any features in the positive feature set, it is extracted from the unlabeled set U and marked as a negative example. 3. the learning method based on the event-related news filtering of subject information enhanced PU learning according to claim 1, characterized in that: the specific steps of the step Step3 are: Step3.1 After completing the initial training and prediction process, retrain the classifier on the newly obtained training set and repeat the entire prediction and training process; Step3.2. After each iteration is completed, the number of unlabeled data will decrease and the number of training sets will increase. When the unlabeled data is completely predicted as a reliable sample, the iterative process ends.