CN110543564B - Domain Label Acquisition Method Based on Topic Model - Google Patents

Abstract

The present invention provides a method for obtaining field tags based on a topic model. On the basis of massive academic data, the inherent characteristics of academic data are analyzed, and academic word frequency features are introduced to construct an FLDA topic model. ‑phrases” extraction. Secondly, the domain system is introduced, and the extraction results of the topic model and the system labels are represented by vectors. After position weighting, the similarity is used to map the system, and finally the domain labels of scholars are obtained. Experiments show that compared with the traditional LDA model, the TFIDF algorithm based on statistics, and the TextRank algorithm based on network graphs, the FLDA model finally obtains better label words and higher accuracy, which shows that the label extraction method based on topic model has an important role in academic research. domain has good applicability.

Description

Domain Label Acquisition Method Based on Topic Model

technical field

The present invention relates to a method for acquiring domain labels based on a subject model, in particular to a method for acquiring domain labels of scholars, and belongs to the technical field of information processing.

Background technique

The vigorous development of the economy and society has prompted the continuous emergence of various scientific and technological projects, which require the participation of cutting-edge scholars from project approval, review to acceptance. In the past experience, the selection of scholars is often manually selected by a special person, and the research field of each scholar is artificially counted, and the scholar who matches the project field is selected. However, the methods of existing technologies often have the following disadvantages: there are a large number of projects at the same time that require the participation of frontier scholars, which virtually increases the workload of manual selection; manual selection is easily affected by human subjectivity and limitations, and in the The selection process is easily affected by factors such as one's own knowledge level, social relations, personal preferences and interests, and the judgment of the scholar's field is not comprehensive, which in turn affects the accuracy of the selection results.

Domain label acquisition in the prior art is mainly divided into traditional domain label acquisition and keyword-based domain label acquisition.

In the traditional field, one method is to use the profiles of scholars on various Internet platforms to extract, which is called network tags. Different from the extracted tags, network tags are usually summed up and added by oneself or others, and there is no uniform specification, and words are used randomly, which leads to complex and diverse tags and low usability. In addition, because the Internet content is relatively random and mostly has the characteristics of the author's writing, it is difficult to distinguish correct data information from useless information in the process of label extraction. When extracting, it is often necessary to design targeted tags for specific platforms and specific scholars. The extraction scheme virtually increases the workload.

Another method is to design a P2P mode scholar research field management system based on ontology technology, and use RDF technology to solve the problem of scholar field acquisition. However, because this method uses a specific template, the scalability of the method is insufficient.

Another method is to use J2EE technology to realize the scholar information management system, manually update the basic information of the scholar and the research field, and provide a consulting scholar recommendation module, and calculate the similarity between the user question text and the scholar's research field by using Pearson similarity To realize the scholar recommendation function, this method requires scholars to log in to the website to update and maintain information, which is only suitable for a small number of scholars in a single field, and cannot complete the functions of large-scale scholar discovery and machine field label extraction.

In terms of keyword-based field label extraction, there are many extraction methods. Commonly used keyword extraction bases include statistics, topics, and network diagrams. Keywords are a series of words with a high degree of generality to the article. Automatic keyword extraction is a technology to identify representative words or phrases in texts. Since this field was proposed, relevant researchers have successively proposed various Methods are generally divided into two categories: supervised and unsupervised. Among them, supervised methods require human-labeled corpus, which is suitable for small texts. However, with the increase of massive Internet data, the cost of manual labeling is increasing. In recent years, to gradually move to unsupervised methods. The core idea of the statistical method is to extract the statistical information of words in the text. This method does not need training data, and directly uses word frequency, position, etc. for judgment and screening. For example, one method is to use the weighting factor and word contribution to modify the weight of the TFIDF result to improve the keyword extraction effect in the subdivision field; another method is to use the N-gram language model and document weighting to realize automatic learning in the field of scholars. Recognition, the N-gram language model directly uses word order for calculation, and can classify documents by domain without performing operations such as word segmentation and feature extraction on documents, and then find out the domain labels of scholars. The topic model is to use the probability distribution to realize the keyword extraction. Currently, the most popular one is the LDA (Latent Dirichlet Distribution) model. Including the use of static LDA and improved fLDA to extract subject words to extract user historical interest in order to study the evolution of user behavior; in order to reduce the problem of data sparsity in microblog hot events, time series features and word frequency weighted features are introduced into the LDA algorithm. The topic keywords obtained by the method have high interpretability, and can better indicate the content displayed by the topic. The prior art further proposes a topic clustering method based on LDA. This method first clusters the keywords obtained by LDA, and uses the results to optimize the results of the topics obtained by LDA. This method can effectively improve the accuracy of the clustering results. Precision and recall. In terms of network graphs, the TextRank algorithm adapted from PageRank is the most famous. In order to solve the problem of poor academic keyword extraction results, prior knowledge is used to calculate the weight of candidate results in the academic field, and then combined with TextRank to comprehensively rank candidate keywords, and finally obtain academic keywords with high relevance; And use the word vector to construct the probability offset matrix, improve the textrank algorithm, and improve the performance of the algorithm.

Although unsupervised methods based on statistics and network diagrams do not need to manually label the corpus in advance, they heavily rely on the effect and scale of the corpus. For example, methods such as TFIDF have a simple structure, and the extracted keywords lack information such as distribution and semantics. Although methods such as TextRank can obtain the distribution information of keywords, the construction of network graphs requires a large amount of data to form edges, and the extracted keywords lack topic relevance. Despite the disadvantages mentioned above, unsupervised methods still have advantages in terms of workload.

The present invention intends to use the topic-based extraction method, regard the collection of academic documents as corpus to be extracted, and use the improved FLDA topic model to extract "topic-phrase" from the corpus to obtain a topic distribution matrix and realize automatic label acquisition.

Contents of the invention

In order to solve the problems existing in the prior art, the present invention proposes a field label acquisition method based on a topic model, analyzes the inherent characteristics of academic data on the basis of massive academic data, introduces academic word frequency features to construct an FLDA topic model, and utilizes the topic model to The academic documents of the same scholar are extracted by "topic-phrase". Secondly, the domain system is introduced, and the extraction results of the topic model and the system labels are represented by vectors. After position weighting, the similarity is used to map the system, and finally the scholar's domain labels are obtained.

In order to achieve the above-mentioned technical purpose, the present invention adopts the following technical solutions.

A topic model-based domain label acquisition method, see Figure 1, includes the following steps:

S1, data preprocessing

Get the initial data set;

S2, keyword extraction

Use FLDA to extract "topic-phrase", assign weights to phrases according to their positions in the text, and use word2vec to represent them as vectors;

S3, domain system mapping

Map the "topic-phrase" to the system to realize the unified management of scholars;

S4, comprehensive sorting

The vector representation results and the weight assignment results are weighted and sorted, and the tag words that best represent scholars are obtained through the threshold.

According to the aforementioned topic model-based domain label acquisition method, specifically, S1, data preprocessing, including S11, data deduplication processing, and S12, word segmentation.

Specifically, in order to eliminate the impact of repeated data caused by crawling from multiple data sources on the calculation results, in S1, the data preprocessing stage, S11, data deduplication processing, is required to obtain the scholar's literature collection.

The present invention uses word co-occurrence, author contribution and keyword coincidence rate to build cleaning model:

For two texts to be compared, first determine whether their DOIs are the same, and directly filter for the same DOI;

For DOIs that are not the same or do not exist, first use word co-occurrence to judge the title, if the co-occurrence is greater than 80%, continue to judge the number of co-occurrences of authors and keywords, if the number of co-occurrences of authors is greater than 1 and keywords If the co-occurrence rate is greater than 0.5, the result is determined as duplicate and cleared.

The co-occurrence formula is as follows:

Among them, A and B are the word sets of the two titles respectively, len(A) is the length of the word set of title A, len(B) is the length of the word set of title B, and len(A∩B) is the length of the two titles The length of the intersection of word sets, min{len(A), len(B)} is the minimum of the two lengths.

S12, word segmentation.

In the word segmentation stage, first extract the keyword data of the paper and add it as the user dictionary of the word segmentation tool, and at the same time use TextRank to extract key phrases before calculation, and add the key phrases to the user dictionary of the word segmentation tool.

In addition, the overall corpus data is sorted by word frequency, the high-frequency irrelevant words are manually screened, and the irrelevant words are added to the stop word list of the word segmentation tool.

According to the aforementioned method for acquiring domain labels based on a topic model, specifically, S2, in keyword extraction, "topic-phrase" extraction is performed through FLDA.

In the prior art, among topic-based keyword extraction models, the mainstream is the LDA topic model, which considers that a batch of documents contains multiple topics, and each topic can use a series of phrases to approximate the current topic. The formation of a document is to select a topic with a certain probability, and then select a phrase under the current topic with a certain probability, and repeat this process until a document is formed. The LDA topic extraction process is the reverse operation of the above process. The LDA topic model is usually widely used in the field of news, but in the field of scientific and technological literature, the effect of topic modeling will be affected by the special word frequency distribution of scientific and technological literature.

Through the statistical analysis of scholars’ academic documents, it is found that the frequency information of academic documents satisfies the power function distribution. Figure 2 shows the top 2000 high-frequency words in academic documents, where the abscissa indicates the corresponding serial number of high-frequency words sorted in descending order of frequency. The vertical axis is the frequency of high-frequency words.

According to statistics, the top 10% of frequent words account for 81.1% of all academic document word sets, which conforms to the Zipf distribution, and the research on word frequency found that the words that best represent the theme are often not extremely high-frequency words and extremely low-frequency words words, but the middle and high frequency words with higher frequency. If the LDA model is directly used to extract documents, some mid-frequency words will be missing. At the same time, feature words with high word frequency usually appear in pairs. Usually, the probability of high-frequency words being assigned to topics is relatively high, which leads to The degree of differentiation of each topic is not high. In S1, although stop words were filtered during the data preprocessing process, complete filtering could not be achieved.

Therefore, the present invention proposes a word-frequency weighted LDA topic model. First, the word frequency information in the document is counted, and the word frequency feature is introduced into the Gibbs sampling process to reduce the influence of high-frequency words, improve the influence of medium-frequency feature words, and construct FLDA model, so that the model does not place too much emphasis on high-frequency feature words. The FLDA model is described as follows:

和θ，获取参数/>

和θ的目的是构造一个收敛的马尔可夫链，进而从马尔可夫链中抽取合适的样本。The LDA model obtains sampling parameters through Gibbs sampling

and θ, get parameters />

The purpose of and θ is to construct a convergent Markov chain, and then draw suitable samples from the Markov chain.

The allocation process of LDA to phrases is the sampling of z _i . Among them, the posterior formula of _zi is as follows:

P(z _i ＝j|z _-i , w)→P(w _i |z _i ＝j, z _-i , w _-i )P(z _i ＝j|z _-i )

z _i =j is assigning topic j to the current word W _i , z _-i is the weight sum of words not assigned to z _i , and W _-i is the word at the non-current position.

相关，因此通过在/>

上积分，得到下式：It is known that P(w|z) is only related to

related, so pass in />

Integrating, we get the following formula:

为Gibbs抽样参数，/>

为当前主题j对应的Gibbs抽样参数，/>

为对参数进行积分，

Sampling parameters for Gibbs, />

Gibbs sampling parameters corresponding to the current topic j, />

To integrate over the parameters,

是“主题-短语”的多项式分布，遵循下式：

is a multinomial distribution of topic-phrases, following the formula:

同时，/>

是/>

的先验分布，因此对后验概率/>

进行积分，即可获得下式：in addition,

At the same time, />

yes />

The prior distribution of , so for the posterior probability />

Integrating, the following formula can be obtained:

是分配给主题j且与词w相同的词的权重和，/>

为分配给主题j且的所有词的权重和，β为Dirichlet分布的参数，v为词库的大小。in,

is the sum of the weights of words assigned to topic j that are the same as word w, />

is the sum of the weights of all words assigned to topic j, β is the parameter of Dirichlet distribution, and v is the size of the lexicon.

In the same way, it can be known that P(z) is only related to θ, so the following formula can be obtained by integrating on θ:

表示d_i中分配给主题i的词语权重和，T为主题数。

Indicates the weight sum of words assigned to topic i in d _i , and T is the number of topics.

结合得到下式：By the formula P(z _i =j|z _-i , w)→P(w _i |z _i =j, z _-i , w _-i )P(z _i =j|z _-i ),

Combined to get the following formula:

After the above calculation, the non-standard distribution of LDA is obtained, but the probability sum of all "topic-phrase" distributions needs to be removed, as shown in the following formula:

表示主题为j且词语与词语w_i相同的权重和，/>

表示文档d_i中主题为i的词语的权重和，/>

表示当前文档中拥有主题的词语的权重和，V表示词库大小，T表示主题个数，P(z_i＝j|z_-i，w_i)为经过重新计算的后验概率。Among them, the i-th word of w _i , z _i =j is to assign the current topic j to the current word w _i , z _-i is the sum of the word weights assigned to non-z _i ,

Indicates that the topic is j and the weight sum of the word is the same as the word w _i , />

Indicates the sum of weights of words with topic i in document d _i , />

Indicates the sum of weights of words with topics in the current document, V indicates the size of the thesaurus, T indicates the number of topics, and P(z _i =j|z _-i , w _i ) is the recalculated posterior probability.

The word frequency weighting formula of the model is as follows:

为当前词出现的个数，/>

为所有词的权重和。参见图3，由于Gibbs采样initialize时单词w分配给主题z的概率是随机的，因此，将计算得到的Fi替换掉Gibbs采样过程中初始化的随机值，并在此基础上循环计算至收敛并获得参数/>

和θ。Among them, n _i represents the word frequency of the current word, n _mid represents the word frequency of the selected medium frequency word, n _max represents the maximum value in the word frequency statistical result, n _min represents the minimum value in the word frequency statistical result, C _i represents the weight of the current word, and takes The value range is [1, 2]. In order to ensure that the total number of feature words remains unchanged after weighting, it is necessary to adjust the weight of each feature word, where F _i is the adjusted weight of feature words,

is the number of occurrences of the current word, />

is the weight sum of all words. Referring to Figure 3, since the probability of assigning word w to topic z is random when Gibbs sampling is initialized, the calculated Fi is replaced with the initialized random value during Gibbs sampling, and on this basis, the calculation is cyclic until convergence and obtained parameter />

and θ.

Word2vec method for word vector representation

Word2vec is trained on millions of dictionaries and hundreds of millions of training corpora through deep learning. The result of the training is the word vector model, which effectively expresses the semantic information of words in space. The vector training model refers to the shallow neural network CBOW or Skip-gram model, where the CBOW model is shown in Figure 4.

The characteristic of the CBOW model is to predict the current word according to the context. During training, first initialize an N-dimensional word vector for all words, and the model accumulates the context words in the input window period, and constructs a Huffmam tree according to the word frequency to obtain the Huffman path. Calculate the probability of leaf nodes according to the path, and then use the method of gradient descent to adjust the parameters of non-leaf nodes and the word vector of the context, and make the result converge to the real result after multiple iterations.

weight assignment

Because academic papers are usually divided into titles, abstracts, keywords, content and other information. According to past experience, the title often contains the central idea of the full text and is an important summary of the full text, so the present invention increases the final weight of the words in the title. The keyword part also has a certain representative ability to the main theme of the full text, while the abstract part is considered to be a brief summary of the full text content.

Preferably, the weight assignment in the present invention is to reset the weight of the title to 4, reset the weight of keywords to 3, and reset the weight of the abstract to 2.

Selection of FLDA model parameters

In the present invention, 20 is selected as the optimal number of topics.

S3, domain system mapping

Due to the differences in the phrases obtained by the topic model in the academic documents of various scholars, it is impossible to manage the scholars uniformly. Therefore, the academic field system is introduced to realize the unified measurement of scholars.

The field system is formulated with reference to the field system of the National Natural Science Foundation of China, which can cover the research scope of each field to the greatest extent.

The present invention maps the results of the topic model to the domain system, and the mapping formula is as follows:

F _{(A, B)} = sim(A, B)*C _A *L _A

Among them, A is the phrase obtained by the topic model, B is the system word, and the corresponding word vector is obtained by using the vector model. For unregistered words, the word vector is used to splice the word vector into a word vector. sim(A, B) is the final calculated cosine similarity. C _A is the probability assigned by the topic model, L _A is the position coefficient of the phrase in the document, and the value range is [2, 3, 4], F _{(A, B)} is the weighted similarity. C _B is the final score of system words.

S4, comprehensive sorting

The final score C _B is obtained through the mapping formula, and all system words corresponding to the current scholar are sorted according to the score C _B from high to low, and the top four system words with the highest scores are taken as the domain label words that best represent the scholar's research field.

The present invention adopts the above-mentioned technical scheme to obtain the following technical effects.

Compared with the traditional LDA model, the TFIDF algorithm based on statistics and the TextRank algorithm based on the network graph, the FLDA model of the present invention has a better tag word effect and a higher accuracy rate, which shows that the tag extraction method based on the topic model is in the academic field. domain has good applicability.

Description of drawings

Fig. 1 is a schematic framework diagram of the subject model-based domain label acquisition method of the present invention;

Figure 2 is a document-word frequency distribution diagram;

Figure 3 is a Gibbs sampling flow chart;

Figure 4 is a schematic diagram of the CBOW model;

Figure 5 is a diagram of perplexity-number of topics.

Detailed ways

In order to make the purpose, technical solution and beneficial effect of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments of the present invention and the accompanying drawings. Obviously, the described embodiments are part of the present invention Examples, not all examples. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.

Example:

A method for obtaining domain labels based on topic models, comprising the following steps:

S1, data preprocessing

To obtain the initial data set, specifically, the following methods are used:

S11, data deduplication processing

In this example, word co-occurrence, author contribution and keyword coincidence rate are used to construct a cleaning model:

For two texts to be compared, first determine whether their DOIs are the same, and directly filter for the same DOI;

For DOIs that are not the same or do not exist, first use word co-occurrence to judge the title, if the co-occurrence is greater than 80%, continue to judge the number of co-occurrences of authors and keywords, if the number of co-occurrences of authors is greater than 1 and the co-occurrence of keywords If the occurrence rate is greater than 0.5, the judgment result is repeated and cleared.

The co-occurrence formula is as follows:

Among them, A and B are the word sets of the two titles respectively, len(A) is the length of the word set of title A, len(B) is the length of the word set of title B, and len(A∩B) is the length of the two titles The length of the intersection of word sets, min{len(A), len(B)} is the minimum of the two lengths.

S12, participle

In the word segmentation stage, first extract the keyword data of the paper and add it as the user dictionary of the word segmentation tool, and at the same time use TextRank to extract key phrases before calculation, and add the key phrases to the user dictionary of the word segmentation tool.

In addition, the overall corpus data is sorted by word frequency, the high-frequency irrelevant words are manually screened, and the irrelevant words are added to the stop word list of the word segmentation tool.

S2, keyword extraction

The "topic-phrase" extraction is performed through FLDA, and the phrases are assigned weights according to the positions in the text, and word2vec is used to represent them as vectors.

"Topic-phrase" extraction through FLDA:

和θ，获取参数/>

和θ的目的是构造一个收敛的马尔可夫链，进而从马尔可夫链中抽取合适的样本。The LDA model obtains sampling parameters through Gibbs sampling

and θ, get parameters />

The purpose of and θ is to construct a convergent Markov chain, and then draw suitable samples from the Markov chain.

The allocation process of LDA to phrases is the sampling of z _i . Among them, the posterior formula of _zi is as follows:

P(z _i ＝j|z _-i , w)→P(w _i |z _i ＝j, z _-i , w _-i )P(z _i ＝j|z _-i )

z _i =j is assigning topic j to the current word W _i , z _-i is the weight sum of words not assigned to z _i , and W _-i is the word at the non-current position.

相关，因此通过在/>

上积分，得到下式：It is known that P(w|z) is only related to

related, so pass in />

Integrating, we get the following formula:

为Gibbs抽样参数，/>

为当前主题j对应的Gibbs抽样参数，/>

为对参数进行积分，

Sampling parameters for Gibbs, />

Gibbs sampling parameters corresponding to the current topic j, />

To integrate over the parameters,

是“主题-短语”的多项式分布，遵循下式：

is a multinomial distribution of topic-phrases, following the formula:

同时，/>

是/>

的先验分布，因此对后验概率/>

进行积分，即可获得下式：in addition,

At the same time, />

yes />

The prior distribution of , so for the posterior probability />

Integrating, the following formula can be obtained:

是分配给主题j且与词w相同的词的权重和，/>

为分配给主题j且的所有词的权重和，β为Dirichlet分布的参数，v为词库的大小。in,

is the sum of the weights of words assigned to topic j that are the same as word w, />

is the sum of the weights of all words assigned to topic j, β is the parameter of Dirichlet distribution, and v is the size of the lexicon.

In the same way, it can be known that P(z) is only related to θ, so the following formula can be obtained by integrating on θ:

表示d_i中分配给主题i的词语权重和，T为主题数。

Indicates the weight sum of words assigned to topic i in d _i , and T is the number of topics.

结合得到下式：By the formula P(z _i =j|z _-i , w)→P(w _i |z _i =j, z _-i , w _-i )P(z _i =j|z _-i ),

Combined to get the following formula:

After the above calculation, the non-standard distribution of LDA is obtained, but the probability sum of all "topic-phrase" distributions needs to be removed, as shown in the following formula:

表示主题为j且词语与词语w_i相同的权重和，/>

表示文档d_i中主题为i的词语的权重和，/>

表示当前文档中拥有主题的词语的权重和，V表示词库大小，T表示主题个数，P(z_i＝j|z_-i，w_i)为经过重新计算的后验概率。Among them, the i-th word of w _i , z _i =j is to assign the current topic j to the current word w _i , z _-i is the sum of the word weights assigned to non-z _i ,

Indicates that the topic is j and the weight sum of the word is the same as the word w _i , />

Indicates the sum of weights of words with topic i in document d _i , />

Indicates the sum of weights of words with topics in the current document, V indicates the size of the thesaurus, T indicates the number of topics, and P(z _i =j|z _-i , w _i ) is the recalculated posterior probability.

The word frequency weighting formula of the model is as follows:

为当前词出现的个数，/>

为所有词的权重和。参见图3，由于Gibbs采样initialize时单词w分配给主题z的概率是随机的，因此，将计算得到的Fi替换掉Gibbs采样过程中初始化的随机值，并在此基础上循环计算至收敛并获得参数/>

和θ。Among them, n _i represents the word frequency of the current word, n _mid represents the word frequency of the selected medium frequency word, n _max represents the maximum value in the word frequency statistical result, n _min represents the minimum value in the word frequency statistical result, C _i represents the weight of the current word, and takes The value range is [1, 2]. In order to ensure that the total number of feature words remains unchanged after weighting, it is necessary to adjust the weight of each feature word, where F _i is the adjusted weight of feature words,

is the number of occurrences of the current word, />

is the weight sum of all words. Referring to Figure 3, since the probability of assigning word w to topic z is random when Gibbs sampling is initialized, the calculated Fi is replaced with the initialized random value during Gibbs sampling, and on this basis, the calculation is cyclic until convergence and obtained parameter />

and θ.

Word2vec method for word vector representation

Word2vec is trained on millions of dictionaries and hundreds of millions of training corpora through deep learning. The result of the training is the word vector model, which effectively expresses the semantic information of words in space.

Specifically, the CBOW model is used for vector training. The characteristic of the CBOW model is to predict the current word according to the context. During training, first initialize an N-dimensional word vector for all words, and the model accumulates the context words in the input window period, and constructs a Huffmam tree according to the word frequency to obtain the Huffman path. Calculate the probability of leaf nodes according to the path, and then use the method of gradient descent to adjust the parameters of non-leaf nodes and the word vector of the context, and make the result converge to the real result after multiple iterations.

weight assignment

In this embodiment, the weight of the title is reset to 4, the weight of keywords is reset to 3, and the weight of the abstract is reset to 2.

As a preferred solution, 20 is selected as the optimal number of topics in this embodiment.

S3, domain system mapping

Map the "topic-phrase" to the system to realize the unified management of scholars;

In this embodiment, the results of the topic model are mapped to the domain system, and the mapping formula is as follows:

F _{(A, B)} = sim(A, B)*C _A *L _A

Among them, A is the phrase obtained by the topic model, B is the system word, and the corresponding word vector is obtained by using the vector model. For unregistered words, the word vector is used to splice the word vector into a word vector. sim(A, B) is the final calculated cosine similarity. C _A is the probability assigned by the topic model, L _A is the position coefficient of the phrase in the document, and the value range is [2, 3, 4], F _{(A, B)} is the weighted similarity. C _B is the final score of system words.

S4, comprehensive sorting

The vector representation results and weight assignment results are weighted and sorted to obtain the tag words that best represent scholars.

Specifically, the final score C _B is obtained through the mapping formula, and all system words corresponding to the current scholar are sorted from high to low scores, and the top four system words with the highest scores are taken as the domain label words that best represent the scholar's research field.

Experimental example:

In order to obtain experimental data as real as possible, the present invention uses web crawler technology to crawl paper data sources in CNKI and Wanfang databases, uses jieba for word segmentation processing of data, and uses word vector model published by Tencent AI Lab for vector representation. The experimental example will be introduced from four aspects: the introduction of evaluation criteria, data preprocessing, the selection of the number of topics in the FLDA model parameters, and the evaluation of the labeling algorithm based on the topic model.

evaluation standard:

Since the LDA topic model is an unsupervised model, there is no intuitive evaluation standard to measure the quality of the model. In this experimental example, we choose to use the "topic-phrase" matrix of the topic model for evaluation, and introduce perplexity as the evaluation standard of the model. It is generally believed that the lower the perplexity, the better the effect of the model. The calculation formula of perplexity is shown in the formula.

p(w)=p(z|d)*p(w|z).

当前语料中所有词的数量和，p(w)为词w出现在矩阵中的概率，p(z|d)表示学术文档d为主题z的概率，p(w|z)表示的是词w出现在主题z中的概率。困惑度衡量主题模型预测出的结果与原样本信息的符合度。Among them, Perplexity (D) represents the perplexity of the current model, d table is the academic document text, M represents the number of academic documents,

The sum of the number of all words in the current corpus, p(w) is the probability that word w appears in the matrix, p(z|d) represents the probability that academic document d is topic z, and p(w|z) represents the word w The probability of appearing in topic z. The perplexity measures the degree of conformity between the results predicted by the topic model and the original sample information.

When calculating the labeling accuracy, the F1 value is used to measure the accuracy of the scholar's labeling, a number of scholars are randomly selected, and the 4 most suitable labels are selected as the correct labels by manually referring to the field labels and combining with the understanding of the scholars. The first four labels and the correct labels obtained according to the algorithm sorting are evaluated and calculated using the above indicators, and finally the average F1 value is calculated. The calculation formula is as follows:

Among them, h _i represents the number of standard labels, m _i represents the number of labels obtained by the algorithm, and h _i ∩ m _i represents the correct number of labels obtained by the algorithm. N is the total number of samples.

data preprocessing

In order to eliminate the impact of repeated data on the calculation results caused by crawling from multiple data sources, the data needs to be deduplicated in the preprocessing stage, and the cleaning model is constructed using word co-occurrence, author contribution and keyword coincidence rate. For the two to be compared For texts with the same DOI, first judge whether their DOIs are the same, and directly filter for the same DOI. For those with different or non-existent DOIs, first use word co-occurrence to judge the title, and if the co-occurrence degree is greater than 80%, continue to judge the author co-occurrence Times and keyword co-occurrence times, if the co-occurrence times of the author is greater than 1 and the key co-occurrence rate is greater than 0.5, the result is judged to be duplicate and cleared.

In the word segmentation stage, a large amount of paper keyword data is first extracted and added as the user dictionary of the word segmentation tool. At the same time, TextRank is used to extract key phrases before calculation, and the key phrases are also added to the user dictionary of the word segmentation tool. In addition, the overall corpus data is sorted by word frequency, the high-frequency irrelevant words are manually screened, and the irrelevant words are added to the stop word list of the word segmentation tool.

Selection of FLDA model parameters

The selection of the number of topics in the topic model is an important factor affecting the results of topic clustering. If the number of topics is set too small, the clustering results of the model will be indistinguishable. If the number of topics is set too large, the current document will be Wrongly divided into other topics, so this section uses experiments to calculate the perplexity under different topic numbers, and determines the final topic number according to the perplexity. In the experiment, only the number of topics is changed when the other parameters remain unchanged, and the experimental results are shown in Figure 4.

The LDA curve is the perplexity curve of the LDA topic model, and the FLDA curve is the perplexity curve of the LDA topic model weighted by word frequency. The abscissa in the relationship diagram is the number of topics, and the ordinate is the perplexity. In the experiment, the same set of parameters was used to repeat the experiment three times, and the average value of the experimental results was taken.

It can be seen from the experimental results that with the increase of the number of topics, the perplexity of the three models shows a downward trend, and the decline slows down or even converges when the number of topics is about 20, so 20 is selected as the optimal number of topics.

Evaluation of Labeling Algorithms

Introduce a field system to achieve a unified measurement of scholars’ academic fields. The field system is formulated with reference to the field system of the National Natural Science Foundation of China, and appropriate modifications are made on this basis. An example of the field system is shown in Table 1.

Table 1 Domain System Example

Due to the lack of more authoritative data sets in the field of label extraction in the academic field, in order to verify the effectiveness of the algorithm, the experimental data used the academic paper data of 12 scholars, and used the TFIDF algorithm, TextRank algorithm, LDA algorithm and FLDA algorithm to obtain scholars The academic labels and compare them, the specific examples are shown in Table 2.

When evaluating the algorithm, human beings select appropriate words in the system based on information such as the homepage introduction of the scholar and the admission brochure as the standard answer for the scholar's domain label, and the label words obtained by the algorithm are called the current answer, and the current answer is compared with the standard answer. evaluate.

Table 2 Comparison of algorithm extraction label results

Table 3 Algorithm F1 value comparison

It can be seen from Table 2 that the labels obtained by the FLDA algorithm have a high coincidence with the standard answers. The algorithm effect is better than LDA and TFIDF algorithm.

The data in Table 3 is the F1 value calculated by referring to formulas (15)(16)(17), where 4-2 represents the use of the algorithm to obtain the 4 highest-scoring labels and calculates them with two standard answers; 4-3 represents the use of The algorithm obtains the 4 highest-scoring labels and calculates them with three standard answers; 4-4 is calculated with four standard answers. According to the analysis, in various prediction situations, the F1 value of FLDA is higher than that of traditional LDA algorithm, TFIDF algorithm based on statistics and TextRank algorithm based on network graph. This shows that by introducing the FLDA model weighted by multi-word frequency features, it can not only analyze the content of documents and their connections from the text, but also reduce the dimensionality of academic data, which is more suitable for processing academic documents of a certain order of magnitude, which is conducive to subsequent labeling Mapping and computing. To a certain extent, the model can reflect the research direction of scholars, so that users can have a comprehensive understanding of scholars more conveniently, saving users' time and energy. It also indirectly shows that the FLDA algorithm weighted by multi-word frequency features can better extract key information in academic texts than traditional algorithms.

The technical solutions provided by the present invention are not limited by the above-mentioned embodiments, and all technical solutions formed by utilizing the structures and methods of the present invention through transformation and substitution are within the protection scope of the present invention.

Claims (7)

1. A domain label acquisition method based on topic model, characterized in that: comprising the following steps, S1, data preprocessing Get the initial data set; S2, keyword extraction Use FLDA to extract "topic-phrase", assign weights to phrases according to their positions in the text, and use word2vec to represent them as vectors; S3, domain system mapping Map the "topic-phrase" to the system to realize the unified management of scholars; S4, comprehensive sorting The vector representation results and weight assignment results are weighted and sorted, and the tag words that best represent scholars are obtained through thresholds; Wherein, the method for extracting "topic-phrase" by FLDA is to obtain sampling parameters by Gibbs sampling

and θ, The posterior formula of z _i is as follows: P(z _i ＝j|z _-i ,w)→P(w _i |z _i ＝j,z _-i ,w _-i )P(z _i ＝j|z _-i ), Among them, z _i =j is to assign topic j to current word W _i , z _-i is the weight sum of words assigned to non- _zi , W _-i is a word in a non-current position, It is known that P(w|z) is only related to

related, so pass in />

Integrating, we get the following formula:

in,

Sampling parameters for Gibbs, />

Gibbs sampling parameters corresponding to the current topic j, />

To integrate over the parameters,

is a multinomial distribution of topic-phrases, following the formula:

in addition,

At the same time, />

yes />

The prior distribution of , so for the posterior probability />

Integrating, the following formula can be obtained:

in,

is the sum of the weights of words assigned to topic j that are the same as word w, />

is the weight sum of all words assigned to topic j, β is the parameter of Dirichlet distribution, v is the size of the vocabulary, In the same way, it can be known that P(z) is only related to θ, so the following formula can be obtained by integrating on θ:

Indicates the weight sum of words assigned to topic i in d _i , T is the number of topics, By the formula P(z _i =j|z _-i ,w)→P(w _i |z _i =j,z _-i ,w _-i )P(z _i =j|z _-i ),

Combined to get the following formula:

After the above calculation, the non-standard distribution of LDA is obtained, and then the probability sum of all "topic-phrase" distributions is removed, as shown in the following formula:

Among them, the i-th word of w _i , z _i =j is to assign the current topic j to the current word w _i , z _-i is the sum of the word weights assigned to non-z _i ,

Indicates that the topic is j and the weight sum of the word is the same as the word w _i , />

Indicates the sum of weights of words with topic i in document d _i , />

Indicates the weight sum of words with topics in the current document, V indicates the size of the thesaurus, T indicates the number of topics, P(z _i =j|z _-i ,w _i ) is the recalculated posterior probability; The word frequency weighting formula of the model is as follows:

Among them, n _i represents the word frequency of the current word, n _mid represents the word frequency of the selected medium frequency word, n _max represents the maximum value in the word frequency statistical result, n _min represents the minimum value in the word frequency statistical result, C _i represents the weight of the current word, and takes The value range is [1,2]. In order to ensure that the total number of feature words remains unchanged after weighting, it is necessary to adjust the weight of each feature word, where F _i is the adjusted weight of feature words,

is the number of occurrences of the current word, />

is the weight sum of all words; Replace the calculated Fi with the random value initialized in the Gibbs sampling process, and on this basis, calculate cyclically until convergence and obtain parameters

and θ. 2. the domain label acquisition method based on subject model according to claim 1, is characterized in that: S1, data preprocessing includes S11, data deduplication processing, and S12, word segmentation; Among them, S11, data deduplication processing, to obtain the literature collection of scholars; Build a cleaning model using word co-occurrence, author contribution and keyword coincidence rate: For two texts to be compared, first determine whether their DOIs are the same, and directly filter for the same DOI; For DOIs that are not the same or do not exist, first use word co-occurrence to judge the title. If the co-occurrence degree is greater than 80%, continue to judge the number of co-occurrences of authors and keywords. If the number of co-occurrences of authors is greater than 1 and the keywords If the co-occurrence rate is greater than 0.5, the result is determined to be duplicate and cleared; The co-occurrence formula is as follows:

Among them, A and B are the word sets of the two titles respectively, len(A) is the length of the word set of title A, len(B) is the length of the word set of title B, and len(A∩B) is the length of the two titles The length of the intersection of word sets, min{len(A), len(B)} is the minimum of the two lengths; Among them, S12, word segmentation, obtains the initial data set; First extract the keyword data of the paper and add it as the user dictionary of the word segmentation tool, and use TextRank to extract key phrases before calculation, and add the key phrases to the user dictionary of the word segmentation tool; Sort the overall corpus data according to word frequency, manually filter high-frequency irrelevant words, and add irrelevant words to the stop word list of the word segmentation tool. 3. The domain label acquisition method based on topic model according to claim 1, characterized in that: the method of vector representation is word2vec method. 4. The domain label acquisition method based on topic model according to claim 1, characterized in that: the weight of title is set to 4, the weight of keywords is set to 3, and the weight of abstract is set to 2. 5. The subject model-based domain label acquisition method according to any one of claims 1 to 4, characterized in that: the subject number of the FLDA model is 20. 6. The domain label acquisition method based on topic model according to claim 1, characterized in that: S3, the mapping formula of domain system mapping is:

F _(A,B) = sim(A,B)*C _A *L _A

Among them, A is the phrase obtained by the topic model, B is the system word, and the corresponding word vector is obtained by using the vector model. For unregistered words, the word vector is used to stitch the word vector, and sim(A,B) is the final calculated cosine similarity , C _A is the probability assigned by the topic model, L _A is the position coefficient of the phrase in the document, and the value range is [2, 3, 4], F _{(A, B)} is the weighted similarity, C _B is is the final score of system words. 7. The field label acquisition method based on topic model according to claim 1, characterized in that: S4, comprehensive sorting, all system words corresponding to the current scholar are sorted from high to low according to the score C _B , and the highest score is obtained before The several system words of the system are used as the field label words that can best represent the research field of scholars.

Images