CN108304488A - A method of utilizing the automatic study ontology of Topic Model - Google Patents

Abstract

The present invention provides a kind of methods of the automatic study ontology using Topic Model, this method supports automatic domain body structure, a kind of measure of Semantic Similarity between the calculating concept of information is invented, the Semantic Similarity between concept for calculating the generation of LDA models, the method for this automatic study ontology are divided into two steps：The first step is to carry out concept identification from text corpus or web corpus；Second step is to carry out the relationship between concept using Semantic Similarity defined herein measurement CP to establish.This method need not have auxiliary of the seed ontology as initial study ontology.The experimental results showed that the method proposed by the present invention for carrying out automated ontology structure using Topic Model is very effective.

Description

A Method of Automatically Learning Ontology Using Topic Model

technical field

The invention relates to a method for constructing an ontology, which utilizes a TopicModel as a basic concept unit, and can learn ontology without ontology seeds to achieve the purpose of constructing ontology.

Background technique

Ontology construction has been applied to various fields, such as artificial intelligence, information extraction, machine translation and other fields. However, manually constructing ontology is a very time-consuming and laborious work. With the continuous expansion and updating of concepts and domain information, building large-scale ontology requires more and more manpower, material resources and energy. Therefore, artificially constructing large-scale ontology such as webdirectories and Wordnet Seeds require more effort and effort. Therefore, there is a strong need to be able to automatically build ontology to keep up with the real demand of this field information surge, to reduce the cost of building and maintaining ontology. Therefore, recently, it is a very meaningful research to automatically construct ontology by means of computer data analysis and data mining, which has attracted many researchers to do a lot of in-depth research on it.

Automatic construction of ontology has become a new research field. Many methods have been proposed for automatic construction of ontology. At present, there are many real-time applications of ontology, which can help knowledge engineers combine automatic or semi-automatic machine learning technology to construct and expand ontology, greatly The cost of manually constructing and maintaining ontology is reduced. Most of the current ontology learning methods focus on expanding and updating the existing ontology seeds, and update and broaden the ontology seeds by extracting concepts or lexical units from the literature lexicon. There are also some methods for automatically learning ontology, but most of these methods are based on ontology construction in special knowledge domains, such as the SKOS model, but these methods have certain limitations.

There are many methods of learning ontology from text corpus, such as ontology construction methods based on lexico-syntactic, these methods mainly use natural language processing technology and existing lexicon resources to learn the is-a relationship between concepts, the so-called Hearst -parterns, but this type of method has a disadvantage that Hearst-parterns, a lexical pattern that needs to appear frequently, does not appear frequently, and at the same time, he can only deal with some very vague lexical semantic relationships. Common sense such as P.Cimicano and F.M.Suchanek use web search engines such as Wikipedia and Wordnet to extract more language patterns.

Statistical learning methods based on clustering and classification are also applied to ontology learning, and these methods usually use similarity measures and dissimilarity measures to establish concept relationships. The limitation of such methods is that ontology learning methods based on clustering and classification are difficult to implement. Ontology learning methods based on information extraction technology learn the hierarchical structure of ontology, which can only extract very generalized concepts like human beings, places, animals and their sub-concepts.

The Topic Model probabilistic model is a model that has been proven to be very effective in identifying concepts from scientific publications without prior knowledge. The Topic Model model has been widely used in the field of text mining. Ontology learning using Topic Model is a new research method. Elias Zavitsanos et al. proposed an automatic ontology learning method based on statistical methods. This method continuously reuses the concept set trained by the Topic Model model, and then uses the conditional independence to judge the connection between the identified concepts. However, this method Methods cannot make a conceptual connection between two hierarchies. Wang Wei et al. proposed two methods based on semantic Web learning ontology structure. This method uses the combination of information theory and Topic Model to show good recall and accuracy, but it needs to limit the nearest root node. The number of child concept nodes.

Contents of the invention

The purpose of the present invention is to provide a method for automatically learning ontology, which can accurately determine the relationship between concepts, and can determine the depth and learning time of ontology in the process of learning ontology without providing prior knowledge end point.

In order to achieve the above object, the technical solution of the present invention provides a method for automatically learning ontology utilizing Topic Model, which is characterized in that it includes the following steps:

The first step is to use the LDA model to extract concepts from a given document corpus, generate a concept set from the extracted concepts, and then carry out conceptual hierarchical subdivision to generate a hierarchical structure G of ontology construction, G={T, E}, In the formula, T = {t1, t2, ..., tm} is a concept set, defined as the upper concept set; T' = {t1', t2', ..., tm'} is a sub-concept set, defined as The concept set of the next layer of the upper concept set T, the concept set T and the sub-concept set T' are two consecutive layers; E is a set of edges, and each eij∈E represents the i-th concept ti and sub-concept in the concept set T. The jth concept tj' in the concept set T' is connected by an edge;

The second step is to use the CosTMI similarity measurement method to identify the semantic similarity between two consecutive layers in the hierarchical structure G, wherein, in the context of the p-th concept tp in the upper-level concept set T and the concept tp, the lower-level concept set The semantic similarity between the sth concept ts' and the rth concept tr' in T′ is CosTMI(ts′, tr′; tp)

In the formula, tp contains the lexical sequence {wp1, wp2, ..., wpn}; ts' contains the lexical sequence {ws'1, ws'2, ..., ws'n}; tr' contains the lexical sequence {wr' 1, wr'2,...,wr'n}; PMI() is the point mutual information of two words, and the point mutual information of two words w and w' is PMI(w, w'), then:

In the formula, P(w,w')=P(w)P(w'|w);

In the formula, z is the topic, P(z=j) is the probability when the topic is j, P(w|z=j) is the conditional probability of the vocabulary w when the topic is j, and k is the number of concepts;

In the formula, P(w'|z=j) is the conditional probability of w' when the topic is j, and P(z=j|w) is the conditional probability of topic j when the vocabulary is w;

If CosTMI(ts', tr'; tp) is greater than a certain threshold thc, a relationship is established between tp and ts', tr';

The third step is to calculate the standard similarity measure L(ts', tr'; tp), In the formula, P(ts′|tp) is (is the occurrence probability of ts’ in the tp context vocabulary environment), P(tr′|tp) is (is the occurrence probability of ts’ in the tp context vocabulary environment) ;

When the standard similarity measure L(ts', tr'; tp) is used to define the relationship between ontology concepts, each concept learned through the Topic Model corresponds to an ontology concept, and each concept ts' or tr' is in The conditional probability in the context of tp is used to calculate the semantic similarity between the concepts of the same layer. The smaller the value, the higher the semantic similarity of the value;

The fourth step is to determine the hierarchy of the ontology

Assume that the TopicModel is used to learn three conceptual levels Th, Tm, and Tl. Th is the highest level, Tm is the middle level, and Tl is the lowest level. The entropy of these three variables is recorded as H(Th), H(Tm), H( Tl), H(Tl|Tm) is the conditional entropy in the information field, then the concept set information gain Δ(I(Th, Tm, Tl)) of two consecutive layers is defined as:

Δ(I(Th, Tm, Tl))=H(Th)-H(Tl|Tm)

When Δ(I(Th, Tm, Tl)) is less than the specified threshold ω, stop using the LDA model to learn the concept set.

Preferably, in the first step, the following rules are followed when subdividing the concept hierarchy to generate the hierarchical structure G of ontology construction:

Rule 1: If ti∈T, tj'∈T', NT<NT', the conclusion is: the sub-concept set T' is greater than the concept set T, where NT and NT' are the layers of the concept set T and the sub-concept set T' respectively high-level;

Rule 2: If ti∈T, tj'∈T', It is very likely that there is a subordinate-subordinate relationship between ti and tj', where, is the empty set.

The invention proposes a new method that can automatically learn ontology from a given text corpus. We use the concepts generated by a widely used probability model, namely the TopicModel model, as the conceptual units required to construct ontology. With these concepts, we also need a method to measure the similarity of these concepts to define the adjacent up and down in the ontology structure. The connection between the two layers of concepts is to build the structure of the topic and establish edges between the concepts to form a hierarchical structure of the ontology. Ensure that there is a connection between the learned concepts, and the connection between the concepts is the most compact and reasonable. To this end, we define two similarity measures, and we propose a new criterion for judging the depth of the ontology hierarchy, that is, a new method for judging the end of the loop when learning an ontology.

The present invention generates concepts by repeatedly using the LDA model, that is, the Topic Model model, and defines a measurement method that can accurately measure the semantic similarity between the concepts to construct the concepts of the ontology and the structural levels between the concepts. The validity and practicability of the ontology construction method proposed in the present invention are verified by using the real GENIA corpus and ontology GENIA ontology.

Description of drawings

Figure 1 is a graph showing the relationship between the accuracy rate and the topic dimension;

Figure 2 is a graph of the change in accuracy rate with the increase of ontology depth under the CP measure;

Fig. 3 is a graph showing the variation of the accuracy rate with the increase of the body depth under the L1 metric.

Detailed ways

Below in conjunction with specific embodiment, further illustrate the present invention. It should be understood that these examples are only used to illustrate the present invention and are not intended to limit the scope of the present invention. In addition, it should be understood that after reading the teachings of the present invention, those skilled in the art can make various changes or modifications to the present invention, and these equivalent forms also fall within the scope defined by the appended claims of the present application.

A method for automatically learning ontology using Topic Model provided by the present invention roughly includes the following steps:

The first step is to use the LDA model to extract concepts from the given literature corpus, and to repeatedly use the model to extract the concept set needed to build the ontology.

In the second step, the CP similarity measure is designed to identify the similarity between concepts in the hierarchy, that is, the potential connection between concepts in adjacent levels; the L1 standard is formulated to determine the number of levels in the ontology construction.

The third step is to verify the validity and effectiveness of this ontology construction method through experiments.

The above steps involve the following technological innovations:

1) Ontology construction process

Figure 1 illustrates the process of ontology construction. Construct a hierarchical structure G, G = {T, E}, where T = {t1, t2, ..., tm} is a concept set, called the concept layer, which is output by the LDA model and can be defined as the upper-level concept set. T'={t1', t2', ..., tm'} is a sub-concept set, which is defined as the next-level concept set of the upper-level concept set T. E is a set of edges, each eij∈E means that the i-th concept ti in the concept set T and the j-th concept tj' in the sub-concept set T′ are connected by an edge G={T, E}, in the figure T= {t1,t2,...,tm} is a collection of concepts.

In order to build the connection between the upper and lower levels of concepts, it is necessary to determine the concept level to which these concept nodes belong, which ones belong to the higher level concept set, and which ones belong to the lower level concept set, and establish the connection between the two level concept sets will be more complicated. The boundaries between concepts using the LDA model are not particularly clear. It is necessary to use certain measurement methods to layer these concepts, and to establish the relationship between layers. Some concepts may have several fathers, and some concepts may not. Children, the more concept layers are generated, the closer the relationship between the concept layers, so the number of layered concepts cannot be increased without limit, and it is necessary to artificially set the number of layers for an ontology construction.

2) Relevant rules

Before proposing a specific implementation of automatic ontology learning methods, we first define two basic rules. In general, the LDA model is continuously reused to generate a concept set, which is used to construct the concepts required for the hierarchical structure. The present invention defines some rules, which are used to limit the concepts generated by the model, and are used when constructing the hierarchy ontology.

According to intuition, the higher the concept is, the more abstract it is, and vice versa, the more concrete it is; the higher the level, the fewer concepts, and vice versa. Then, based on these common senses, the following rules are defined:

Rule 1: If ti∈T, tj'∈T', NT<NT', the conclusion is: the sub-concept set T' is greater than the concept set T, where NT and NT' are the layers of the concept set T and the sub-concept set T' respectively high-level.

When using the LDA model to repeatedly learn to generate concept sets, it must first be determined that NT<NT'. Therefore, this rule is very important for the method of constructing ontology.

Each concept of each layer learned by LDA through the document corpus is a vocabulary that appears frequently in the literature. The concept set that appears frequently in the high-level is very likely to appear frequently in the low-level concept set. Therefore, in the process of building the ontology It is unreasonable that these same words in So define the following rules:

Rule 2: If ti∈T, tj'∈T', It is very likely that there is a subordinate-subordinate relationship between ti and tj', where, is the empty set.

This rule can help us define the similarity measure to be introduced below in this patent between concepts.

3) Similarity measure

The present invention utilizes the method of similarity measurement to construct the hierarchical structure of ontology, that is to say, the connection between concepts is established through the similarity between concepts. Only when the two concepts in the two levels of concept sets reach a certain similarity value can a connection be established, otherwise they are considered to be unconnected. In order to calculate the semantic similarity between two concepts, the LDA model is used to generate the concept matrix generated by the concept set, and each matrix input is the possibility of the concept appearing in the ontology.

Usually, the similarity between concepts is measured by Pointwise Mutual Information (PMI). The present invention defines a new semantic similarity measurement method between words w and w', and uses the expectations of two concepts to define PMI. , each concept has a series of vocabulary components, which is also a special property of the LDA model. The point mutual information of two words w and w' is PMI(w, w'), then:

In the formula, P(w,w')=P(w)P(w'|w);

In the formula, z is the topic, P(z=j) is the probability when the topic is j, P(w|z=j) is the probability of the vocabulary w when the topic is j, and k is the number of concepts;

In the formula, P(w'|z=j) is the probability of w' when the topic is j, and P(z=j|w) is the conditional probability of the topic j when the vocabulary is w.

The calculation formula of the point mutual information of two vocabularies provided by the present invention is to prepare for the subsequent organization and construction of the hierarchical structure of concepts between ontology, and the formula is also used to define the semantic similarity between another concept.

Each concept generated by LDA corresponds to a concept in the ontology structure. Semantic similarity measure is to measure the semantic similarity between two concepts. In the context of a particular context, the semantic similarity of two other concepts. In the context of the p-th concept tp in the upper-level concept set T and the concept tp, the semantic similarity CosTMI(ts′, tr'; tp)

In the formula, tp contains the lexical sequence {wp1, wp2, ..., wpn}; ts' contains the lexical sequence {ws'1, ws'2, ..., ws'n}; tr' contains the lexical sequence {wr' 1, wr'2, ..., wr'n}.

The threshold thct is preset, and if the value of CosTMI(ts', tr'; tp) is greater than a certain threshold thct, a relationship is established between tp and ts, ts'. Through the above definitions and the calculation of semantic similarity, the concepts that can be related are all concepts in the ontology in the ontology construction. Threshold Thct is a value to be determined through experiments. The larger the value, the greater the semantic similarity between two concepts, otherwise the smaller the semantic similarity.

L1 standard similarity measure:

In the formula, P(ts′|tp) is (is the probability of occurrence of ts' in the tp context vocabulary environment), P(tr'|tp) is (is the occurrence probability of tr' in the tp context vocabulary environment) ;

When the standard similarity measure L(ts', tr'; tp) is used to define the relationship between ontology concepts, each concept learned through the Topic Model corresponds to an ontology concept, and each concept ts' or tr' is in The conditional probability in the context of tp is used to calculate the semantic similarity between concepts of the same layer. The smaller the value, the higher the semantic similarity of the value.

Determine the hierarchy of the ontology:

According to rule 1, low-level concepts are more concrete than high-level concepts, and in natural language processing, we are able to discover the most abstract concepts. That is, the most abstract concepts are those that cannot be further subdivided. Therefore, in the process of learning ontology, concepts cannot always be subdivided. We propose a new method to determine the appropriate size of ontology, that is, determine the number of ontology levels in a given domain knowledge base.

Assume that the TopicModel is used to learn three conceptual levels Th, Tm, and Tl. Th is the highest level, Tm is the middle level, and Tl is the lowest level. The entropy of these three variables is recorded as H(Th), H(Tm), H( Tl), H(Tl|Tm) is the conditional entropy in the information field, then the concept set information gain Δ(I(Th, Tm, Tl)) of two consecutive layers is defined as:

Δ(I(Th, Tm, Tl))=H(Th)-H(Tl|Tm)

When Δ(I(Th, Tm, Tl)) is less than the specified threshold ω, stop using the LDA model to learn the concept set. This inequality means that when the value of Δ(I(Th, Tm, Tl)) is less than a certain threshold, the concept distribution semantics of Tm and Tl are very similar, and LDA concept learning reaches the highest expectation of an ontology concept level learning. At this time, the concept level Quantity is the number of conceptual levels of the ontology. In actual experiments, we set the value of ω close to 0.

The ontology construction method proposed by us is verified experimentally by the GENIA ontology corresponding to the GENIA corpus. The GENIA corpus is a biological corpus. The corpus contains 1,999 medical words collected from MeSH, human, and blood cells. The GENIA ontology contains 45 concepts and 42 relations. The content of our experiment is to input the GENIA prediction into the LDA model, and calculate the concepts required to construct the ontology. We compared the method we proposed with the method proposed by Zavitsanos et al. The execution of the algorithm was completed on a Pentium 4 PC with 2GB of memory. We compared the CI method proposed by CosTMI and Zavitsanos et al., CI, CP and L1. The parameter settings of the method are shown in Table 1.

Table 1 Parameter settings of the similarity measurement method

The following describes our experimental results in detail. The GENIA ontology hierarchy consists of two different ontologies. We utilize recall, precision, and F1 metrics to evaluate the performance efficiency of our proposed method and the quality of the resulting ontology structures. The formula for calculating the recall rate Rec is as follows:

In the formula, n _rc is the number of correct concepts learned by the algorithm, and N _r is the number of concepts calculated by the model as a whole.

The accuracy rate Prec is defined as follows:

In the formula, n _pc is the number of concepts learned, and N _p is the number of concepts learned by the algorithm.

The formula for calculating the F1 metric is as follows:

The comparison results of the two methods in terms of learning concepts are shown in Table 2:

Table 2 Concept C of Algorithm Learning Based on the Execution Results of Similarity Measures

From Table 2, we can see that the AOL implementation results of our proposed method are very effective, and can be used for ontology construction of knowledge in other domains, and the accuracy and recall rates are higher than the CI method.

Table 3. Execution results of different similarity measures to construct relationships between reconstructions

As shown in Table 2 and Table 3, the comparison results are very satisfactory, and the proposed algorithm is proved to be very effective in constructing domain ontology. It can be seen from the experimental comparison results that our algorithm is indeed slightly inferior in the accuracy of recognizing concepts. The reason may be that the algorithm is still lacking in recognizing very specific concepts.

Figure 1 shows the number of words contained in each concept. During our experiments, we found that the number of words contained in each concept will affect the accuracy of ontology construction. Experimental results show that if each concept contains less than 10 words, the accuracy of ontology construction will be seriously affected. Conversely, if each concept contains more words, the accuracy of the constructed ontology will be higher. However, the more concepts it contains, the better. Through experimental testing and analysis, each concept contains 16 words. If the concept contains too many words, some low-frequency words that appear in the corpus will appear in the concept. Ontology construction The abstract meaning of the concept in the ontology is not significant, but will affect the actual quality of ontology construction. As shown in the results in Fig. 1, the effect of using the CP and L1 metrics proposed in this patent is very good. The experimental results also show that the lexical semantic similarity measure defined in this patent is effective in measuring conceptual semantic similarity in the process of constructing an ontology.

We also use the algorithm in the process of building ontology, the relationship between the change of ontology level depth and the accuracy rate. The experimental results are shown in Figure 2 and Figure 3, which mainly show the change of accuracy F1 with the change of ontology depth when using CP and L1 metrics. The parameters th _cp =0.93 and th _fl =1.24 are set in our experiments. The criterion for the end of the algorithm is according to the value of DMI defined by us, which is set as ω=0.01. Figure 2 depicts that when the depth of the ontology reaches 7, the accuracy metric F1 reaches the highest. For Figure 3, when the body depth reaches 8, the accuracy rate F1 reaches the highest.

Finally, we must mention some factors that affect ontology construction. Automatic ontology construction is an open research field, and there is no fixed standard to evaluate the quality and effect of all ontologies. In addition, the ontology GENIA used in this patent experiment is also Constructed by domain experts, that is, human subjective or comparatively basic methods. So it will be more difficult to measure a subjective method by an objective method. In addition, the evaluation of automatically constructing ontology is more complex and difficult than expanding and updating ontology based on seed ontology.

Claims (2)

1. A method utilizing the automatic learning ontology of Topic Model, is characterized in that, comprises the following steps: The first step is to use the LDA model to extract concepts from a given document corpus, generate a concept set from the extracted concepts, and then carry out conceptual hierarchical subdivision to generate a hierarchical structure G of ontology construction, G={T, E}, In the formula, T = {t1, t2, ..., tm} is a concept set, which is defined as the upper-level concept set; T' = {t1', t2', ..., tm'} is a sub-concept set, which is defined as the upper-level concept The concept set of the next layer of the set T, the concept set T and the sub-concept set T' are two consecutive layers; E is a set of edges, and each eij∈E represents the i-th concept ti and the sub-concept set in the concept set T The jth concept tj' in T' is connected by an edge; The second step is to use the CosTMI similarity measurement method to identify the semantic similarity between two consecutive layers in the hierarchical structure G, wherein, in the context of the p-th concept tp in the upper-level concept set T and the concept tp, the lower-level concept set The semantic similarity between the sth concept ts' and the rth concept tr' in T' CosTMI(ts', tr'; tp) In the formula, tp contains the lexical sequence {wp1, wp2, ..., wpn}; ts' contains the lexical sequence {ws'1, ws'2, ..., ws'n}; tr' contains the lexical sequence {wr' 1, wr'2,...,wr'n}; PMI() is the point mutual information of two words, and the point mutual information of two words w and w' is PMI(w, w'), then: In the formula, P(w, w’)=P(w)P(w’|w); In the formula, z is the topic, P(z=j) is the probability when the topic is j, P(w|z=j) is the conditional probability of the vocabulary w when the topic is j, and k is the number of concepts; In the formula, P(w'|z=j) is the conditional probability of w' when the topic is j, and P(z=j|w) is the conditional probability of topic j when the vocabulary is w; If CosTMI(ts', tr'; tp) is greater than a certain threshold thc, establish a relationship between tp and ts', tr'; The third step is to calculate the standard similarity measure L(ts', tr'; tp), In the formula, P(ts'|tp) is (is the occurrence probability of ts' in the tp context vocabulary environment), P(tr'|tp) is (is the occurrence probability of tr' in the tp context vocabulary environment) ; When the standard similarity measure L(ts', tr'; tp) is used to define the relationship between ontology concepts, each concept learned through TopicModel corresponds to an ontology concept, and each concept ts' or tr' is in tp The conditional probability in the context environment is used to calculate the semantic similarity between the concepts of the same layer, and the smaller the value, the higher the semantic similarity of the value; The fourth step is to determine the hierarchy of the ontology Assume that the TopicModel is used to learn three conceptual levels Th, Tm, and Tl. Th is the highest level, Tm is the middle level, and Tl is the lowest level. The entropy of these three variables is recorded as H(Th), H(Tm), H( Tl), H(Tl|Tm) is the conditional entropy in the information field, then the concept set information gain Δ(I(Th, Tm, Tl)) of two consecutive layers is defined as: Δ(I(Th, Tm, Tl))=H(Th)-H(Tl|Tm) When Δ(I(Th, Tm, Tl)) is less than the specified threshold ω, stop using the LDA model to learn the concept set. 2. a kind of ontology construction method based on Topic Model as claimed in claim 1, it is characterized in that, in the first step, follow the following rules when carrying out concept hierarchy subdivision to produce the hierarchical structure G of ontology construction: Rule 1: If ti∈T, tj'∈T', The conclusion is: the sub-concept set T' is higher than the concept set T, where NT and NT' are the higher levels of the concept set T and the sub-concept set T'respectively; Rule 2: If ti∈T, tj'∈T', It is very likely that there is a subordinate-subordinate relationship between ti and tj', where, is the empty set.