CN107895012A - A kind of body constructing method based on Topic Model - Google Patents

Abstract

A kind of body constructing method based on Topic Model provided by the invention.The present invention proposes AOL methods, this method supports automatic domain body structure, the measure of Semantic Similarity between a kind of calculating concept of information is invented, for calculating the Semantic Similarity between concept caused by LDA models, AOL methods do not limit the child node quantity of root node, it is not necessary to have auxiliary of the seed body as initial study body.Test result indicates that the method proposed by the present invention that automated ontology structure is carried out using Topic Model is very effective.

Description

A kind of body constructing method based on Topic Model

Technical field

The present invention relates to a kind of method of ontological construction, by the use of TopicModel as basic conception unit is produced, does not have to Body seed can learn the purpose that body reaches structure body.

Background technology

In recent years, ontological construction has been applied to various fields, such as artificial intelligence, information extraction, machine translation field. But artificial constructed body is the work of very time and effort consuming, due to this reason, analyzed using computer data, data mining It is a research highly significant that mode builds body automatically, has attracted many researchers are largely deep to this progress to grind Study carefully.Most of present Method for Ontology Learning concentrate on extension, update existing body seed, are extracted using from document dictionary Go out concept or lexical unit to update and spread body seed.Also have some methods for learning body automatically, but it is most this The method of kind of automatic study body is all based on the ontological construction in special knowledge field, and such as SKOS models, but these methods are all With certain limitation.

Topic Model probabilistic models are a kind of in the case where no priori provides, and are known from scientific publications Do not go out the very effective model by industry-proven of concept.Topic Model models have been widely applied to text now This excavation applications.

Elias Zavitsanos etc. propose a kind of automated ontology learning method based on statistical method, and this method is to pass through The concept set that constantly recycling Topic Model model trainings go out, conditional independence is then recycled to judge to identify Concept between contact, but this method can not carry out the contact of concept between two hierarchical structures.Wang wei et al. are carried The method that two methods are all based on Semantic Web study body construction is gone out, this method utilizes information theory and Topic Model phases With reference to mode, show good recall rate and accuracy rate, but need to limit the number of the sub- concept node of nearest root node Amount.

The content of the invention

It is an object of the invention to provide the measure of Semantic Similarity between a kind of calculating concept of information, for calculating Semantic Similarity between concept caused by LDA models.

In order to achieve the above object, the technical scheme is that providing a kind of body structure based on Topic Model Construction method, it is characterised in that comprise the following steps：

The first step, using LDA models concept extraction is carried out from given document corpus, produced by the concept being drawn into Go out concept set, then the hierarchical structure G, G={ T, E } of progress concept hierarchy subdivision generation ontological construction, in formula, T=t1, T2 ..., tm } it is concept set, it is defined as Upper Concept set；T '=t1 ', t2 ' ..., tm ' } it is sub- concept set, definition For Upper Concept set T next layer of concept set；E is the set on side, and each eij ∈ E represent i-th in concept set T Concept ti has side to be connected with j-th of concept tj ' in sub- concept set T '；

Second step, using CosTMI method for measuring similarity, the similitude in identification hierarchical structure G between each concept, The potential contact of concept between i.e. adjacent level, wherein, in Upper Concept set T in p-th of concept tp and concept tp context, In next layer of concept set T ' two concepts of s-th of concept ts ' and r-th of concept tr ' semantic similarity CosTMI (ts ', tr′；tp)

In formula, tp includes sequence of words { wp1, wp2 ..., wpn }；Ts ' includes sequence of words { ws ' 1, ws ' 2 ..., ws ' n}；Tr ' includes sequence of words { wr ' 1, wr ' 2 ..., wr ' n }；PMI () is the point mutual information of two vocabulary, two vocabulary w with W ' point mutual information is PMI (w, w '), then has：

In formula, P (w, w ')=P (w) P (w ' | w)；

In formula, z is theme, and P (z=j) is probability when theme is j, and P (w | z =j) when to be theme be j, vocabulary w conditional probability, k is the quantity of concept；

In formula, P (w ' | z=j) is theme when being j, and w ' condition is general Rate, P (z=j | w) is vocabulary when being w, the conditional probability that theme is j.

Preferably, in the first step, carry out concept hierarchy subdivision produce ontological construction hierarchical structure G when follow with Lower rule：

Rule 1：If ti ∈ T, tj ' ∈ T ', NT ＜ NT ', conclusion are：Sub- concept set T ' than concept set T, wherein, NT and NT ' is concept set T and sub- concept set T ' floor height rank respectively；

Rule 2：If ti ∈ T, tj ' ∈ T ',In ti and tj ' between relationship between superior and subordinate very likely be present, Wherein,It is empty set.

The present invention proposes AOL methods, and this method supports automatic domain body structure, invented a kind of calculating of information The measure of Semantic Similarity between concept, for calculating the Semantic Similarity between concept caused by LDA models, AOL side Method does not limit the child node quantity of root node, it is not necessary to has auxiliary of the seed body as initial study body.Experimental result table Bright, the method proposed by the present invention that automated ontology structure is carried out using Topic Model is very effective.

The present invention by recycle LDA models i.e. Topic Model models produce concept, definition can accurate measurement it is general The measure of Semantic Similarity builds the layer of structure between the concept of body and concept between thought.

Brief description of the drawings

Fig. 1 is the process of structure body construction；

Fig. 2 is graph of a relation of the accuracy with vocabulary dimension of concept；

Fig. 3 is body layer sub-quantity and the relativity figure of F1 measurements.

Embodiment

With reference to specific embodiment, the present invention is expanded on further.It should be understood that these embodiments are merely to illustrate the present invention Rather than limitation the scope of the present invention.In addition, it is to be understood that after the content of the invention lectured has been read, people in the art Member can make various changes or modifications to the present invention, and these equivalent form of values equally fall within the application appended claims and limited Scope.

A kind of body constructing method based on Topic Model provided by the invention, comprises the following steps：

The first step, using LDA models concept extraction is carried out from given document corpus, it is thin then to carry out concept hierarchy Divide the hierarchical structure for producing ontological construction；

Second step, design CosTMI method for measuring similarity, identify the similitude between level structuring concept, i.e. adjacent layer The potential contact of concept between secondary；

Above-mentioned steps are related to following technological innovation：

One) ontological construction process

Fig. 1 illustrates the process of ontological construction.Hierarchical structure a G, G={ T, E } are built, in formula, T=t1, t2 ..., Tm } it is concept set, referred to as conceptual level, by LDA model outputs, Upper Concept set can be defined as.T '=t1 ', T2 ' ..., tm ' } it is sub- concept set, it is defined as Upper Concept set T next layer of concept set.E is the set on side, each Eij ∈ E represent i-th of concept ti in concept set T and j-th of concept tj ' in sub- concept set T ' have side be connected G=T, E }, T={ t1, t2 ..., tm } is the set of a concept in figure.

In order to build the contact between upper and lower two layers of concept, it is thus necessary to determine that the concept hierarchy belonging to these concept nodes, which Belong to high one layer of concept set, which belongs to low one layer of concept set, and sets up the connection between this two layers of concept set System can be more complicated.It is not especially clearly, it is necessary to will using certain measure using the boundary between the concept of LDA models These Concept Hierarchies, and relation between layers is also set up, some concepts may have several fathers, and some concepts can Can there is no child, caused concept hierarchy is more, and the relation between conceptual level is closer, so quantity caused by level concept is not Can unrestrictedly it increase, it is necessary to be manually set the level quantity of an ontological construction.

Two) relevant rule

Before proposing to implement automated ontology learning method, two basic rules are defined first.In general situation Under be that continuous recycling LDA models produce concept set, for building the concept required for hierarchical structure.The present invention is fixed Some rules of justice, these rules are used for limiting the concept that the model produces, used when building hierarchical structure body.

According to intuition, it is more abstract to be more in the concept of high level, otherwise more specific；It is fewer to be more in the concept of high level, it is on the contrary It is more.These general knowledge are so based on, are defined as follows rule：

Rule 1：If ti ∈ T, tj ' ∈ T ', NT ＜ NT ', conclusion are：Sub- concept set T ' than concept set T, wherein, NT and NT ' is concept set T and sub- concept set T ' floor height rank respectively.

When being repeated with LDA models when going study to produce concept set, it is necessary to determine NT ＜ NT ' first.Therefore should Rule is very important for the method for building body.

By document corpus by LDA each of the every layer concepts for learning be in the literature high frequency occur word Converge, in the concept set that high-rise high frequency occurs, very likely same high frequency occurs in low layer concept set, so in structure originally These identical vocabulary may establish contact during body, and this is irrational.Therefore it is defined as follows rule：

Rule 2：If ti ∈ T, tj ' ∈ T ',In ti and tj ' between relationship between superior and subordinate very likely be present, Wherein,It is empty set.

The rule can help the similarity measurement to be introduced below this patent between our defined notions.

Three) similarity measurement

The present invention builds the hierarchical structure of body using the method for similarity measurement, that is to say, that the contact between concept It is to be established by the similarity between concept.Reach certain similarity between two concepts that two level concepts are concentrated Value, could establish contact, otherwise it is assumed that being not in contact between them.It is semantic similar between two concepts in order to calculate Property, utilize LDA models produce concept set symphysis into concept matrix, each Input matrix is that concept is appeared in body Possibility size.

Similitude between usual concept is spent using point mutual information PMI (Pointwise Mutual Information) Amount, invention defines Semantic Similarity measure between a kind of new vocabulary w and w ', is determined using the expectation of two concepts Adopted PMI, each concept are made up of a series of vocabulary, and this is also a special nature of LDA models.Two vocabulary w's and w ' Point mutual information is PMI (w, w '), then has：

In formula, P (w, w ')=P (w) P (w ' | w)；

In formula, z is theme, and P (z=j) is probability when theme is j, and P (w | z =j) when to be theme be j, vocabulary w probability, k is the quantity of concept；

In formula, P (w ' | z=j) is theme when being j, w ' probability, P (z=j | w) is vocabulary when being w, the conditional probability that theme is j.

The calculation formula that the present invention provides the point mutual information of two vocabulary is the concept between follow-up tissue construction body Hierarchical structure is prepared, and the Semantic Similarity defined between another concept can also use the formula.

Each concept corresponds to a concept inside body construction as caused by LDA.Semantic Similarity measurement is measurement two Semantic similarity between individual concept.In the context of special linguistic context, the semantic similarity of two other concept.Upper Concept In set T in p-th concept tp and concept tp context, in next layer of concept set T ' s-th of concept ts ' and r-th it is general Read semantic similarity CosTMI (ts ', tr ' of two concepts of tr '；tp)

In formula, tp includes sequence of words { wp1, wp2 ..., wpn }；Ts ' includes sequence of words { ws ' 1, ws ' 2 ..., ws ' n}；Tr ' includes sequence of words { wr ' 1, wr ' 2 ..., wr ' n }.

Preset threshold value thct, if CosTMI (ts ', tr '；Tp) value is more than certain threshold value thct, in tp and ts, Ts ' opening relationships.By above-mentioned definition and the calculating of Semantic Similarity, what is drawn all can be body with the concept of opening relationships A concept in structure in body.Threshold value Thct is by testing the value to be determined, this value two concepts of bigger explanation Between Semantic Similarity it is bigger, otherwise Semantic Similarity is smaller.

The having set forth herein body constructing method following with real GENIA corpus and the checking of body GENIA bodies Effect property and practicality.

Structure bulk process proposed by the invention, GENIA bodies are tested to carry out experiment corresponding to GENIA corpus Card.GENIA corpus is a biological corpus.The corpus includes 1,999 medical vocabularies, be from MeSH, human and Collect and obtain in blood cells.45 concepts and 42 relations are included in GENIA bodies.The present invention experiment content be by GENIA expectations are input to LDA models, calculate the required concept of body to be built.The present invention compared for side proposed by the present invention The algorithm that method AOL and Zavitsanos et al. are proposed, execution is completed in Pentium 4, internal memory 2GB PC, compared for The CI methods that CosTMI and Zavitsanos et al. are proposed, the threshold value of parameter setting is 0.93 and 3*10-6 respectively.

Algorithm proposed by the present invention is finally measured with recall rate, accuracy rate and F1 to assess the matter of validity and body construction Amount.It is as shown in table 1 that two methods perform comparing result.

The implementing result of the concept C of table 1 and relation S based on similarity measurement

From table 1 it will be seen that it is proposed that method AOL implementing results be that effectively, can be used for The ontological construction of other field knowledge, accuracy rate and recall rate are all above CI methods.

Fig. 2 illustrates the vocabulary quantity that each concept includes, and is found in we do experimentation, each concept is included Vocabulary quantity influence whether the accuracy of ontological construction.Test result indicates that if each concept includes the vocabulary of less than 10 Quantity, the accuracy of ontological construction can be had a strong impact on., whereas if the vocabulary quantity that each concept includes is more, this is constructed The accuracy of body is also higher.But be not that the concept included is The more the better, by experimental test and analysis, each concept includes 16 Individual vocabulary result can be relatively good, if concept is too many comprising vocabulary, some low-frequency words occurred in corpus occur in concept Converge, it is little to the abstract sense of concept in ontological construction, the actual mass of ontological construction is influenced whether on the contrary.

In figure 3 we show the algorithm performs degree of accuracy a detail view, CosTMI measurements are illustrated in figure and are downloaded During threshold value thct=0.93, algorithm performs are the situations of change of F1 values, in figure 3 it will be seen that working as body layer sub-quantity For 7 when F1 value highests.

Claims (2)

1. a kind of body constructing method based on Topic Model, it is characterised in that comprise the following steps：

The first step, using LDA models concept extraction is carried out from given document corpus, produced generally by the concept being drawn into Set is read, then the hierarchical structure G, G={ T, E } of progress concept hierarchy subdivision generation ontological construction, in formula, T=t1, T2 ..., tm } it is concept set, it is defined as Upper Concept set；T '=t1 ', t2 ' ..., tm ' it is sub- concept set, definition For Upper Concept set T next layer of concept set；E is the set on side, and each eij ∈ E represent i-th in concept set T Concept ti has side to be connected with j-th of concept tj ' in sub- concept set T '；

Second step, using CosTMI method for measuring similarity, the similitude in identification hierarchical structure G between each concept, i.e. phase The potential contact of concept between adjacent bed time, wherein, it is next in Upper Concept set T in p-th of concept tp and concept tp context Semantic similarity CosTMI (ts ', tr ' of two concepts of s-th of concept ts ' and r-th of concept tr ' in layer concept set T '； tp)

<mrow> <mi>C</mi> <mi>o</mi> <mi>s</mi> <mi>T</mi> <mi>M</mi> <mi>I</mi> <mrow> <mo>(</mo> <msup> <mi>ts</mi> <mo>&amp;prime;</mo> </msup> <mo>,</mo> <msup> <mi>tr</mi> <mo>&amp;prime;</mo> </msup> <mo>;</mo> <mi>t</mi> <mi>p</mi> <mo>)</mo> </mrow> <mo>=</mo> <mfrac> <mrow> <msubsup> <mo>&amp;Sigma;</mo> <mrow> <mi>i</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>n</mi> </msubsup> <mi>P</mi> <mi>M</mi> <mi>I</mi> <mrow> <mo>(</mo> <msup> <mi>ws</mi> <mo>&amp;prime;</mo> </msup> <mi>i</mi> <mo>,</mo> <mi>w</mi> <mi>p</mi> <mi>i</mi> <mo>)</mo> </mrow> <mi>P</mi> <mi>M</mi> <mi>I</mi> <mrow> <mo>(</mo> <msup> <mi>wr</mi> <mo>&amp;prime;</mo> </msup> <mi>i</mi> <mo>,</mo> <mi>w</mi> <mi>p</mi> <mi>i</mi> <mo>)</mo> </mrow> </mrow> <mrow> <msqrt> <mrow> <msubsup> <mo>&amp;Sigma;</mo> <mrow> <mi>i</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>n</mi> </msubsup> <mi>P</mi> <mi>M</mi> <mi>I</mi> <mrow> <mo>(</mo> <msup> <mi>ws</mi> <mo>&amp;prime;</mo> </msup> <mi>i</mi> <mo>,</mo> <mi>w</mi> <mi>p</mi> <mi>i</mi> <mo>)</mo> </mrow> <mn>2</mn> </mrow> </msqrt> <msqrt> <mrow> <msubsup> <mo>&amp;Sigma;</mo> <mrow> <mi>i</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>n</mi> </msubsup> <mi>P</mi> <mi>M</mi> <mi>I</mi> <mrow> <mo>(</mo> <msup> <mi>wr</mi> <mo>&amp;prime;</mo> </msup> <mi>i</mi> <mo>,</mo> <mi>w</mi> <mi>p</mi> <mi>i</mi> <mo>)</mo> </mrow> <mn>2</mn> </mrow> </msqrt> </mrow> </mfrac> </mrow>

In formula, tp includes sequence of words { wp1, wp2 ..., wpn }；Ts ' includes sequence of words { ws ' 1, ws ' 2 ..., ws ' n }； Tr ' includes sequence of words { wr ' 1, wr ' 2 ..., wr ' n }；PMI () is the point mutual information of two vocabulary, two vocabulary w and w ' Point mutual information be PMI (w, w '), then have：

In formula, P (w, w ')=P (w) P (w ' | w)；

In formula, z is theme, and P (z=j) is probability when theme is j, P (w | z=j) When to be theme be j, vocabulary w conditional probability, k is the quantity of concept；

In formula, P (w ' | z=j) is the conditional probability of w ' when theme is j, P (z=j | w) is vocabulary when being w, theme j conditional probability.

2. a kind of body constructing method based on Topic Model as claimed in claim 1, it is characterised in that described the In one step, carry out following following rule when concept hierarchy subdivision produces the hierarchical structure G of ontological construction：

Rule 1：If ti ∈ T, tj ' ∈ T ',Conclusion is：Sub- concept set T ' than concept set T, wherein, NT and NT ' It is concept set T and sub- concept set T ' floor height rank respectively；

Rule 2：If ti ∈ T, tj ' ∈ T ',In ti and tj ' between relationship between superior and subordinate very likely be present, wherein,It is empty set.