KR20230082457A - Unsupervised domain ontology extraction apparatus and method - Google Patents

Abstract

The present invention relates to an unsupervised domain ontology extraction apparatus and method for automatically extracting ontology information consisting of terms and their relationships from a domain corpus.

Description

Unsupervised domain ontology extraction apparatus and extraction method {UNSUPERVISED DOMAIN ONTOLOGY EXTRACTION APPARATUS AND METHOD}

The present invention relates to an unsupervised domain ontology extraction apparatus and extraction method, and more particularly, to an unsupervised domain ontology extraction apparatus and extraction method for automatically extracting ontology information consisting of terms and their relationships from a domain corpus. It is about.

The present invention relates to an unsupervised domain ontology extraction apparatus and extraction method, and more particularly, to an unsupervised domain ontology extraction apparatus and extraction method for automatically extracting ontology information consisting of terms and their relationships from a domain corpus. It is about.

Korean Patent Publication No. 10-2019-0147203

In order to solve the above problems, the present invention intends to provide an unsupervised domain ontology extraction apparatus and method for automatically extracting ontology information consisting of terms and relationships between them from a domain corpus.

An unsupervised domain ontology extraction apparatus according to an embodiment of the present invention includes a pre-processing unit that inputs a domain corpus and performs morpheme analysis, dependency parsing, and entity name recognition, a term extraction unit that extracts domain term candidates in a sentence, and the dependency A syntax triple extraction unit extracting syntax triples based on the parsing result, a clustering unit generating non-entity names and entity name syntax patterns for the extracted syntax triples, and determining the association of each entity name tag with respect to the generated entity name syntax patterns. and a class generator that generates a class consisting of keywords and subterms and a related keyword generation unit that generates related keywords showing the highest correlation.

In one embodiment, the type of object name recognized by the pre-processing unit may include date, time, place, person, amount, phone number, number of people, and number.

In one embodiment, when the entity name of the input domain corpus is recognized, the pre-processing unit may replace the recognized entity name with an entity name tag.

In one embodiment, the term extracting unit may use a domain term extracting method and a keyword extracting method.

In one embodiment, the clustering unit performs filtering on the generated syntactic pattern to generate a primary syntactic pattern, and generates a secondary syntactic pattern by integrating primary syntactic patterns having similar vocabularies among the generated primary syntactic patterns. can create

In one embodiment, the clustering unit includes a first syntax pattern generation unit that generates a first syntax pattern from the extracted syntax triple, and a filter that removes a first syntax pattern that does not satisfy a predetermined condition from the generated first syntax pattern. and a second syntax pattern generation unit for integrating syntax patterns similar to sub-instance sets while having the same keyword for the first syntax pattern.

In one embodiment, the related keyword generation unit may determine the association using the frequency of co-occurrent with a specific entity name in the entity name syntax pattern or point-by-point mutual information (PMI).

An unsupervised domain ontology extraction apparatus according to another embodiment of the present invention inputs a domain corpus and performs morphological analysis, dependency parsing, and object name recognition including date, time, place, person, amount, phone number, number of people, and number. and, if the entity name for the input domain corpus is recognized, domain term candidates are extracted from the sentence using a preprocessing unit that replaces the recognized entity name with entity name tags, a domain term extraction method, and a keyword extraction method. a term extraction unit that extracts syntax triples based on the dependency parsing result; a syntax triple extraction unit that extracts syntax triples; a clustering unit that generates non-entity name and entity name syntax patterns for the extracted syntax triple; It may include a related keyword generating unit that determines the relevance of each entity name tag and generates a related keyword showing the highest relevance, and a class generating unit that creates a class composed of keywords and subterms.

In another embodiment, the clustering unit includes a first syntax pattern generation unit that generates a first syntax pattern from the extracted syntax triples, and filtering that removes a first syntax pattern that does not satisfy a predetermined condition from the generated first syntax pattern. and a second syntax pattern generation unit for integrating syntax patterns similar to sub-instance sets while having the same keyword for the first syntax pattern.

An unsupervised domain ontology extraction method according to an embodiment of the present invention includes the steps of inputting a domain corpus in a preprocessing unit to perform morpheme analysis, dependency parsing, and object name recognition, and extracting domain term candidates in a sentence in a term extraction unit. extracting a syntax triple based on the dependent parsing result in a syntax triple extractor, generating non-entity name and entity name syntax patterns for the syntax triple extracted in the clustering unit, It may include determining the association of each entity name tag with respect to the entity name syntax pattern, generating a keyword having the highest correlation, and generating a class composed of keywords and subterms in a class generator.

In one embodiment, the step of inputting the domain corpus in the pre-processing unit and performing morpheme analysis, dependency parsing, and entity name recognition may include, when the entity name for the input domain corpus is recognized, the recognized entity name in the pre-processing unit. It may include a step of replacing with an object name tag.

In an embodiment, the step of extracting the domain term candidates within the sentence by the term extractor may include extracting the domain term candidates within the sentence using a domain term extraction method and a keyword extraction method.

In one embodiment, the step of generating a non-entity name and entity name syntax pattern for the syntax triple extracted by the clustering unit generates a first syntax pattern by performing filtering on the generated syntax pattern in the clustering unit. and generating a second syntax pattern by integrating the first syntax patterns having a similar vocabulary among the generated first syntax patterns.

In one embodiment, the step of determining the relevance of each entity name tag with respect to the entity name syntax pattern generated by the relevant keyword generation unit and generating a relevant keyword showing the highest correlation, in the relevant keyword generation unit, the entity name A step of determining correlation using a frequency of co-occurrent with a specific entity name in a syntax pattern or point-by-point mutual information (PMI) may be included.

According to one aspect of the present invention, concept and related sub-term information can be extracted from a domain corpus in an unsupervised manner, and a person can easily build domain knowledge using this.

In addition, according to one aspect of the present invention, slot tagging can be performed on a conversation corpus in an unsupervised manner using the constructed information, and such knowledge has an advantage of being used to build a conversation system and the like.

1 is a diagram schematically showing the configuration of an unsupervised domain ontology extraction apparatus 100 according to the present invention.
2 is a diagram illustrating an example of a dependency parsing tree.
3 is a diagram illustrating a process of clustering syntax triples in a clustering unit.
4 is a diagram showing an example of an entire domain ontology extraction result.

Advantages and features of the present invention, and methods of achieving them, will become clear with reference to the detailed description of the following embodiments taken in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but will be implemented in various different forms, only these embodiments make the disclosure of the present invention complete, and common knowledge in the art to which the present invention belongs. It is provided to fully inform the holder of the scope of the invention, and the present invention is only defined by the scope of the claims. Meanwhile, terms used in this specification are for describing the embodiments and are not intended to limit the present invention. In this specification, singular forms also include plural forms unless specifically stated otherwise in a phrase. As used herein, "comprises" and/or "comprising" means that a stated component, step, operation, and/or element is the presence of one or more other components, steps, operations, and/or elements. or do not rule out additions.

1 is a diagram schematically showing the configuration of an unsupervised domain ontology extraction apparatus 100 according to the present invention.

1, an unsupervised domain ontology extraction apparatus 100 according to an embodiment of the present invention includes a preprocessing unit 110, a term extraction unit 120, a clustering unit 130, and a related keyword generation unit 140. and a class generator 150.

The pre-processing unit 110 serves to perform morpheme analysis, dependency parsing, and object name recognition by taking the domain corpus as an input. At this time, the types of object names recognized by the preprocessing unit 110 include date (date), time (time), poi (place), person (name of person), price (amount), phone.number (telephone number), person. Can include count(number of people), qt.count(number), etc. In addition, when the entity name for the input domain corpus is recognized, the pre-processing unit 110 may replace the recognized entity name with an entity name tag.

The term extractor 120 serves to extract domain term candidates within a sentence. Here, term extraction may use domain terms and keyword extraction methods that are generally used.

The syntax triple extractor 130 serves to extract syntax triples based on the dependency parsing result. More specifically, the syntax triple extractor 130 extracts a syntax triple of (a, rel, b) from the dependency parsing result. Here, a is a dependent element on the dependent syntax tree, rel is a syntax dependency relationship path, and b is a dominant or dependent element. Syntax triples may include not only parental relationships, which are primary dependency relationships, but also grandparent relationships and sibling relationships, which are secondary dependency relationships. This will be examined in more detail with reference to FIG. 2 .

2 is a diagram illustrating an example of a dependency parsing tree.

Referring to Figure 2, it shows the dependency syntax tree for "Is there a seat by the window?", where ('window', 'vmod', 'there'), ('seat', 'subj', 'there') are 1 It becomes the primary dependency syntax triple, and ('window', 'vmod:subj', 'Jari') becomes the secondary dependency syntax triple.

In this case, the syntax triple extractor 130 may further subdivide the dependency relationship as needed. For example, in the case of an adverb (vmod), if a function word exists in a word, the function word may be additionally displayed. Then, in the above example, ('window', 'vmod', 'there') is displayed as ('window', 'vmod/', 'there'). Since these syntax triples are processed only for the vocabulary extracted as terms, only one syntax triple ('window', 'vmod:subj', 'place') is actually extracted in the above example.

Returning to FIG. 1 again, the clustering unit 140 serves to generate non-entity name and entity name syntax patterns for the extracted syntax triples. This will be examined in more detail with reference to FIG. 3 .

FIG. 3 is a diagram illustrating a process of clustering syntax triples in the clustering unit 140 .

Referring to FIG. 3 , the clustering unit 140 performs filtering on the generated syntactic patterns to generate primary syntactic patterns, integrates primary syntactic patterns having similar vocabularies among the generated primary syntactic patterns, and produces a second syntactic pattern. Secondary syntax patterns can be created. To this end, the clustering unit 140 includes a primary syntactic pattern generation unit 141 that generates a primary syntactic pattern from the extracted syntactic triples, and a primary syntactic pattern that does not satisfy a preset condition in the generated primary syntactic pattern. It may include a filtering unit 142 that removes and a secondary syntax pattern generation unit 143 that integrates similar syntax patterns of sub-instance sets while having the same keyword for the primary syntax pattern.

The primary syntax pattern generating unit 141 generates a syntax pattern from syntax triples. A syntax pattern is created by replacing a or b with "*" in the syntax triple (a, rel, b). In the case of ('time', 'obj', 'change') and ('person', 'obj', 'change'), the following syntax patterns can be created.

(*, 'obj', 'change'): {"time", "person"}

'Change', which is a term not substituted with "*" above, becomes a keyword, and the substituted terms become sub-instances.

The filtering unit 142 removes syntax patterns that do not satisfy the following conditions. In the syntax pattern, semantic upper and lower word relationships must be formed between keywords and sub-instances, or there must be high correlation or semantic similarity between sub-instances.

In the case of upper and lower word relationships, the average of the upper and lower word scores must be above the threshold, and for association, the PMI average between the keyword and the instance must be above the threshold value, and for semantic similarity, the variance of the word embedding vector for the instances of the phrase pattern is the threshold should be within

In the case of high and low word scores, it can be extracted using a high and low word classifier. The upper and lower word classifier may be constructed by learning the classifier with training data of pairs of (lower word, upper word) and (lower word, random word) from the upper level word dictionary. Since the final output node of the classifier is a probability distribution value for binary classification, an output value for a high-level word can be used as a score.

At this time, not only filtering for the syntax pattern itself, but also filtering for individual instances within the syntax pattern is performed based on the same criteria.

In the case of entity names, entity names having the same entity name tag become one class, and the syntax triple itself including the entity name tag becomes one syntax pattern.

The secondary syntax pattern generation unit 143 integrates the syntax patterns when keywords are similar to the primary syntax patterns and sub-instance sets are semantically similar. Keyword similarity is determined when the semantic similarity is higher than a certain threshold, and similarity between two sub-instance sets is determined when there is an intersection between the sub-instance sets or when a word pair with the semantic similarity higher than a certain threshold exists between the two sets. When the two syntax patterns are combined, the rel part in (a, rel, "*") is replaced with "*" again to form (a, "*", "*").

Returning to FIG. 1 again, the related keyword generation unit 150 determines the relevance of each entity name tag with respect to the generated entity name syntax pattern, and serves to generate a related keyword showing the highest relevance. Here, the degree of association can be obtained through co-occurrent frequency or point-wise mutual information (PMI) with a specific entity name in a syntactic pattern.

The reason for generating these related keywords is to distinguish them because there may be ambiguity when processing entity names as one class. For example, in "Is there a flight from Incheon to New York?" both 'Incheon' and 'New York' have object name tags that correspond to cities, but 'Incheon' is the departure city and 'New York' is the destination city. and it may be necessary to distinguish them. Accordingly, in the present invention, information for making such a selection can be provided by generating a related keyword based on the relevance of each entity name tag with respect to the syntax pattern through the related keyword generation unit 150.

The class generating unit 160 serves to create a class composed of keywords and subordinate terms when a non-entity name exists in the process of generating a related keyword through the related keyword generating unit. In this case, the keyword can be a class name, the lower term can be an instance, and the relationship can be a 'type_of'. At this time, in the case of an entity name, the entity name tag becomes a class name, and a related keyword becomes a lower term, and at this time, the relationship becomes a 'keyword_of'.

Hereinafter, an actual driving example of the unsupervised domain ontology extraction apparatus 100 according to the present invention will be described through examples.

<example>

In the above example, the preprocessor 110 performs morpheme analysis and object name recognition as follows.

Here, "<poi=Nolbu/NNG old days/NNG barrel/XPN chicken/NNG>" means that "Nolbu old whole chicken" is tagged with the entity name tag poi.

The following is an example of a dependency parsing result.

In the above example, word information is expressed as "word: n (dominant word number, dependent relationship)". That is, "this:0 (1, 'mod') week:1 (2, 'mod')" indicates that the dominion of the 0th word "this" is the 1st word "main" and establishes a dependency relationship of "mod". means to have

Next, in the result of morpheme analysis and dependency parsing, the entity name is replaced with the entity name tag as follows.

- Example of morphological analysis:

between <time=6/SN hour/NNB> to/JX between <time=6/SN hour/NNB 20/SN minute/NNB>/NNG to/JKB change/VV uh/EC week/VX s/EC ./SF

>> Between time/JX time/NNG/JKB change/VV uh/EC share/VX please/EC ./SF

- Example of dependency parsing result:

This time: 0 (1, 'mod') Week: 1 (2, 'mod') Saturday: 2 (3, 'mod') 7:00+To:3 (4, 'vmod') Reservation+H+B:4 (5, 'vp_mod') Gil-kwon Jeon+Lee+Nde+Yo:5 (6, 'vnp') .:6 (-1, 'mod')

>> date:0 (1, 'mod') date:1 (2, 'mod') date:2 (3, 'mod') time+to:3 (4, 'vmod') reserved+ha+b:4 (5, 'vp_mod') person+i+bde+yo:5 (6, 'vnp') .:6 (-1, 'mod')

Next, in the term extraction unit 120, tf-idf is used to extract words and finite adjectives. The list of words and finite adjectives extracted by tf-idf in order of frequency is as follows.

Reservation:224, person.count:98, date:92, time:83, change:70, seat:70, person:61, room:43, name:35, window:34, number:23, time:22, Cancel:19, pax:13, room:13, hall:11, phone:11, chair:10, date:9, table:9, name:8, qt.count:7, general:7, contact:5, Completed: 4, Restaurant: 4, Closed: 2

Next, an example of a result of extracting a syntax triple in the syntax triple extractor 130 is provided.

Next, it is an example of a result of arranging the primary phrase patterns for the corpus in the order of upper and lower word score averages in the clustering unit 140.

In the above example, 'freq' is the frequency of the pattern, 'var' is the variance of the embedding vector of sub-terms, 'sim' is the average semantic similarity between sub-terms and keywords, 'pmi' is the average PMI between sub-terms and keywords, 'hyper' means the average of upper and lower word scores between lower terms and keywords.

When filtering on the syntax pattern is performed, ('*', 'vmod/to', 'place') and ('*', 'vmod/to:subj', 'place'), ('*', ' mod', 'place') is selected because the average of the upper and lower word scores is high, and ('*', 'vmod', 'change') and ('*', 'obj', 'change') are higher and lower word scores Although low, it is selected because the PMI average is high. ('reservation', 'subj', '*') are removed because they are both not high.

In the case of 'time' in ('*','vmod/to:subj','digit'), only the corresponding item is removed because the upper and lower scores and PMI values are too low. In the case of "normal" in ('*', 'mod', 'digit'), the upper and lower scores are low, but the PMI value is high and maintained.

Therefore, finally, the following primary syntax pattern is extracted.

In the above syntax pattern, ('*', 'vmod/to', 'place') and ('*', 'vmod/to:subj', 'place'), ('*', 'mod', 'place') ) is integrated into one because the keywords are all the same as "seat", and sub-entities share "window" or words with high semantic similarity between 'room' and 'room'. ('*', 'vmod', 'change') and ('*', 'obj', 'change') are also merged into one.

Then, the following two classes are finally created.

('*', '*', 'seat'): {'hall', 'window', 'room', 'regular'}

('*', '*', 'change'): {'time', 'person', 'date'}

Next is an example of a syntax triple extraction result for entity names.

In the result of the syntax pattern, the related keywords aired by each entity name tag are listed in order of frequency as follows.

Basically, entity names form one class, and if necessary, they can be divided by combining related keywords. For example, in the case of person.count, in the two sentences below, the use of person.count is used differently and needs to be distinguished.

Sentence 1: Could I change the reservation to <person.count=8> window seats around 1:00 on Tuesday?

Sentence 2: We only have seats for <person.count=4> by the window.

That is, the first person.count means the number of people, and the second person.count means the number of digits. Therefore, in this case, it is necessary to divide into person.count_ seat and person.count_ number of people.

Also in the statement below, the two person.counts are used for different purposes.

Sentence: I made a reservation for <person.count=6> people, but I want to change it to <person.count=4> people.

The first person.count is the number of people booked, and the second person.count is the number of people to change. Therefore, for this case, it may be necessary to distinguish person.count_reservation, person.count_change, etc., and finally, the following classification should be made.

person.count: person.count_reservation_number of people, person.count_change_number of people, person.count_places

Keyword information according to the present invention can provide information so that a person can determine such classification. To this end, the top N keywords are provided.

The result of extracting the entire domain ontology for the above example may appear as shown in FIG. 4 .

Although the above has been described with reference to preferred embodiments of the present invention, those skilled in the art will variously modify and change the present invention within the scope not departing from the spirit and scope of the present invention described in the claims below. You will understand that it can be done.

100: unsupervised domain ontology extraction device
110: pre-processing unit
120: term extraction unit
130: syntax triple extraction unit
140: clustering unit
150: related keyword generation unit
160: class generator

Claims (1)

a pre-processing unit that inputs the domain corpus and performs morpheme analysis, dependency parsing, and object name recognition;
a term extraction unit for extracting domain term candidates within a sentence;
a syntax triple extraction unit extracting a syntax triple based on the dependency parsing result;
a clustering unit generating non-entity name and entity name syntax patterns for the extracted syntax triple;
a related keyword generation unit for generating a related keyword showing the highest correlation by determining the relevance of each entity name tag with respect to the generated entity name syntax pattern; and
An unsupervised domain ontology extraction device comprising: a class generator for generating a class composed of keywords and subterms.

Images