KR20100068929A - System and method for annotating semantic tags to electronic documents - Google Patents

Abstract

PURPOSE: A system and a method for annotating semantic tags to electronic documents are provided to partially automatise the procedure for annotating a natural language sentence in order to reduce annotation time. CONSTITUTION: An electronic document collecting device(5) collects an electronic document corresponding to user's request on a web. An electronics document group analysis device(6) selects the electronic document. A sentence extractor(7) naturally extracts the natural language sentences from the selected electronic document. A simple sentence divider(8) classifies the natural language sentences into simple sentences. A triple extractor(9) extracts the triple from the simple sentence. A triple matcher(11) matches each part of the triple with an existing machine-readable knowledge structure.

Description

System and method for attaching semantic information to electronic documents {System and method for annotating semantic tags to electronic documents}

The present invention relates to a system and method for attaching semantic information to an electronic document. More particularly, the present invention relates to a system and method for converting semantic information to existing natural language data into a machine-readable form by semi-automatically adding the semantic information.

Current information transmission technology through the Internet is so advanced that anyone can access the information anytime, anywhere. In the present situation, which is a flood of information, many researchers are developing and developing meaningful information transfer technologies for their own purposes. Among these conventional techniques, a system for adding semantic information of each word in a HyperText Markup Language (HTML) document has been proposed (see J. Heflin, J. Hendler, and S. Luke. Shoe: A knowledge representation language for internet applications.In Technical Report CS - TR- 4078 , volume UMIACS TR-99-7. Dept. of Computer Science, University of Maryland at College Park, 1999), a graphical user interface (GUI) based annotation tool that adds semantic information of a machine-readable knowledge structure to an HTML page through a simple mouse drag-and-drop interface. A technique has been proposed (CREME: S. Handschuh and S. Staab. Authoring and annotation of web pages in cream.In Proceedings of the 11th international conference on World Wide Web, pages 462-473, New York, NY, USA, 2002. ACM.), And Annotizer, a system that provides the ability to annotate a single HTML page over the web at the same time, has been proposed (see User-friendly www annotation system for collaboration in research and education environments. The IASTED International Conference on Web Technolgies, Applciations and Services, WTAS 2006).

The conventional technique may take a long time because the user views and annotates the natural language sentence of a given electronic document one by one. In order to solve the above disadvantages, the system and method for attaching semantic information to an electronic document developed in accordance with the present invention partially automates the process of annotating natural language sentences, thereby reducing the number of times a user must view natural language sentences. To reduce annotation time.

In addition, the above-described conventional technology has a problem that the annotation of the user may be involved and annotation output may not be consistent because the user manually annotates a given natural language sentence. However, the system and method for attaching semantic information to an electronic document according to the present invention can reduce the subjective intervention of the user by partially processing the annotation process of the natural language sentence, thereby obtaining a relatively consistent annotation output.

In addition, the above-described prior art does not have a device for recording a subjective judgment of the user, and thus it is not easy to improve the device itself or to prepare a guideline used to take over an annotation work to another user. The system and method for attaching semantic information to an electronic document according to the present invention allows the user to attach semantic information to the electronic document according to the present invention by leaving a record by classifying all subjective judgments used by the user in the process of annotating. Not only can it be used to improve the internal workflow of the device, it can also be used as a guideline for creating guidelines for other users to perform the same tasks consistently.

In addition, the above-described prior art can make unnecessary annotations because there is no device for comparing and analyzing an electronic document and its annotation outputs, but the system and method for attaching semantic information to an electronic document according to the present invention is an annotation output. By matching the semantic information described by the existing machine-readable knowledge structure to annotate natural language sentences that are already annotated and described as semantic information in the machine-readable knowledge structure. Reduce.

As described above, an apparatus for attaching semantic information to an electronic document according to the present invention for solving the conventional problems,

An electronic document collection device for collecting an electronic document meeting a user's information needs on a web; An electronic document set analyzing apparatus for selecting an electronic document by analyzing electronic documents in the collected electronic document set or an electronic document set provided by a user; A sentence extracting device extracting a natural language sentence from the selected electronic document; A simple sentence device for separating the sentence into simple sentences; A triple extraction device for extracting triples from the simple sentences; A triple mapping device for mapping each part of the triple to an existing machine-readable knowledge structure; A machine-readable knowledge structure document output device for outputting a machine-readable knowledge structure as an electronic document from the triples and mapping information; An error log input device for allowing a user to leave a record of what has been determined; And an error log analysis device that analyzes and records the user's records in numerical form.

In addition, the electronic document includes natural language sentences such as HTML documents, Wikipedia documents, Portable Document Format (PDF) documents, Microsoft Word documents, Korean documents, electronic documents made using an optical reader, and can be processed by a machine. Electronic documents are all possible.

In addition, the natural language includes all languages that can be recorded as electronic documents, such as English, Korean, Japanese, Chinese, German, and French.

The electronic document set may be an electronic document set, such as an electronic document set arbitrarily selected by a user, a Wikipedia document set, or an HTML document set collected on the Web.

In addition, the electronic document set may include link information between documents.

In addition, the language for describing the machine-readable knowledge structure includes an interpretation processing device such as OWL (Web Ontology Language) and KIF (Knowledge Interchange Format), and any language in which the machine can interpret and interpret the meaning is possible. .

In addition, the triple may be mapped to an RDF (Resource Description Framework) triple as well as other machine-readable knowledge structure languages, so long as the existing machine-readable knowledge structure can be extended or a machine-readable knowledge structure can be newly generated.

In addition, the apparatus for attaching semantic information to the electronic document may be used through a web browser by providing a web interface, and may be manufactured and used in the form of a general application application.

In addition, a device for attaching semantic information to the electronic document may be stored only on the user's computer and used only on the computer, or the information may be stored on a server and used by a plurality of people over a network.

In addition, the machine-readable knowledge structure document output device may be output in the form of a graph on the screen, and may be output as a file using a machine-readable language such as XML (Extensible Markup Language).

In addition, the error log input device presents an error log type to the user, and the user selects one of them and inputs another comment.

In addition, the error log analysis apparatus provides statistical values for the error logs input as described above to quantify the subjective judgment of the user and use it as data for improving other devices.

In addition, the error log input device and the error log analysis device are all devices that the user's manual intervention (electronic document collection device, electronic document set analysis device, sentence extraction device, simple sentence device, triple extraction device, triple mapping device) It is linked with the machine-readable knowledge structure document output device so that the user can input and analyze the error log at every step.

As described above, the method of attaching semantic information to an electronic document according to the present invention for solving the conventional problems,

A first step of analyzing a given set of electronic documents and selecting electronic documents most overlapping with a pre-generated machine readable knowledge structure document; Selecting a sentence among sentences in the selected electronic document; A third step of dividing the selected sentence into simple sentences using a simple sentence device; A fourth step of extracting triples from the simple sentence using a triple extracting device; A fifth step of mapping each part of the extracted triple to an existing machine-readable semantic structure using a triple mapping device; And a sixth step of outputting the machine-readable knowledge structure document using the extracted triple and the mapping information by using the machine-readable knowledge structure document output device.

In addition, the degree of overlap between the machine-readable knowledge structure document previously generated in the electronic document set analyzing apparatus and the given electronic document is proportional to the frequency in which contents related to the words in the previously generated knowledge structure appear in the given electronic document. Another definition of the overlapping degree is that a given electronic document is classified into similar documents, and the contents related to its representative word appear in the generated knowledge structure.

In addition, the third step includes a process of automatically converting into a form suitable for the triple extraction apparatus and a process of manually converting the sentence into simple sentences through a semi-automatic interface.

In addition, in the third step, natural language interpretation such as display of noun clauses / phrases, adjective clauses / phrases, and the like as the minimum unit of the knowledge structure, display of the dependency relationship between words, and the like to assist the user in converting the sentence into a simple sentence by hand. It involves providing the user with all the information that can be analyzed using the processing device in the form of text or pictures. The natural language analysis processing apparatus includes an apparatus for performing natural language sentence analysis, pattern morphology analysis, and analysis through dependency grammar tree construction using a pattern.

In addition, the third step includes a process of providing a user with information for resolving the ambiguity of a natural language sentence, such as presenting a suggestion word corresponding to a pronoun, presenting an existing sentence, and a document of the related content. .

In addition, the third step is a method of connecting the information related to the word of the sentence and sentence in the process of any word in any sentence of any document through a line or other form of the figure and how it is related to the letter This includes providing the user with useful information in performing the third step.

In addition, the fourth step may include both a method of extracting triples using a pattern and a method of extracting using a dependent grammar tree.

In addition, the method using the pattern in the fourth step finds a pattern that matches the sentence to be processed by using a table of natural language sentence patterns and triple pairs corresponding thereto, and extracts triples corresponding thereto.

In addition, the method of extracting the dependency grammar tree in the fourth step includes obtaining a dependency grammar tree using a parser and then extracting a triple by applying some rules.

The method also includes a method of mapping the subject and object portions of the triple to the class of the existing machine-readable knowledge structure and the predicate portion of the triple to the predicate of the existing machine-readable knowledge structure. Then, using a machine-readable electronic dictionary such as WordNet or a web search engine such as the Google search engine, find the subject of the triple subject or object, and use the information to convert the subject or object into a class of existing machine-readable knowledge structure. And calculating and ranking similarity considering the surrounding structure and providing the same to the user. Predicates likewise include how to rank and provide them to the user.

According to the apparatus for attaching semantic information to an electronic document according to the present invention as described above, and a method for generating a machine-readable knowledge structure document from an electronic document set using the same, a natural language sentence can be easily read in a machine-readable form. This makes it easier to build knowledge-based web applications because computer programs can read many existing documents on the Web written in natural language.

In addition, the system and method for attaching semantic information to an electronic document according to the present invention as described above partially automates the process of annotating natural language sentences, thereby reducing the number of times a user must view the natural language sentences one by one, thereby reducing the annotation time. give.

In addition, the system and method for attaching semantic information to an electronic document according to the present invention as described above partially automates the process of annotating natural language sentences, thereby reducing the subjective intervention of the user, thereby obtaining a relatively consistent annotation output.

In addition, the system and method for attaching semantic information to the electronic document according to the present invention as described above, by leaving a record by dividing all subjective judgments used by the user in the process of annotating by type, the electronic document according to the present invention It is not only used to improve the internal workflow of the device that attaches semantic information to the data, but also as a resource to prepare guidelines for other users to perform the same tasks consistently.

In addition, the system and method for attaching semantic information to an electronic document according to the present invention as described above is already annotated by mapping the annotation output to semantic information described by an existing machine-readable knowledge structure. Natural language sentences described as semantic information can be found to not need to be annotated again, thus reducing annotation time.

Hereinafter, with reference to the accompanying drawings will be described an embodiment of the present invention; First, it should be noted that the same components or parts in the drawings represent the same reference numerals as much as possible. In describing the present invention, detailed descriptions of related well-known functions or configurations are omitted in order not to obscure the gist of the present invention.

1 is a block diagram conceptually illustrating an apparatus for attaching semantic information to an electronic document according to the present invention.

As shown in FIG. 1, an apparatus for attaching semantic information to an electronic document according to the present invention includes an electronic document collection device 5, an electronic document set analysis device 6, a sentence extraction device 7, and a simple sentence. It comprises a summarization apparatus 8, a triple extraction apparatus 9, a triple mapping apparatus 11, and a machine-readable knowledge structure output apparatus 12.

The electronic document collecting device 5 is inputted through various methods. When input through the web (1), the web document is collected through a web document search engine such as Google. In this case, the user may be prompted to input a search keyword. In addition, the machine-readable knowledge structure database 14 may be analyzed to automatically generate keywords related to the insufficient contents using a machine-readable electronic dictionary such as WordNet, and then collect the documents through a web document search engine. In the case of input through the database 2, the user collects the target document in advance and stores it in the database 2. The case where the electronic document is input through the electronic document input device 3 is when a user directly creates an electronic document using the electronic document input device 3 such as a keyboard of a desktop computer, a tablet input device, or an input device of a mobile device. . In the case of input through the optical reader 4, the electronic document is read using the optical reader 4 to generate an electronic document.

The electronic document set analyzing apparatus 6 compares and analyzes the input or collected electronic document set with the existing machine-readable knowledge structure database 14 to separately display documents or sentences that are already machine-readable knowledge structure. Analyze the degree to which words in a given electronic document appear in the machine-readable knowledge structure database 14, or classify a given electronic document into similar documents according to its contents, and then represent its words in the machine-readable knowledge structure database 14 ) Can be analyzed. In the analysis, when the electronic document is a web document, link information between the web documents may be utilized. If there is an electronic dictionary for words in the electronic document, the similarity between the words may be determined by using information such as upper words, lower words, and similar words about the words provided by the electronic dictionary. Combining this information makes it possible to distinguish between a statement describing the content already described in the machine-readable knowledge structure database 14 and a statement describing the new content. The number is counted and the electronic documents are ranked according to their respective scales and displayed to the user. The user selects one of the electronic documents and proceeds to the next step.

The sentence extracting apparatus 7 extracts sentences from the electronic document selected by the user. Since electronic documents can contain various data such as pictures and music as well as texts, it is necessary to extract only natural language sentences targeted by the system. At this time, there is a method of dividing by a sentence terminator (.!?) And the like, and includes a method using a machine learning technique.

The simple sentence device 8 separates sentences into simple sentences and replaces pronouns with original words. If the sentence is complex, it is necessary to separate it into a simple sentence because the natural language sentence parsing device 10 is not easy to interpret and process, and the pattern-triple mapping table 13 is also complicated and difficult to define the pattern. There may be rules such as separating sentences based on clauses or phrases or generating words in special symbols as separate sentences. The simple sentence device 8 presents this information to the user and the user refers to it and divides it into simple sentences.

The triple extraction device 9 is a device for extracting triples from sentences. When using an RDF triple, the triple <dog> <is_a> <animal> is extracted from the sentence "Dog is an animal." In this case, a triple may be determined by analyzing a dependency relationship between words using a natural language sentence syntax analysis device, or a triple may be determined by applying a predefined pattern.

The triple mapping device 11 is a device for determining which words in each of the triples are associated with the machine-readable knowledge structure database 14. For example, the knowledge <lecture> <is_taught_by> <professor>, <student> <attend> <lecture> is described in the machine-readable knowledge structure database 14, and the triple is defined in <Prof. If Kim> <teach> <Database>, <Prof. Kim> is associated with <professor>, <teach> with <is_taught_by>, and <Database> with <lecture>. This may be determined by the similarity between words, or by providing additional information to the triple to determine similarity by analyzing a portion of the structure in the triple and machine readable knowledge structure database 14 with the additional information added.

The machine-readable knowledge structure output device 12 adds a triple mapped to the machine-readable knowledge structure database 14 through the above process and outputs it as a file or on the screen in the form of a graph. The mapped triple is updated in the machine-readable knowledge structure database 14, so that if the same sentence exists in another electronic document, the electronic document set analyzing apparatus 6 can indicate that the content is already in the sentence, and thus the same content. Will not work twice.

Electronic document input / collection step through the electronic document collection device 5 as described above, electronic document selection step through the electronic document set analysis device 6, sentence extraction and sentence selection step by the sentence extraction device 7, simple Simple sentence generation step through sentence device 8, triple extraction step through triple extraction device 9, triple mapping step through triple mapping device 11, machine through machine readable knowledge structure output device 12 In the process of outputting the readable knowledge structure, etc., the user's subjective judgment may be input in a log form. This log can be stored in a database for analysis and use to improve the system.

1 is a block diagram conceptually illustrating an apparatus for attaching semantic information to an electronic document according to the present invention.

Claims (23)

An electronic document collecting device for collecting electronic documents on a web meeting a user's information needs; An electronic document set analyzing apparatus for selecting an electronic document by analyzing electronic documents in the collected electronic document set or an electronic document set provided by a user; A sentence extracting device extracting a natural language sentence from the selected electronic document; A simple sentence device for separating the sentence into simple sentences; A triple extraction device for extracting triples from the simple sentences; A triple mapping device for mapping each part of the triple to an existing machine-readable knowledge structure; A machine-readable knowledge structure document output device for outputting a machine-readable knowledge structure as an electronic document from the triples and mapping information; An error log input device for allowing a user to leave a record of what has been determined; And And attaching meaning to an electronic document including an error log analysis device that analyzes and records statistics of the user. The method of claim 1, The electronic document consists of an HTML document, a Wikipedia document, a Portable Document Format (PDF) document, a Microsoft Word document, a Korean document, an electronic document made using an optical reader, and an electronic document composed of all natural language sentences that can be processed by the machine. A system for attaching meaning to an electronic document comprising a. 3. The method of claim 2, The natural language attaches meaning to an electronic document including all languages that can be recorded as an electronic document, including English, Korean, Japanese, Chinese, German, and French. The method of claim 1, And the electronic document set attaches meaning to an electronic document containing link information between the documents. The method of claim 1, The language describing the machine-readable knowledge structure includes all languages in a form in which an interpretation processing device exists, such as OWL (Web Ontology Language) and KIF (Knowledge Interchange Format), so that the machine can interpret and process the meaning. A system for attaching meaning to electronic documents. The method of claim 1, The triple maps to a machine readable knowledge structure language, including RDF (Resource Description Framework) triples, to extend all existing machine readable knowledge structures or to generate all machine readable knowledge structures. A system for attaching meaning to an containing electronic document. The method of claim 1, The apparatus for attaching semantic information to the electronic document is a system for attaching meaning to an electronic document that can be used through a web browser by providing a web interface and can be produced and used in the form of a general application application. The method of claim 1, A device for attaching semantic information to the electronic document is a system for attaching meaning to an electronic document in which relevant information is stored only on the user's computer and used only on the computer or the information is stored on a server so that the information can be used over a network by several people. . The method of claim 1, The machine-readable knowledge structure document output device attaches meaning to an electronic document that can be output in a graph form on a screen and can be output as a file using a machine-readable language such as XML (Extensible Markup Language). system. A first step of analyzing a given set of electronic documents and selecting electronic documents most overlapping with a pre-generated machine readable knowledge structure document; Selecting a sentence among sentences in the selected electronic document; A third step of dividing the selected sentence into simple sentences using a simple sentence device; A fourth step of extracting triples from the simple sentence using a triple extracting device; A fifth step of mapping each part of the extracted triple to an existing machine-readable semantic structure using a triple mapping device; And And a sixth step of outputting a machine-readable knowledge structure document using the extracted triples and the mapping information by using a machine-readable knowledge structure document output device. The method of claim 10, The overlapping degree of the machine-readable knowledge structure document and the electronic document previously generated by the electronic document set analyzing apparatus is proportional to the frequency of occurrence of the contents related to the words in the previously generated knowledge structure in the given electronic document. How to attach semantic information. The method of claim 10, The degree of overlap between the machine-readable knowledge structure document and the given electronic document previously generated by the electronic document set analyzing apparatus is classified into similar documents, and the contents related to the representative word appear in the previously generated knowledge structure. A method of attaching semantic information to an electronic document so that it is proportional. The method of claim 10, The third step is a method of attaching the semantic information to the electronic document comprising the step of automatically converting into a form suitable for the triple extraction device and a semi-automatic conversion by the user manually into a simple sentence. The method of claim 10, In the third step, a natural language includes a phrase / phrase display such as a noun clause / phrase, an adjective clause / phrase, a word display as a minimum unit of the knowledge structure, and a dependency relationship between words to help the user convert a simple sentence into a manual sentence. A method of attaching semantic information to an electronic document that includes providing the user in the form of a text or a picture with all information that can be analyzed using an interpretation processing device. The method of claim 10, In the third step, the natural language interpretation processing apparatus attaches semantic information to an electronic document, including performing a natural language sentence analysis using a pattern, a morpheme analysis, or an analysis using a dependent grammar tree. The method of claim 10, The third step includes a process of providing a user with information for resolving the ambiguity of a natural language sentence, such as presenting a suggestion word corresponding to a pronoun, presenting an existing sentence and related document in which the related content is described. How to attach semantic information to a document. The method of claim 10, The third step is to connect the information related to the word of the sentence and the sentence being processed in which word of which sentence in which document in a certain way including a line, and to indicate what relations are represented by letters. A method of attaching semantic information to an electronic document comprising providing information useful in performing the third step. The method of claim 10, In the fourth step, both a method of extracting triples using a pattern and a method of extracting using a dependent grammar tree are possible. The method of using a pattern includes a table of natural language sentence patterns and triple pairs corresponding thereto. To find a pattern that matches the sentence to be processed and extract the triple corresponding to it, and to extract it using the dependency grammar tree, use the parser to get the dependency grammar tree and apply some rules to extract the triple. How to attach semantic information to an electronic document. The method of claim 10, In the fifth step, the subject and object portions of the triple correspond to classes of the existing machine-readable knowledge structure, and the predicate portion of the triple is pre-kit of the existing machine-readable knowledge structure. A method of attaching semantic information to an electronic document, the method comprising a corresponding method. The method of claim 10, Uses machine-readable electronic dictionaries such as WordNet or web search engines such as the Google search engine to find the parent of a triple subject or object and then uses that information to classify the subject or object into a class of existing machine-readable knowledge structure and its surroundings. A method of attaching semantic information to an electronic document, comprising: calculating and ranking similarity in consideration of structure and ranking the same and providing the same to a user. The method of claim 10, A method of attaching semantic information to an electronic document, comprising the step of inputting an error log, thereby presenting the error log type to the user and the user selecting one of them to enter a comment. The method of claim 21, It includes an error log analysis step to provide the statistical values for the error log input as described above to quantify the subjective judgment of the user and to use it as data for improving other devices How to attach. The method of claim 21, The error log input step and the error log analysis step include all steps of user intervention (electronic document collection step, electronic document set analysis step, sentence extraction step, simple sentence step, triple extraction step, triple mapping step, machine readable type). A method of attaching semantic information to an electronic document including a process of inputting and analyzing an error log every step in conjunction with a knowledge structure document output step).

Images