JPH0350631A - Information extraction device - Google Patents

Abstract

PURPOSE:To improve the identity of information and facility for understanding a result by dividing an information extraction function into a function making information on a noun phrase into a local table and a function connecting local tables and using the character of correspondence to the connection function. CONSTITUTION:The word of modification structure from a syntax analysis part 1 is replaced with the groups of concept and classification in a table generation part 2 and a participial adjective and a pronoun are replaced with concept so as to treat them as nouns. Then, concept is connected by using a knowledge relation by fields and the local tables are generated. A table connection part 3 sequentially selects the local tables from the head of a document and checks the possibility of a corresponding relation with all the local tables which are previously generated by using a corresponding relation check rule. Then, one table is selected by using the rule of a corresponding relation selection rule base 9 and the tables having the corresponding relation by using the rule of a table connection rule base 10. Thus, the identity of information and facility for understanding the result improve.

Description

DETAILED DESCRIPTION OF THE INVENTION (Field of Industrial Application) The present invention relates to a device that automatically extracts information from an electronic document and outputs information in a tabular format.

(Conventional technology) Conventionally, software that automatically extracts information from documents and displays it in a table format includes "Automatic Extraction of Related Information from Technical Abstracts," by Mr. Takamatsu, Mr. Kusaka, and Mr. Nishida, Information. There is something disclosed in Processing Society Transactions, Vol. 25 No. 2. Published March 1984, P. 216-P. 224J. It was intended to be converted into a tabular format.

Another method is to embed the words of the document in a pre-prepared table at hoc.

(Problem to be Solved by the Invention) However, although the above method for converting the dependency structure to alternating current has field transferability and expandability, it does not perform processing to link information extracted from different sentences, so I couldn't put all the information together. In addition, since tables are generated using only the information in a sentence, various tables are generated depending on how the sentence is expressed, making it difficult to understand, and in some cases, it may not be possible to use only the information in a single sentence. There was a problem that the table could not be generated.

In addition, although some of the above adhoc methods have partially solved these problems, they have strong restrictions on sentence structure and content (
, since it was created procedurally, it was not easy to improve, and there were problems in that it was not transferable or extensible to different fields.

This invention first solves the problems of information homogeneity and understandability of results while satisfying the portability and extensibility of the above-mentioned method using the dependency structure. It is an object of the present invention to provide an excellent apparatus for extracting information, which solves the above-mentioned problems by automatically generating tables and then combining the tables through anaphora processing.

(Means for Solving the Problems) In order to solve the above-mentioned problems, the present invention includes a syntax analysis unit that converts an input document into a dependency structure using cases;
A concept conversion dictionary for nominal language that describes a list of concepts and concept classification pairs for nominal words, a concept conversion dictionary for predicates that describes the target concepts that the main case can take, and a concept conversion dictionary for nominal language that describes the target concepts that the main case can take. A field-specific knowledge/base in which concepts are connected in a network to show the relationship between the two, and an abbreviation supplementary rule base in which omitted supplementary rules are registered to supplement omitted concepts. Nominal concept conversion dictionary,
Based on the concept conversion dictionary for pragmatics, field-specific knowledge, and rules for supplementary abbreviations, - a table generation unit that generates local tables using information in sentences, and all tables generated by this table generation unit; An anaphora relationship check rule base in which anaphora relationship check rules are registered to check the possibility of anaphora relationships, and an anaphora relationship selection rule that uses the linguistic properties of anaphora to select one table. The anaphora relationship rule base in which anaphora relationships are registered, the table join rule base in which table join rules for joining tables that have anaphora relationships are registered, anaphora relationship check rules, anaphora relationship selection rules, and table A feature of the document is that it includes a table join unit that joins a local table with a preceding local table in order from the beginning of the document based on a join rule.

(Operation) According to the present invention having the above configuration, an input document is converted by the syntax analysis unit into a dependency structure using cases. Then, based on this dependency structure, the table generation unit generates a local table using the information in the -sentence, based on the nominal concept conversion dictionary, the predicate concept conversion dictionary, field-specific knowledge, and omission supplementary rules. . The table join part determines the identity of local tables based on anaphora relationship check rules, anaphora relationship selection rules, and table join rules, and sequentially connects local tables to the previous local tables from the beginning of the document. join with other tables. At this time, information that is ambiguous within one sentence may become complete information by joining tables. Furthermore, information extraction for each sentence can be performed in the same way as the conventional system. Furthermore, field transitions can be easily handled by modifying field-specific knowledge and concept conversion dictionaries.

Therefore, the present invention can solve the above-mentioned problems, and can provide an information extraction device that can improve the sameness of information and the ease of understanding of results while satisfying portability and expandability.

(Example) Hereinafter, one embodiment of the present invention will be described based on the drawings.

FIG. 1 is a system configuration diagram showing an embodiment of the present invention. In the figure, 1 is a syntax analysis unit that converts an input document into a dependency structure using cases as shown in Figure 2;
2 is a table generator that generates a table (hereinafter referred to as a "local table") using information in one sentence; 3 is a table generator that generates local tables sequentially from the beginning of the document before it. 4 is a list of pairs of concepts and concept classifications for words expressing concepts that are not conjugated, such as nouns, pronouns, and numerals. 5 is a concept conversion dictionary for nominals in which concepts that may be included in the main case noun phrase are described.
6 indicates the relationship between concepts as shown in Figure 3, and the knowledge base by field in which the concepts are connected in a network, and 7 indicates the omitted supplementary rule as shown in Figure 4, for example. 8 is a registered omitted supplementary rule base, 8 is a anaphoric relationship check rule base in which anaphoric relationship check rules as shown in FIG. 5 are registered, and 9 is an anaphoric relationship as shown in FIG. 6, for example. The anaphora selection rule base in which selection rules are registered, IO, is a table join rule base in which table join rules as shown in FIG. 7 are stored, for example. Here, FIG. 8 is a diagram showing the format of the table that is output in this embodiment. 9th
The figure shows the format of a local table. In FIG. 9, local tables are assigned numbers called identifiers in the order in which they are generated. FIG. 1O is a diagram showing the classification of concepts. Common symbols given to words that are different in notation, such as synonyms and different notations, but should be interpreted as having the same meaning, are hereinafter referred to as "concepts." Concepts are categorized based on where they fit into the table. Hereinafter, concepts classified as object, attribute, and attribute value will be referred to as "object concept,""attributeconcept," and "attribute value concept," respectively. FIG. 11 is a diagram showing formats and examples of a nominal concept conversion dictionary and a predicate concept conversion dictionary for converting words in a dependency structure into concepts.

Next, we will describe the procedure for generating a local table from a document using information within one sentence.

In the nominal language concept conversion dictionary 4 shown in FIG. 1, a list of concepts and concept classification pairs is described for nominal words.

For demonstrative words such as pronouns, the highest-ranking concept that does not include lower-level concepts that cannot be denotative is written from among the concepts of field-specific knowledge concepts. The target concepts that the main case can take are described.

Multiple target concepts can be described with conditions for selection. The field-specific knowledge 6 shows the relationship between concepts, and the concepts are connected in a network. In addition, field-specific knowledge is an extension of a thesaurus, and while a thesaurus classifies concepts only by upper and lower ranks, field-specific knowledge uses the above-mentioned relationships and can have two or more higher-level concepts. The table generation unit 2 in FIG. 1 first replaces words in a dependency structure with pairs of concepts and categories. Adnominals and pronouns are also replaced with concepts and treated in the same way as nouns.

For words that are not registered in the dictionary, the words are used as they are as target concepts. However, the concept conversion dictionary described above is equipped with an unknown word processing function, and even if the words are not registered in the dictionary, proper nouns consisting of the alphabet or words representing monetary amounts are treated as attribute value concepts. Then, concepts are connected using relationships in field-specific knowledge to generate local tables. FIG. 12 shows an example of local table generation. When a plurality of tables are generated from one noun phrase, the tables within the same noun phrase are joined prior to the table joining process described below. At this time, if the target concepts have one of the relationships shown in Figure 13, the tables are joined using that relationship, and if there is no relationship between the target concepts, the modifying relationships of the dependency structure are used. Join tables as a relationship between tables. In generating the table, omitted concepts are supplemented using field-specific knowledge, a concept conversion dictionary for terminology, and omitted supplementary rules.

In the manner described above, a local table is generated from all sentences of the document.

Next, we will describe the process of joining local tables to form a final table.

The table joining unit 3 in Figure 1 joins local tables with previous local tables in order from the beginning of the document (hereinafter, the relationship between local tables is referred to as an ``anaphora relationship''). In addition, various rules shall be applied in order from the top until the processing is completed.Select local tables in order from the first sentence of the document, and perform the anaphora relationship check in Rule Base 8 for anaphora relationship check.・Use rules to check the possibility of anaphora relationships with all previously generated local tables.However, if local tables are already joined, check them all at once. Therefore, the number of local tables to be checked does not necessarily increase linearly as the number of words increases.

Record all identifiers of tables that are found to have a possibility of anaphora relationship during the check. In the above anaphora check, since there are usually multiple tables that may have an anaphora relationship, the anaphora relationship selection rule of Base 9 is used to select from among the tables that may have an anaphora relationship. Te, 1
Select one table.

Anaphoric relation selection rules utilize the linguistic properties of anaphors. The table join rules of the table join rule base 10 are used to join tables that have anaphora relationships. In the example in Figure 7, even if there is a superior-subordinate relationship with one table and a whole-part relationship with another table, if one of them is determined, the result shown in note) The result of the table join rule processing is the same as joining with both relationships.

FIG. 14 is a diagram showing an example of a combined table obtained after the above processing. As a result of the table join process, tables are grouped into fewer tables that have the same identifier or are linked to each other. Among them, all tables excluding tables with only headings and no items or values are output.

(Effects of the Invention) As described above, according to the present invention, the information extraction function is divided into the function of converting noun phrase information into a local table and the function of combining local tables, and furthermore, the By using the properties of anaphora, the following effects can be expected.

■-Bunkoen has ambiguous information, but due to table join processing,
It is clarified because it is connected to information in other sentences.

■Results are not affected by sentence structure and are easy to understand.

■When transferring to a field, there is no need to change procedures, and only the field-specific knowledge and concept conversion dictionary need to be changed, making it field-transferable.

■Since the property of linguistic correspondence is used to determine the identity of information, it is highly accurate and scalable.

[Brief explanation of drawings]

Figure 1 is a block diagram showing the system configuration of an embodiment of the present invention, Figure 2 is a diagram showing an example of a dependency structure, Figure 3 is a diagram showing an example of field-specific knowledge, and Figure 4 is omitted. Figure 5 is a diagram showing an example of a rule, Figure 5 is a diagram showing an example of a comparison relation check rule, Figure 6 is a diagram showing an example of a comparison relation selection rule, Figure 7 is a diagram showing an example of a table join rule, Figure 8 shows the format of the output table in this embodiment, Figure 9 shows the format of the local table, Figure 1O shows the concept classification, and Figure 11 shows the concept conversion dictionary. Figure 12 shows an example of local table generation, Figure 13 shows relationships between field-specific knowledge, and Figure 14 shows an example of a combined table. . 1 ・ 2 ・ 3 ・ 4 ・ 5 ・ 6 ・ 7 ・ 8 ・ 9 ・ IO・ Syntax analysis section, table generation section, table combination section, concept conversion dictionary for nominals, concept conversion dictionary for predicates, field-specific knowledge/base , Omitted Supplementary Rule Base, Anaphora Check Rule Base, Anaphora Selection Rule Base, Table Join Rule Base. The main force and ζ 4 clothes 0 form Takeshi Riju I Af 12 Scarce - 9 Iji・j Figure 2 Kotan 11th Order,) Inside 122] Grass 3 and now gt monument, a Fig. 2 - J of the Ministry of Fire and Fire Prevention Rules - J Fig. 4 M,...; ! ! , <&'14'g 'J! y r ray−1,=
=, /FJ fig. Takumi--Sen Takumi kS Kunimuki Buntan 1'! l<,'l'Mr”f ”WuζI9S 1st Fig. 9 Zaibetsu Shumeishiki゛9 Intercrop Fig. 3

Claims (1)

[Scope of Claims] A syntactic analysis unit that converts an input document into a dependency structure using cases, and a concept for nominal language in which a list of pairs of concepts and concept classifications is described for words in nominal language. A conversion dictionary, a pragmatic concept conversion dictionary that describes the concepts that the main cases can take, and a field-specific knowledge/base that shows the relationships between concepts and connects them in a network. , an abbreviation supplementary rule base in which omitted supplementary rules for supplementing omitted concepts are registered, the aforementioned concept conversion dictionary for nominal language, the aforementioned concept conversion dictionary for predicates, the field-specific knowledge and the omitted supplementary rules; a table generator that generates a local table about the contents of a sentence based on the information in a sentence, and a anaphora relation check to check the possibility of local tables generated by the table generator・An anaphora relationship check rule base in which rules are registered, and an anaphora relationship selection rule that uses the linguistic properties of anaphora to select one table from among multiple join possibilities. The anaphora selection rule base that has anaphora relations, and the table join rule base that has registered table join rules for rewriting tables that have anaphora relationships and joining them into one table.
a table joiner that joins a local table with a previous local table in order from the beginning of the document based on the anaphora relationship check rule, the anaphora relationship selection rule, and the table join rule; An information extraction device comprising: