JP5317922B2 - Information extraction rule creation support system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a technique that can support so that general users can easily create information extraction rules for use in extracting company information having a semantic structure of company names-company activities-subjects of activities from natural sentences. <P>SOLUTION: A support system 10 includes a dictionary storage part 22; a morphological analysis process part 12; a subject identifying process part 14; a parsing process part 16; an abstraction process part 18 for imparting an abstraction tag to each segment and storing the segments in a material sentence storage part 26; and an information extraction rule creation part 20 which sends a rule editing screen 40 with a plurality of material sentences shown thereon to a client terminal 30 and urges selection of material sentences describing the company activities and the subjects of activities of a particular company, and which, when material sentence selection information is transmitted, takes out a modification structure or the like between segments including the company names, segments including the company activities, and segments including the subjects of activities in the material sentences to create information extraction rules 52 described in accordance with a predetermined format for storage in an information extraction rule storage part 28. <P>COPYRIGHT: (C)2011,JPO&amp;INPIT

Description

  The present invention relates to an information extraction rule creation support system, and in particular, an information extraction rule used when extracting an information element having a semantic structure from a natural sentence using a parsing technique, The present invention relates to a technology that supports a user so that it can be easily created.

  Parsing technology is used to extract a set of information elements with a semantic structure from unstructured text data written in natural language, such as web pages published on news sites on the Internet. Various uses have been proposed.

  For example, in the case of the information extraction device described in Patent Document 1, a character string in a document described in a natural language is sequentially compared with a predetermined character pattern, and the type of proper noun is determined for a character string portion in which a match is recognized. A character pattern processing unit that provides tag information indicating morphological analysis, a morpheme analysis processing unit that sequentially divides other character string portions excluding tag information into word information, and a word obtained as a result of morphological analysis A syntactic analysis unit that compiles information into clauses, and parses the word information after the compilation using a grammatical syntax rule and a syntactic pattern that appears characteristically in the expression of certain information, and the above syntax An information extraction unit is provided for extracting, as necessary information, dependency relationships obtained by analysis based on patterns and tag information included in the dependency relationships.

By using this conventional information extraction device, for example, “At around 5:00 am on the 5th, Chuo-cho, Osaka city, disinfection industry, Mr. Isamu Suzuki (50) broke out, and about 125 square meters of wooden one-storied houses were burnt down. ”Can be extracted from the sentence“ <person name> Isamu Suzuki><placename> Chuo-cho, Osaka city | <industry name> disinfection industry ”.
JP-A-11-272695

  However, in the case of the information extraction method based on the syntax analysis process in this way, in order to accurately extract necessary information, it is necessary to prepare in advance a large number of syntax patterns for designating morphemes to be extracted. was there.

  For example, in the sentence “Toyo Motor Company's transmission manufacturer Pylon has announced that it will produce a continuously variable transmission, which is a highly fuel efficient transmission, in Guangzhou, China.” ), A plurality of objects (transmission, continuously variable transmission), and a plurality of activities (production, announcement) are included in form. Therefore, when the true activity subject (subject), activity target (object), and activity content (predicate) are extracted from this sentence, as shown in FIG. After clarifying the receiving structure and referring to various dictionaries and adding tags (<company name>, etc.) indicating the type to each phrase, as shown in FIG. 18, the type of morpheme to be extracted and the positional relationship in the sentence It is necessary to prepare the defined syntax pattern A.

  By applying this syntax pattern A to the syntax analysis result of FIG. 17, the clause shown in FIG. 19 (a) is extracted from the sentence, and by applying the necessary shaping processing to this, as shown in FIG. 19 (b). In addition, structured company information consisting of a combination of subject, predicate and object is obtained.

  Assuming that the sentence is expressed as “Toyo Motors subsidiary transmission manufacturer Pylon produces continuously variable transmissions in Guangzhou, China”, as shown in FIG. In addition, it is necessary to prepare another syntax pattern B.

  Furthermore, in the case of natural sentences, inverted expressions may be used for rhetorical purposes, but in order to extract necessary information from such sentences, syntactic patterns based on inverted expressions are preliminarily used. It was necessary to prepare a large number.

  In this way, natural sentences have a complex and diverse structure, so even if a sentence that has been parsed in advance is extracted, information with a specific semantic structure can be extracted from it. It is essential to prepare a large number of information extraction rules (syntax patterns) in order to accurately extract the information. To that end, it was necessary to mobilize a large number of specialists who have knowledge of language analysis.

  The present invention has been devised in order to solve such a conventional problem, and an information extraction rule used when extracting information having a specific semantic structure from a parsed text, The purpose is to provide a technology that can be easily supported by general users who do not have specialized knowledge.

  In order to achieve the above object, an information extraction rule creation support system according to claim 1 includes a dictionary in which a correspondence between a specific expression character string and an abstract character string indicating the type is registered, a text Means for decomposing each sentence in the data into morpheme units and means for referring to the above dictionary and associating corresponding abstract tags with those corresponding to the company name, company activity, and activity target in each morpheme; , Means for searching for the company name subject in each sentence, means for storing a sentence with the company name subject in the material sentence storage means as material sentences, and taking out the material sentences from the material sentence storage means, Means for generating a rule editing screen by filling a template, means for transmitting the rule editing screen to a client terminal, prompting selection of a material sentence in which a company activity and an activity target of a specific company are described, and a client When the selection information of a specific material sentence is transmitted from the mobile terminal, the clause including the company name in the material sentence, the clause including the corporate activity and the dependency structure between the clauses including the activity target, or these company names Dependent structure between clauses containing clauses, clauses containing business activities and clauses containing activity objects, and clauses containing character strings other than company names, business activities, and activity targets are extracted as extraction conditions, and this is formatted in a predetermined format. The information extraction rule generating means for generating the information extraction rule described according to the above and the means for storing the information extraction rule in the information extraction rule storage means are provided.

  These information extraction rules are applied as information extraction conditions when generating structured company information from natural sentences. In other words, morphological analysis and syntactic analysis are performed on the target natural sentence, the company name, the company activity, and the abstract tag of the activity target are added to the corresponding phrase by referring to the dictionary, and each information extraction rule is set. When matching is performed, the character string at the position defined in the information extraction rule is extracted as the company name, the company activity, and the activity target.

  The information extraction rule creation support system according to claim 2 is the system according to claim 1, and each material sentence in the rule editing screen includes a character string portion indicating a company name and characters indicating a company activity. It is characterized in that markings (character decorations such as shading and coloring) for identifying the respective types are applied to the line part and the character string part indicating the activity target.

  The information extraction rule creation support system according to claim 3 is the system according to claim 1 or 2, and further refers to the dictionary and corresponds to a morpheme corresponding to a place. Means for associating an abstract tag, and the information extraction rule generating means includes a morpheme associated with a location abstract tag in the material sentence when selection information of a specific material sentence is transmitted from the client terminal. In this case, a dependency structure between a clause including a morpheme indicating the place and another clause is taken out as an arbitrary extraction condition, and an information extraction rule including this is generated.

  The information extraction rule creation support system according to claim 4 is the system according to claims 1 to 3, and further, when there is no subject in the sentence, the company name subject in the sentence located before the sentence is If there is a sentence that has a subject other than the company name subject and a morpheme other than the company name between the sentence and the sentence in which the company name subject exists, the company name subject It is characterized in that it has means for associating as a company name subject of a sentence in which the subject does not exist.

  The information extraction rule creation support system according to claim 5 is the system according to claims 1 to 4, and further, when a corporate pronoun is included as a subject in the sentence, the sentence positioned before the sentence If there is a company name subject and there is no sentence with a subject other than the company name subject and a morpheme other than the company name between the sentence and the sentence in which the company name subject exists, It has a feature that replaces the company pronoun with the company name subject to the company name.

  The information extraction rule creation support system according to claim 6 is the system according to claims 1 to 5, and further applies a regular expression rule set in advance to each sentence and matches the regular expression rule. The morpheme is identified as any one of a company name, a company activity, and an activity object, and a means for associating a corresponding abstract tag with the morpheme is provided.

  The information extraction rule creation support system according to claim 7 is the system according to claims 1 to 6, further applying a predetermined dependency rule to each sentence and matching the dependency rule. The morpheme of the clause to be identified is any one of a company name, a company activity, and an activity object, and a corresponding abstract tag is associated with the morpheme.

  Since the natural sentence has a complicated and ambiguous structure, the selection of the material sentence that should be the basis of the information extraction rule generation must be relied on the human sense, but the information extraction rule according to claim 1 According to the creation support system, the user determines the semantic content of each material sentence on the rule editing screen displayed on the client terminal, and the material sentence that substantially describes the business activity and activity target of the specific company. Since the system automatically processes the generation of material sentences and information extraction rules, information extraction rules can be easily created even for users who do not have expertise in language analysis. It becomes possible to do.

  In the case of the information extraction rule creation support system according to claim 2, since the character string portion indicating the company name, the character string portion indicating the company activity, and the character string portion indicating the activity target are marked, Makes it possible to recognize the checkpoints included in the material sentence at a glance, and to achieve efficient selection work.

As described above, company information consisting of a combination of company name, company activity, and activity object has the following format: subject (company name) → predicate (company activity) → object (activity object). It can be said that it expresses a series of semantic structures. However, it is possible to further enhance the value and effectiveness of corporate information by incorporating a location element into this.
According to the information extraction rule creation support system according to the third aspect, it is possible to create an information extraction rule that can extract a place element related to a corporate activity from a natural sentence.

  According to the information extraction rule making support system described in claim 4, even if a subject does not contain a subject, the subject name subject is contained in a sentence positioned before that. In this case, it is presumed that the description of the company name is omitted, and it is recognized as a company name subject of a sentence without a subject. As a result, it is possible to effectively use a sentence that does not contain a subject as a material sentence.

  According to the information extraction rule making support system described in claim 5, for a sentence including a pronoun such as “Company” representing a company as a subject, it indicates a company name subject of a sentence positioned before that. The company pronoun is replaced by the company name subject to this company name. As a result, it is possible to effectively use a sentence whose subject is a corporate pronoun as a material sentence.

  According to the information extraction rule creation support system described in claims 6 and 7, it is possible to associate each morpheme with a business name, business activity, and an abstract tag of an activity object required on a rule basis, and on a dictionary basis. It is possible to complement the abstraction.

FIG. 1 is a block diagram showing the overall configuration of an information extraction rule creation support system 10 according to the present invention. The morpheme analysis processing unit 12, the subject identification processing unit 14, the syntax analysis processing unit 16, and the abstraction processing unit 18, an information extraction rule generation unit 20, a dictionary storage unit 22, an abstraction rule storage unit 24, a material sentence storage unit 26, and an information extraction rule storage unit 28.
A client terminal 30 operated by a user is connected to the information extraction rule generation unit 20 via a communication network.

The morpheme analysis processing unit 12, the subject identification processing unit 14, the syntax analysis processing unit 16, the abstraction processing unit 18 and the information extraction rule generation unit 20 execute necessary processing according to the OS and application program by the computer CPU. Is realized.
The dictionary storage unit 22, the abstraction rule storage unit 24, the material sentence storage unit 26, and the information extraction rule storage unit 28 are provided in the hard disk of the computer.

  The dictionary storage unit 22 stores a company name dictionary, a company activity dictionary, an activity target dictionary, a person name dictionary, a country name dictionary, a region name dictionary, a prefecture name dictionary, a municipality name dictionary, an animal and plant name dictionary, a synonym dictionary, and the like. Has been.

  FIG. 2 exemplifies the registered contents of the corporate activity dictionary, and “production”, “manufacturing”, “processing” are performed on an abstract character string of a high-level conceptual “production activity” that is a type of corporate activity. ], Specific expression character strings to be predicates such as “Assembly” are associated in advance. Similarly, for the abstract string of high-level conceptual “sales activities” that is a type of corporate activity, there are specific expression strings that should be predicates such as “sales”, “release”, “sell”, etc. Corresponding in advance. Instead of “production activities” and “sales activities”, a higher-level conceptual “business activity” abstract character string may be used as a group.

FIG. 3 exemplifies the registered contents of the activity target dictionary, and “liquid crystal”, “liquid crystal television”, “liquid crystal panel”, “liquid crystal” are used for the abstract conceptual character string “production target”. Specific expression character strings to be objects such as “monitor”, “portable music player”, and “gigabyte” are associated in advance. In the same way, for the abstract string of “sale target”, which is a high-level concept, “liquid crystal”, “liquid crystal television”, “liquid crystal panel”, “liquid crystal monitor”, “portable music player”, “gigabyte”, etc. A specific expression character string to be an object is associated in advance. “General” in parentheses indicates a general noun, and “proprietary” indicates a proper noun.
Instead of “production object” and “sales object”, an abstract character string of a more conceptual “activity object” may be used.

  Although illustration is omitted, in the company name dictionary, a number of specific company names (formal names and abbreviations) to be the subject are registered in association with the abstract character string “company name”.

Next, the processing contents by the system 10 will be described.
First, the morpheme analysis processing unit 12 performs morpheme analysis on text data 32 such as a web file input from the outside. Here, “morpheme analysis” refers to a process of decomposing a sentence described in a natural language into morphemes, which are the smallest meaningful language units, and identifying each part of speech.

For example, when a sentence “Hatsushiba will release a new product of portable music player“ Gigabyte ”in Kansai on the 15th” is given, as shown in FIG. / Noun, proper noun "," ha / participant, collaborative particle "," portable music player / noun, general ", and specify each part of speech.
This morpheme analysis itself is a known technique. For example, the following free software can be used as a morpheme analysis engine.
(1) MeCab (http://mecab.sourceforge.net/)
(2) ChaSen (http://chasen.naist.jp/hiki/ChaSen/)

The morpheme analysis processing unit 12 can refer to the activity target dictionary stored in the dictionary storage unit 22 and reflect the registered contents in the result of the morpheme analysis.
For example, the word “portable music player” is subdivided into “mobile / noun, common connection”, “music / noun, general noun”, “player / noun, general noun” when put into a general morphological analysis engine. However, since “portable music player (general)” is registered as a production target and a sales target in the activity target dictionary, the morphological analysis processing unit 12 uses the combined word “portable music player” as one. Certified as a morpheme.
The word “gigabyte” is also subdivided into “giga / noun, general noun” and “byte / noun, general noun” according to a general morphological analysis engine. Since “Gigabyte (unique)” is registered as a production target and a sales target, the morpheme analysis processing unit 12 can recognize the combined word “Gigabyte” as one morpheme.

Next, the morpheme analysis processing unit 12 refers to the company name dictionary, the company activity dictionary, and the activity target dictionary stored in the dictionary storage unit 22, and supplements the abstract tag corresponding to the part of speech of the specific morpheme.
For example, for “Hatsushiba”, since there is a registered example in the company name dictionary, an abstract tag “company name” is added to the part of speech item. Further, since “portable music player” and “gigabyte” are registered as production targets and sales targets in the activity target dictionary, abstract tags of “production target” and “sale target” are added to the part of speech item. Since “Kansai” is registered as a place in the regional name dictionary, an abstract tag of “place” is added to the part of speech item. Furthermore, regarding “release”, since a registered example exists as a type of sales activity in the corporate activity dictionary, an abstract tag “sales activity” is added to the part of speech item.

Next, the subject identification processing unit 14 is activated, and the company name as the subject is identified in sentence units based on the data decomposed into morpheme units by the morpheme analysis processing unit 12.
Specifically, as shown in FIG. 5, immediately after the company name, "ga ~" (particle-case particle), "ha ~" (particle-indicative particle), or ", ~" (symbol-reading mark). When it is detected that the subsequent character string is present in the sentence, the subject identification processing unit 14 recognizes the company name as the subject.

In the sentences of (a) and (b) in FIG. 6, “Hatsushiba” and “Hatsushiba wa” exist, and therefore the company name “Hatsushiba” is recognized as the subject name of the company.
In addition, when a sentence that does not have a subject follows the sentence that identifies the company name subject, the subject identification processing unit 14 inherits the company name subject that has been identified earlier.
For example, the character string corresponding to the subject is not detected in the sentence of (c) of FIG. 6, but in this case, it is presumed that the company name subject appearing in the previous sentence is omitted, and “Hatsushiba” is Associated as the subject name of the company in the sentence.

Further, the subject identification processing unit 14 executes a process (anaphora process) for replacing a sentence whose main subject is “company”, which is a pronoun representing a company, with the subject name subject of the previous sentence.
For example, if there is a sentence “(New company's new product is a public wireless LAN service provided by Sofcom…)” in FIG. 6 (d), the company pronoun “Company” is Replaced by the company name “Hatsushiba” associated with the previous sentence.

The process of inheriting and responding to the company name subject continues until a sentence in which the existence of another company name subject is recognized appears. For example, in the sentence (x) in FIG. 6, the character string “Sally is” exists, and “Sally” corresponds to the company name. Therefore, the subject identification processing unit 14 uses the company name subject of the sentence (x). As “Sally”.
In addition, when a sentence that does not have a character string corresponding to the subject continues thereafter, the subject identification processing unit 14 associates “Sally” with these sentences as the subject name of the company.
Further, if a sentence having the subject as a business pronoun appears thereafter, the subject specifying processing unit 14 replaces the business pronoun with “Sally”.

Succession of a company name subject is also stopped when a sentence with a subject other than the company name (for example, a person name, country name, prefecture name, animal name, plant name, etc.) appears. As a premise, in the morpheme analysis process, identification tags of person name, country name, and prefecture name are added to each morpheme item.
However, since this system 10 is intended to create a rule for extracting information related to a company, a sentence whose subject is a morpheme other than the company name is excluded from processing.

Next, the dependency structure of each sentence is analyzed by the syntax analysis processing unit 16.
FIG. 7 is a table showing the analysis result, and stores the related phrase ID for each phrase. FIG. 8 is an image of this dependency structure.
This syntax analysis itself is a known technique. For example, the following free software can be used as the syntax analysis engine.
(1) KNP (http://nlp.kuee.kyoto-u.ac.jp/nl-resource/knp.html)
(2) CaboCha (http://chasen.org/~taku/software/cabocha/)

  Next, the abstraction processing unit 18 includes the character string (morpheme) corresponding to the company name, business activity (production activity, sales activity), activity target (production target, sales target, etc.) and location in each phrase unit. In the case where it is included, the corresponding abstract tag is associated with the clause as shown in FIG. 9B.

At the time of this abstraction, the morpheme analysis processing unit 12 refers to the dictionary DB 22 in advance, and since the assignment of abstract tags such as company names, sales activities, and sales targets has been completed in units of morphemes, first of all, The abstract tag associated with the morpheme unit is inherited as it is to the clause including the morpheme.
Specifically, as shown in FIG. 9 (a), a table with morpheme ID, morpheme, part of speech, and clause ID is generated on the memory, and the abstract tag added to the morpheme is transferred to the corresponding clause. The

  However, the number of words recorded in the dictionary is naturally limited, and there is a possibility that leakage will occur only with the dictionary-based abstraction processing. Therefore, the abstraction processing unit 18 uses the regular expression rule and the dependency rule. It has a function of executing abstraction processing and associating a corresponding abstract tag with a company name or activity target not recorded in the dictionary.

[Abstract processing by regular expression rule]
For example, if the expression "ABC is a new product" is present in the sentence, the part "ABC" is recognized as "Production object", and the phrase "Production object" appears in the phrase "ABC". Means to assign an abstract tag. Or, if the phrase “America Mart, a major retailer,” is present in the sentence, the part “AAA Mart” is recognized as “Company Name” and the phrase “Company Name” appears in the “AAA Mart” clause. This corresponds to assigning the abstract tag.
For this reason, the abstraction rule storage unit 24 stores a large number of abstraction rules in advance.

  (A) of FIG. 10 shows an example of an abstraction rule. “<Company_size> <country> (<feature: noun> +)” is “company_size (a character string representing a company size)” + “ It is defined that the noun immediately following "no" + "country (character string representing country)" is recognized as a company name. Further, “first, major, middle-ranking” is defined as an alias expression (alias) of “company_size”, and “US, UK, Europe” is defined as an alias expression of “company_size”.

  Here, as shown in FIG. 10 (b), if the sentence “America Mart, a major retailer has announced a personnel reduction plan,” is given, the abstraction processing unit 18 converts this to FIG. 10 (c). ) And replace it with the noun unit OR expression, extract the `` retail major US AAA mart '' that matches the rule, then use the regular expression `` backward reference '' to extract `` AAA mart '' Authorize.

[Abstract processing by dependency rules]
(1) Abstraction of company name This is a clause related to a clause that includes corporate activities.If the particle is `` ga '' or `` ha '', the rule that certifies the morpheme immediately before the particle as the company name. Say.
For example, in the example of FIG. 11, the phrase “US oratel is” is related to the phrase “shipping”, which means sales activity, and has the particle “ha”. The morpheme “Olatel” is recognized as the company name.
(2) Abstraction of Activity Targets This is a rule that recognizes a morpheme immediately before a particle as an activity target when the particle is a phrase related to a clause that includes corporate activities.
In the example of FIG. 11, the phrase “Core o7” is related to the phrase “shipping”, which means sales activities, and has the particle “ The morpheme “Core o7” is certified for sale.

  The sentence for which the abstraction processing has been completed is stored in the material sentence storage unit 26 by the abstraction processing unit 18. Each sentence stored in the material sentence storage unit 26 is used as a material for generating an information extraction rule.

  That is, when a request for creating an information extraction rule is transmitted from the client terminal 30 operated by the user, the information extraction rule generation unit 20 extracts a material sentence from the material sentence storage unit 26 according to a certain rule (for example, new arrival order), After the rule editing screen in which these are embedded is generated, it is transmitted to the client terminal 30.

FIG. 12 shows an example of the rule editing screen 40 displayed on the Web browser of the client terminal 30. In the sentence display field 42, a large number of structured texts are listed.
In addition, each character string portion corresponding to a company name, each character string portion corresponding to a company activity, and each character string portion corresponding to an activity target in each text is provided with a unique marking (decoration). When the subject name subject of the preceding sentence is inherited, the company name is displayed in parentheses, such as “(Hatsushiba)”, and is distinguished from the sentence in which the company name is clearly specified.
In the figure, an example is shown in which different patterns of shading are applied to the company name, company activity, activity target, and location, but marker display in different colors for each (for example, company name = blue, company Activity = yellowish green, activity target = orange, place = yellowish green, etc.).
As a result, the user can identify at a glance the company name, company activity, activity target, and place to be checked from the text.
When marking a character string, an abstract tag associated with each character string and company name information (company name inheritance information) associated with the sentence are referred to.

  In this case, since the “sales activity” tab 44 of the rule editing screen 40 is selected, the character string related to “sales activity” (for example, “release”) is marked in the corporate activity. When the tab is switched to the “production activity” tab 46, the character string related to “production” (for example, “manufacturing”) is marked, and the marking applied to “release” is cancelled.

  Similarly, when the “sales activity” tab 44 on the rule editing screen 40 is selected, the character string related to “sales object” (for example, “portable music player”) is marked among the activity objects. When the “production activity” tab 46 is switched, the character string related to “production target” is marked, and the marking applied to “sales target” is cancelled. However, there are many cases where the same character string is registered in both the production target and the sales target, and there is a case where no change is felt in appearance.

  On this rule editing screen 40, the user reads each text described in the sentence display field 42, and a sentence in which the subject (company name), predicate (sales activity), and object (sales target) are correctly aligned. If it finds, click on the markings on each character string.

  For example, the first sentence “[Hatsushiba], [moving music player] that can watch [movie] on the go” is a so-called title sentence, and the predicate is omitted, so the user can create an information extraction rule Exclude from the target sentence.

On the other hand, the second sentence “[Hatsushiba] [releases] a new product of [portable music player]“ [Gigabyte] ”in [Kansai] on the 15th” is the subject, predicate, purpose. Because the words are aligned correctly and the company name, sales target, sales activities, and location can be extracted effectively, the markings are [Hatsushiba], [Portable Music Player], [Gigabyte], [Kansai], [Release] Click the section.
As a result, the character string (Hatsushiba, portable music player, gigabyte, release, Kansai) selected by the user is displayed in the extraction node display field 48. In this state, when the user clicks the anchor text 50 of “Generate rule”, the information extraction rule generation unit 20 generates an information extraction rule.

FIG. 13 explains the mechanism of generating this information extraction rule. FIG. 13 (a) shows the dependency structure between clauses included in the sentence.
As shown in this figure, the phrase “Hatsushiba wa” including “Hatsushiba” selected by the user is related to the phrase “to be released” with an abstract tag of sales activity. "Gigabyte" clause, "Gigabyte" clause "New product" clause, "New product" clause "Launch" clause, "Kansai" The “de” clause is associated with the “release” clause.

  Therefore, the information extraction rule generation unit 20 uses <company name>, <business activity>, <sales target>, and <location> for clauses with abstract tags as shown in FIG. In this way, the clause is expressed by an abstract tag, and for the clause not attached with the abstract tag, the same character string is adopted as in (new product), and the dependency relationship between clauses is recognized. At this time, since the same abstract tag is continuous between the phrases “portable music player” and ““ gigabyte ”” as “sale object → sale object”, they are combined into one abstract tag.

Next, the information extraction rule generation unit 20 generates the information extraction rule 52 according to a predetermined notation based on the dependency relationship of (b) as shown in FIG.
In this information extraction rule 52, first, as “extraction (extraction target)”, “subject (subject) = <company name>”, “predicate (predicate) = <sales activity>”, “object (object) = < “Sales target>” and “option = <location>” are defined.
In addition, as “necessary (required condition)”, “<company name> clause is related to <sales activity> clause”, “<sales target> clause is related to (new product) clause “The phrase“ (New product) is related to the phrase “Sales activity” ”).
Furthermore, “optional (optional condition)” specifies that “the clause of <location> is related to the clause of <sales activity>”.
The information extraction rule 52 is stored in the information extraction rule storage unit 28 by the information extraction rule generation unit 20.
Since “place” is not an essential condition but an optional condition, it is a rule applicable to a sentence that does not include a character string related to the place.

By applying this information extraction rule 52, for example, the following information can be extracted from the sentence “Toyo Motors will release a new Skyliner product in the US next month”.
Subject (Company Name): Toyo Motors Predicate (Business Activity): Sales Object (Activity Target): Skyliner Option (Location): USA These information elements have a subject-predicate-object-location relationship with each other. Therefore, by accumulating a large amount of such information as a search database, a highly accurate search process with less noise can be realized.

Note that the information extraction rule 52 described above has been transformed into the phrase “Toyo Motors will release a Skyliner next month” because the phrase “New product” needs to be related to the phrase of sales activities. In this case, the information does not match the extraction condition, and information such as “subject (company name): Toyo Motor Corporation—predicate (business activity): sales—object (activity target): skyliner” cannot be extracted.
However, the user can create a rule that can be adapted to such a sentence by repeatedly creating information extraction rules based on many other material sentences.

  For example, as shown in FIG. 14A, a sentence “Annie will release male cosmetics targeting the 20s next month” is written in the sentence display field 42 of the rule editing screen 40. The user clicks the marking parts of “Any (company name)”, “Men's cosmetics (sales target)”, “Release (sales activities)”, and clicks the anchor text 50 of “Create rule”.

Since each clause constituting this sentence has a dependency structure shown in FIG. 14 (b), the information extraction rule generation unit 20 indicates that “<company name> → <sales” as shown in FIG. 14 (c). The dependency relationship “activity>” and “<sales target> → <sales activity>” is extracted, and the information extraction rule 54 shown in FIG.
By applying this information extraction rule 54 to the sentence “Toyo Motors will release Skyliner next month.”, The following information can be successfully extracted.
Subject (Company Name): Toyo Motors Predicate (Business Activity): Sales Object (Activity Target): Skyliner

In editing the information extraction rule, the user understands the meaning of the material sentence and selects a marking portion in the sentence only when it is determined that the sentence expresses an active business activity.
For example, as shown in FIG. 15 (b), when the sentence “Hatsushiba announced that it will postpone the release of a new product of portable music player“ Gigabyte ”on the 15th” is given, “Hatsushiba will release a new product of portable music player“ Gigabyte ”on the 15th.” Like “Hatsushiba”, “Portable music player”, “Gigabyte”, “Release” will be marked. .
However, when an information extraction rule is generated based on the sentence in (b), it can be said that a company has actively engaged in corporate activities from the statement that a company has announced the postponement of a specific corporate activity. As a result, the information shown is extracted in error.
Therefore, the user consciously excludes such a sentence from the information extraction rule creation target.

  On the other hand, as shown in FIG. 15 (c), when a sentence “Hatsushiba has announced that portable music player“ Gigabyte ”will be released in the United States” is given, the predicate of the sentence is “announced”. Although it is a general term and a term representing a company's production activities is not a predicate, it is described that the company sells the object to be sold in the meaning of the whole sentence. Click, and click the “Create Rule” anchor text 50.

Receiving this, the information extraction rule generation unit 20 generates an information extraction rule from the sentence.
Since each clause constituting this sentence has a dependency structure shown in FIG. 16 (a), the information extraction rule generation unit 20 indicates that “<company name> → <sales” as shown in FIG. 16 (b). Activity ”,“ <Sales target> → <Sales activity> ”,“ <Sales activity> → (announced) ”and“ <Location> → <Sales activity> ”are taken out, and the figure (c) The information extraction rule 56 shown in FIG.

By applying this information extraction rule 56, for example, the following company information can be extracted from the sentence “Toyo Motors announced that it will sell hybrid vehicles in Europe next month”.
Subject (Company Name): Toyo Motors Predicate (Business Activity): Sales Object (Activity Target): Hybrid Vehicle Location: Europe

Although illustration is omitted, an information extraction rule can also be generated from a sentence that inherits the subject name subject of the preceding sentence. For example, if the sentence “[(Hatsushiba)]“ [Gigabyte] ”[releases] in late October” is written in the sentence display field 42 of the rule editing screen 40, the user selects [(Hatsushiba) ] (Inherited company name subject), [Gigabyte] and [Marketing] marking part is clicked, click the anchor text 50 of "Create rule", information extraction rule generation unit 20 to extract the following information A rule is generated.
============================
extraction:
subject <company name>
predicate: <Sales activities>
object: <Sales target>

necessary:
<Sales target>-><Salesactivities>
============================

  In this information extraction rule, “necessary (required condition)” does not specify a dependency on <company name>. However, when this information extraction rule is applied, the target sentence inherits from the preceding sentence. The name will be extracted as the subject.

The user can additionally register a dictionary through the rule editing screen 40 described above.
For example, in the sentence display field 42, if the character string “input” is not marked to indicate corporate activity (in this case, “sales activity”), the user selects “input” by the mouse operation. After that, click the anchor text 60 of "Add to activity as dictionary".

Receiving this, the information extraction rule generation unit 20 additionally registers the character string “input” in the corporate activity dictionary of the dictionary storage unit 22 as one of the sales activities.
As a result, the next time a sentence including “input” is displayed in the sentence display field 42, the marking indicating the sales activity is performed.

Similarly, when there is a character string that is clearly a company name but is not included in the marking target, the user selects the character string and selects the anchor text “add to company as dictionary” 58. Click.
Receiving this, the information extraction rule generation unit 20 additionally registers the character string in the company name dictionary of the dictionary storage unit 22 as one of the company names.

In addition, when the character string indicating the activity target is not clearly marked, the user selects the character string and then clicks on the anchor text 62 of “Add to dictionary as target” to expand it. On the selection window (not shown), a general noun and proper noun are selected and input.
Receiving this, the information extraction rule generation unit 20 additionally registers the character string in the activity dictionary of the dictionary storage unit 22 as one of sales activities.

It is a block diagram which shows the function structure of the information extraction rule preparation assistance system which concerns on this invention. It is a chart which illustrates the registration contents of a corporate activity dictionary. It is a chart which illustrates the registration contents of an activity object dictionary. It is a chart which shows the result of morphological analysis. It is explanatory drawing which illustrates the method of certifying the company name subject in a sentence. It is a chart explaining inheritance of a company name subject and anaphora processing. It is a chart which shows the result of parsing. It is the figure which imaged the dependency structure which is a result of a parsing. It is a graph which shows a mode that the abstract tag provided to the morpheme is succeeded by the clause. It is explanatory drawing which shows the abstraction process by a regular expression rule. It is explanatory drawing which shows the abstraction process by a dependency rule. It is a screen block diagram which shows an example of a rule edit screen. It is explanatory drawing which shows the process of information extraction rule production | generation. It is explanatory drawing which shows the process of another information extraction rule production | generation. It is explanatory drawing which shows the other example of the sentence used as the raw material of information extraction rule production | generation. It is explanatory drawing which shows the process of another information extraction rule production | generation. It is explanatory drawing which imaged the dependency structure which is a result of a parsing case. It is explanatory drawing which imaged the information extraction rule. It is explanatory drawing which shows a mode that the structured company information is extracted by applying an information extraction rule to the parsed sentence. It is explanatory drawing which imaged other information extraction rules.

10 Information extraction rule creation support system
12 Morphological analysis processor
14 Subject identification processing section
16 Parsing processing section
18 Abstraction processing section
20 Information extraction rule generator
22 Dictionary storage
24 Abstraction rule storage
26 Material sentence storage
28 Information extraction rule storage
30 client terminals
32 Text data
40 Rule edit screen
42 Sentence display field
44 Sales Activity tab
46 Production Activity tab
48 Extraction node display column
50 “Create Rule” Anchor Text
52 Information extraction rules
54 Information extraction rules
56 Information extraction rules
58 Anchor text for "Add to company as dictionary"
60 Anchor text for "Add to activity as dictionary"
62 Anchor text for "Add to dictionary as target"

Claims (7)

A dictionary in which the correspondence between a specific expression character string and an abstract character string indicating the type is registered,
Means for decomposing each sentence in text data into morpheme units;
Referencing the above dictionary, means for associating a corresponding abstract tag with a morpheme that corresponds to a company name, business activity, or activity target,
A means of searching for the subject name of the company in each sentence;
Means for storing a sentence having a company name subject as a material sentence in a material sentence storage means;
Means for taking out material sentences from the material sentence storage means, filling them in a predetermined template and generating a rule editing screen;
Means for sending the rule editing screen to the client terminal and prompting selection of a material sentence in which the business activity and activity target of the specific company are described;
When the selection information of a specific material sentence is transmitted from the client terminal, the clause including the company name in the material sentence, the clause including the business activity and the dependency structure between the clauses including the activity target, or these company names Dependent structure between clauses containing clauses, clauses containing business activities and clauses containing activity objects, and clauses containing character strings other than company names, business activities, and activity targets are extracted as extraction conditions, and this is formatted in a predetermined format. Information extraction rule generation means for generating an information extraction rule described in accordance with
Means for storing the information extraction rule in the information extraction rule storage means;
An information extraction rule creation support system characterized by comprising:   Each material sentence in the above rule editing screen is marked to identify the type of the character string part indicating the company name, the character string part indicating the company activity, and the character string part indicating the activity target. The information extraction rule making support system according to claim 1, wherein Means for referring to the dictionary and associating a corresponding abstract tag with a corresponding one of the morphemes,
The information extraction rule generation means indicates the location when the selection information of the specific material sentence is transmitted from the client terminal and the material sentence includes a morpheme associated with the location abstraction tag. The information extraction according to claim 1 or 2, wherein a dependency structure between a clause including a morpheme and another clause is extracted as an arbitrary extraction condition, and an information extraction rule including the extraction structure is generated. Rule creation support system.   If there is no subject in the sentence, there is a company name subject in the sentence that precedes the sentence, and another company name subject and the sentence in which the company name subject exists. The method according to any one of claims 1 to 3, further comprising means for associating the company name subject as a company name subject of a sentence that does not have the subject when a sentence whose subject is a morpheme other than the company name is not present. Information extraction rule creation support system described in Crab.   If a company pronoun is included as a subject in a sentence, the company name subject exists in the sentence located before the sentence, and there is another between the sentence and the sentence in which the company name subject exists. The method according to claim 1, further comprising means for replacing the company pronoun with the company name of the company name subject when no sentence having a subject name of the company name and a morpheme other than the company name is present. The information extraction rule creation support system according to any one of the above.   Apply a regular expression rule set in advance to each sentence, identify a morpheme that matches the regular expression rule as one of the company name, corporate activity, or activity target, and handle the morpheme 6. The information extraction rule creation support system according to claim 1, further comprising means for associating the abstract tag.   Apply predetermined dependency rules to each sentence, and identify the morpheme of the clause that matches the dependency rule as either the company name, the company activity, or the activity object, and 7. The information extraction rule creation support system according to claim 1, wherein a corresponding abstract tag is associated.

Images