JP2014153862A - Information extraction apparatus, information extraction method, and information extraction program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To accurately extract information on a related category from a document.SOLUTION: An event extraction model learning function 50 calculates a feature vector representing features of category combinations for combinations of candidates of categories stored in an extraction candidate DB 20, calculates a loss for a collect solution for a category combination which has the maximum score calculated by the feature vector and an extraction model, and updates the event extraction model when the loss is equal to or less than a predetermined value, to prepare a more proper event extraction model. An event extraction function 90 acquires a category combination having the maximum score calculated by the feature vector of the category combination and the event extraction model prepared by the event extraction model learning function 50, for combinations of candidates of categories stored in a prediction extraction candidate DB 80.

Description

  The present invention relates to a technique for retrieving information from a document.

  By automatically extracting event information from web pages and blog articles that describe local event information, it is possible to construct an event information database without human labor, and use it for event recommendation services, etc. can do.

  In order to extract the event information from the text, for example, using Non-Patent Documents 1 to 3, candidates for each category such as event name, location, date and time can be extracted. In addition, if there is correct data tagged manually for each category, it is possible to construct a discriminator that automatically discriminates against the event name, location, date and time using a supervised machine learning framework. Event name, location, date and time can be automatically extracted from web documents.

Yamada, et al., “Japanese Named Expression Extraction Using Support Vector Machine”, Transactions of Information Processing Society of Japan, Information Processing Society of Japan, January 2002, Vol. 43, No. 1, pp.44-53 Hirano and two others, “Resolving ambiguity of place names using geographical distance and famousness”, 70th Information Processing Society of Japan Conference, Information Processing Society of Japan, 2008, pp.2-85-2-86 Takashima and three others, “Specified date and time search considering the valid range of written date and time”, 3rd Forum on Web and Database, 2010 Hira, et al., “Predicate term structure analysis using structural learning”, Proc. Of the 14th Annual Conference of the Language Processing Society of Japan, 2008, pp.556-559 Crammer K., et al., “Online Passive-Aggressive Algorithms”, Journal of Machine Learning, 2006, Vol. 7, pp. 551-585

  However, the method for constructing the discriminator for each category does not take into account the output results of the discriminators of each category at the same time, so it cannot handle features such as the expression of each category easily appearing in the document. The event extraction accuracy may be reduced.

  In addition, some applications require that the date and time of event information be obtained accurately even if other information is incorrect. In such a situation, date and time errors may be more important than location errors. . In the conventional method, the importance of the error cannot be taken into account, and there is a possibility that the event extraction accuracy may be lowered.

  The present invention has been made in view of the above, and an object of the present invention is to accurately extract information on relevant categories from a document.

  The information extraction method according to the first aspect of the present invention is directed to all possible combinations of each category candidate stored in the extraction candidate storage means that extracts and stores candidates for each related category from the document information. Calculating a feature vector representing the characteristics of the combination; obtaining an extraction model stored in the extraction model storage means; and a combination having a maximum score calculated using the feature vector; and The combination of correct answers is read from the correct answer storage means storing the correct answer, the score is calculated, and if the loss of the score of the combination acquired in the acquiring step with respect to the score of the correct answer combination is within a predetermined range, the extraction Updating the extracted model stored in the model storage means, and having a category that For all possible combinations of the candidates for each category stored in the prediction target extraction candidate storage means that extracts and stores these candidates from the prediction target document information, a feature vector representing the characteristics of the combination is calculated. The method further comprises a step of obtaining a combination having a maximum score calculated using the extracted model stored in the extracted model storage unit and the feature vector.

  In the information extraction method, the step of updating the extraction model includes reading the error cost from a cost storage unit storing error costs between categories or within categories, and acquiring the combination using the error cost. The cost is calculated, and the calculated cost is set to the predetermined range.

  An information extraction apparatus according to a second aspect of the present invention is an extraction model storage means for storing an extraction model for extracting information of related categories, and extracts candidates for each related category from document information. A feature vector representing the characteristics of the combination of the stored extraction candidate storage means, the correct storage means for storing the correct answer of each category, and all possible combinations of the respective category candidates stored in the extraction candidate storage means A feature vector calculation means for calculating the combination, an extraction model stored in the extraction model storage means and a combination acquisition means for obtaining a combination with a maximum score calculated using the feature vector, and the correct answer storage means from the The combination of the correct combination is read out, the score is calculated, and the combination of the correct combination is scored An extraction model update unit that updates the extraction model stored in the extraction model storage unit when the loss of the combination score acquired by the acquisition unit is within a predetermined range; A prediction target extraction candidate storage unit extracted from the document information and stored, and for all possible combinations of each category candidate stored in the prediction target extraction candidate storage unit, a feature vector representing the characteristics of the combination is obtained. And an information extraction means for obtaining a combination that calculates and obtains a maximum score calculated using the extraction model and the feature vector stored in the extraction model storage means.

  The information extraction apparatus further includes cost storage means for storing error costs between categories or within categories, wherein the extraction model update means reads the error costs from the cost storage means and uses the error costs to perform the combination. The cost for the combination acquired by the acquisition means is obtained, and the obtained cost is set in the predetermined range.

  An information extraction program according to a third aspect of the present invention causes a computer to execute each step of the information extraction method.

  According to the present invention, related category information can be accurately extracted from a document.

It is a functional block diagram which shows the structure of the event information extraction apparatus in this Embodiment. It is a figure which shows the example of the data stored in the document database. It is a figure which shows the example of the data stored in the extraction candidate database. It is a figure which shows the example of the data stored in the correct database. It is a figure which shows the example of the data stored in the cost database. It is a flowchart which shows the flow of a process of an event extraction model learning function. It is a figure which shows the example of the event extraction model stored in the event extraction model database. It is a figure which shows the example of the data stored in the prediction object document database. It is a figure which shows the example of the data stored in the prediction object extraction candidate database. It is a flowchart which shows the flow of a process of an event extraction function. It is a figure which shows the example of the event information stored in the event database. The experimental result which calculated | required the precision of the prediction with respect to each category of the conventional method and the event information extraction apparatus in this Embodiment is shown.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is a functional block diagram showing the configuration of the event information extraction apparatus in the present embodiment. 1 includes a document DB 10, an extraction candidate DB 20 (corresponding to extraction candidate storage means), a correct answer DB 30 (corresponding to correct answer storage means), a cost DB 40 (corresponding to cost storage means), and an event extraction model learning function. 50 (corresponding to feature vector calculation means, combination acquisition means and extraction model update means), event extraction model DB 60 (corresponding to extraction model storage means), prediction target document DB 70, prediction target extraction candidate DB 80 (prediction target extraction candidate storage means) Correspondence), an event extraction function 90 (corresponding to information extraction means), and an event DB 100. Each unit included in the event information extraction device may be configured by a computer including an arithmetic processing device, a storage device, and the like, and the processing of each unit may be executed by a program. This program is stored in a storage device included in the event information extraction device, and can be recorded on a recording medium such as a magnetic disk, an optical disk, or a semiconductor memory, or provided through a network.

  First, the event extraction model learning function 50 will be described. The event extraction model learning function 50 outputs an event extraction model with the information stored in the document DB 10, the extraction candidate DB 20, the correct answer DB 30, and the cost DB 40 and the number of repetitions T as inputs.

  The document DB 10 assigns and stores a document ID to text information (text data) from which event information is extracted. FIG. 2 shows an example of data stored in the document DB 10.

  The extraction candidate DB 20 stores event name candidates, location candidates, and date / time candidates extracted from each of the text information stored in the document DB 10 for each document ID. FIG. 3 shows an example of data stored in the extraction candidate DB 20. In the example illustrated in FIG. 3, “event” and “Otaru Long Christmas 2012 Final” are stored as event name candidates extracted from the body information with the document ID of 1. Although not shown in FIG. 3, appearance position information in the text information is given to event name candidates, place candidates, and date / time candidates. Extract non-patent document 1 for extracting event name candidates, non-patent document 2 for extracting location candidates, and non-patent document 3 for extracting date / time candidates. Data to be stored in the candidate DB 20 can be generated. If the event name, location, and date / time can be extracted, event recommendation based on the location / date / time is possible. In this embodiment, these three pieces of information are used as a unit of event information. The event name, location, date and time are called event information categories. In the present embodiment, the event name, location, and date / time are used as the event information category, but other information (for example, fee, host organization, etc.) can be realized in the same framework as in the present embodiment.

  The correct answer DB 30 stores the correct answer of each category, that is, the correct event name, the correct answer place, and the correct date and time for each of the text information stored in the document DB 10. FIG. 4 shows an example of data stored in the correct answer DB 30. These correct answers are prepared in advance by hand. In addition, like the extraction candidate DB 20, appearance position information in the text information is given.

  The cost DB 40 stores an error cost for each category. FIG. 5 shows an example of data stored in the cost DB 40. In FIG. 5, event represents an event name error cost, geo represents a location error cost, and time represents a date and time error cost. The higher the value, the higher the cost. These costs are preliminarily created manually.

  Next, the process flow of the event extraction model learning function 50 will be described.

  FIG. 6 is a flowchart showing a process flow of the event extraction model learning function 50.

First, the weight vector w is initialized as w = (0, 0, 0,..., 0) ^T, and the repetition counter t is initialized as t ← 1 (step S11). The weight vector w is an event extraction model stored in the event extraction model DB 60. The number of dimensions of the weight vector w is the same as the number of feature vectors described later.

  Subsequently, one record is selected at random from the correct answer DB 30 (step S12). The document ID of the record selected here is d.

  Subsequently, a record with the document ID d is selected from the extraction candidate DB 20, and a set of possible candidate combinations in all categories (hereinafter referred to as "category combination set") is created and included in the category combination set. Feature vectors are created for all combinations (step S13). The category combination set created from the record whose document ID is 2 in the extraction candidate DB 20 shown in FIG. 3 is as follows.

The 20th Yokosuka Festival-Yokosuka City, Kanagawa Prefecture-October 20, 2012 The 20th Yokosuka Festival-Yokosuka City, Kanagawa Prefecture-December 20, 2012 ・ ・ ・
Festival-Tokyo-January 1, 2013

  Thus, the category combination set in the present embodiment is a set of all possible candidate combinations for event name candidates, place candidates, and date / time candidates. In the example of document ID 2 in FIG. 3, there are two event name candidates, two place candidates, and three date / time candidates, so there are 2 · 2 · 3 = 12 combinations. Here, the combination having the event name, location, and date / time stored in the selected record of the correct answer DB 30 is treated as a correct answer, and other combinations are regarded as incorrect combinations, and information on which category is incorrect is held simultaneously. It shall be. For example, if the location and date / time are different from the correct answer, it is determined that the location / date / time is incorrect. Thus, an error combination has one or more incorrect categories.

  In step S13, a feature vector Φ (y, x) representing the characteristics of each category combination is further created based on the created category combination set and the text information acquired from the document DB 10. Here, x is a vector representation of text information corresponding to the document ID, and y is an element (category combination) of the category combination set. Φ (y, x) is an M-dimensional vector, and is composed of an output of a feature function φ (y, x) for capturing the likelihood of M pieces of event information having y and x as inputs. As an example of the feature function φ (y, x), for example, in order to capture the feature that “three candidates of y appear in close positions in the document”, 1 when three expressions appear within 50 characters, Otherwise, a feature function that outputs 0 is given. Another example is a feature function that outputs 1 when a character string included in an event name also appears elsewhere in the text, and outputs 0 otherwise. As a basic feature function based on other character strings, for example, the method of Non-Patent Document 4 can be used.

Subsequently, using the error cost stored in the cost DB 40, the cost is calculated for each category combination (step S14). The cost for each category combination is calculated using a preset cost function ρ. In the category combination set, if the correct vector of the category combination is y and the category combination vector in which only the location is incorrect is y _erroreo , the cost of the category combination in which only the location is incorrect is calculated by ρ (y, y _erroreo ). If the category combination vector in which only the date and time is incorrect is y _errortime , the cost of the category combination in which only the date and time is incorrect is calculated by ρ (y, y _errortime ).

  For an error combination, the cost of an incorrect category is read from the cost DB 40. If there is one wrong category, use the cost of that category. When there are a plurality of erroneous categories, the costs of the corresponding categories are added together.

For example, when the error cost for the location error is larger than the error cost for the date / time error, the following equation (1) is obtained.

When a plurality of categories are wrong, for example, the vector y _{error_and_geo of the} category combination in which the event name and the place are wrong can be obtained by adding the cost as shown in the following equation (2).

Note that unequal costs can be set not only for the cost function for a category but also for candidates within the same category. For example, for the correct answer for the location category Yokosuka City, Kanagawa Prefecture, in the event information, the error in selecting Yokohama City, Kanagawa Prefecture makes a closer guess than the error in selecting Obihiro City, Hokkaido. The loss can be considered small. Therefore, by setting the cost for the error to select Obihiro City, Hokkaido larger than the cost for the error to select Yokohama City, Kanagawa Prefecture, it is possible to generate a model that makes a guess closer to the correct answer. In addition, in the event name, the edit distance of the character string with respect to the correct answer expression, the geographical distance of the place name in the place, the number of days shifted from the correct date in the date and time, and the like can be used. When it is desired to set a cost that emphasizes even a small deviation, for example, a function such as f (x) = | x | ^r using a value of 0 <r <1 may be used.

Subsequently, the category combination that provides the maximum score with the current weight vector w is obtained (step S15). The category combination that gives the maximum score is calculated by the following equation (3).

Here, Y _t is the category combination sets in the document selected in the t-th iteration, the y _t category combinations of answer, is x _t is the body information of the document.

Subsequently, the loss of the category combination with the maximum score obtained in step S15 is calculated. If the loss is greater than 0, the weight vector w is updated (step S16). The loss l _t in the t-th iteration is calculated by the following equation (4).

Here, the cost calculated in step S14 is used as the third term on the right side. If the loss l _t > 0, the weight vector w is updated according to the loss l _t . For example, the method of Non-Patent Document 5 can be used to update the weight vector.

  Then, the repeat counter t is incremented (step S17), and if it is less than or equal to the predetermined repeat count T (Yes in step S18), the process returns to step S12, and if the repeat counter t exceeds the repeat count T (No in step S18). ), And outputs the weight vector w to the event extraction model DB 60 (step S19).

  Next, the event extraction function 90 will be described. The event extraction function 90 extracts event information from information stored in the prediction target document DB 70 and the prediction target extraction candidate DB 80 using the event extraction model stored in the event extraction model DB 60.

The event extraction model DB 60 stores the event extraction model obtained by the event extraction model learning function 50. The event extraction model is composed of a weight vector w = (w ₁ , w ₂ ,..., W _M ) ^T for M-dimensional features. FIG. 7 shows an example of an event extraction model stored in the event extraction model DB 60.

  Similar to the document DB 10, the prediction target document DB 70 assigns and stores a document ID to the text information from which event information is extracted. FIG. 8 shows an example of data stored in the prediction target document DB 70.

  Like the extraction candidate DB 20, the prediction target extraction candidate DB 80 stores event name candidates, place candidates, and date / time candidates extracted from the text information stored in the prediction target document DB 70. FIG. 9 shows an example of data stored in the prediction target extraction candidate DB 80.

  Next, the process flow of the event extraction function 90 will be described.

  FIG. 10 is a flowchart showing a process flow of the event extraction function 90.

  First, an unprocessed record is selected from the prediction target document DB 70 (step S21). It is assumed that the document ID of the record selected here is d ′.

  A record having a document ID of d 'is selected from the prediction target extraction candidate DB 80 to create a category combination set, and feature vectors are created for all category combinations included in the category combination set (step S22). A feature vector Φ (y, x) is created by the same processing as step S13 by the event extraction model learning function 50.

Subsequently, the category combination that gives the maximum score is acquired using the event extraction model stored in the event extraction model DB 60 (step S23). Specifically, as shown in the following equation (5), the inner product of the weight vector w stored in the event extraction model DB 60 and the feature vector Φ (y, x) created in step S22 is calculated, and the maximum Get the category combination that gives the score.

Here, Y _test is a set of category combinations in the input document, and x is text information of the input document.

  Each category of the category combination acquired in step S23 is output to the event DB 100 (step S24).

  If there is an unprocessed record in the prediction target document DB 70 (Yes in step S25), the process returns to step S21 to obtain the next record. If there is no unprocessed record (No in step S25), the process is performed. finish.

  FIG. 12 shows an example of event information stored in the event DB 100. The event DB 100 stores event information extracted for each document ID.

  Next, experimental results of the event information extraction device in the present embodiment will be described.

  309 Japanese blog articles including local event information are stored in the document DB 10, and a unique expression is extracted by using the methods of Non-Patent Documents 1 to 3, and an extraction candidate DB 20 is created, and an event name, a place, and a date are extracted. The correct DB 30 was constructed.

  As a method for comparison, a baseline method was used in which a rule-based decision was made using the extracted specific expression type.

  In the method using the event information extraction device of the present embodiment, the data was divided into 5 blocks, 4 blocks were used as training data, 1 block was used as test data, and 5 combinations were tried.

  FIG. 12 shows the experimental results for obtaining the prediction precision (number of correct answers / number of predictions) for each category. As shown in the figure, in any item, the method using the event information extraction apparatus of this embodiment is superior to the baseline method.

  As described above, according to the present embodiment, the event extraction model learning function 50 calculates a feature vector representing the characteristics of each category combination for each category candidate combination stored in the extraction candidate DB 20, For the category combination that maximizes the score calculated by the feature vector and the event extraction model, calculate the loss for the correct answer, and update the event extraction model when the loss is within the specified value to create a more accurate event extraction model. As for the combination of candidates for each category stored in the prediction target extraction candidate DB 80 by the event extraction function 90, the feature vector of the category combination and the score calculated by the event extraction model created by the event extraction model learning function 50 are calculated. Get the largest category combination And by, the event name, location, extraction of high-precision event information in consideration of the relationship between categories, such as date and time can be achieved.

  According to the present embodiment, it is possible to learn an event extraction model that reflects the importance of errors between categories or within categories by adding error costs between categories or within categories to loss evaluation. For example, it is possible to generate an event extraction model that reduces location errors even at the expense of date and time. Thereby, it is possible to prevent generation of an event extraction model that outputs a result that significantly reduces user satisfaction.

10 ... Document DB
20 ... Extraction candidate DB
30 ... Correct DB
40 ... Cost DB
50 ... Event extraction model learning function 60 ... Event extraction model DB
70 ... prediction target document DB
80 ... prediction target extraction candidate DB
90 ... Event extraction function 100 ... Event DB

Claims (5)

A step of calculating a feature vector representing a feature of the combination for all possible combinations of each category candidate stored in the extraction candidate storage means that extracts and stores candidates for each related category from the document information When,
Obtaining a combination that maximizes the score calculated using the extracted model and the feature vector stored in the extracted model storage means;
When the correct combination of the categories is read out from the correct storage means storing the correct combination, the score is calculated, and the loss of the combination score acquired in the acquiring step with respect to the correct combination score is within a predetermined range Updating the extraction model stored in the extraction model storage means,
For each possible combination of each candidate for the category stored in the prediction target extraction candidate storage means that extracts and stores candidates for each category from the prediction target document information, a feature vector representing the characteristics of the combination is calculated. An information extraction method comprising: obtaining a combination that maximizes a score calculated using the extraction model stored in the extraction model storage unit and the feature vector.   The step of updating the extraction model reads the error cost from a cost storage unit storing error costs between categories or within categories, and obtains a cost for the combination acquired in the acquisition step using the error cost. The information extraction method according to claim 1, wherein the predetermined cost is within the predetermined range. Extraction model storage means for storing an extraction model for extracting relevant category information;
Extraction candidate storage means for extracting and storing candidates for each relevant category from document information;
Correct answer storage means storing correct answers for each category;
Feature vector calculation means for calculating a feature vector representing the characteristics of the combination for all possible combinations of each category candidate stored in the extraction candidate storage means;
A combination acquisition unit that acquires a combination with a maximum score calculated using the extraction model stored in the extraction model storage unit and the feature vector;
When the combination of the correct answers acquired by the combination acquisition means with respect to the correct combination score is within a predetermined range by reading the correct combination from the correct storage means and calculating a score, the extracted model storage means An extraction model updating means for updating the extraction model stored in
Prediction target extraction candidate storage means for extracting and storing candidates for each category from the prediction target document information;
For all possible combinations of candidates for each category stored in the prediction target extraction candidate storage unit, a feature vector representing the characteristics of the combination is calculated, and the extraction model stored in the extraction model storage unit and the And an information extraction unit that obtains a combination that maximizes the score calculated using the feature vector. A cost storage means for storing error costs between categories or within categories;
The extraction model update unit reads the error cost from the cost storage unit, calculates a cost for the combination acquired by the combination acquisition unit using the error cost, and sets the calculated cost as the predetermined range. The information extraction device according to claim 3.   An information extraction program causing a computer to execute each step of the information extraction method according to claim 1.

Images