JP2015200962A - Inter-document relation extraction device and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To allow a variety of relations to be extracted in extracting inter-document relations.SOLUTION: A keyword pair generation section outputs a pair of keywords extracted respectively from two documents, as a keyword pair, on the basis of an inter-word relation retrieved from an inter-word relation dictionary storage section, in the case that the pair of keywords has a direct relation to each other or in the case that the pair of keywords has a direct relation to a common other word. As for the keyword pair outputted by the keyword pair generation section, an inter-keyword relation extraction section outputs the direct relation in the case that the keyword pair has the direct relation to each other, on the basis of the inter-word relation retrieved from the inter-word relation dictionary storage section, and outputs the other word and a relation of the keyword contained in the keyword pair to the other word in the case that the keyword pair has the direct relation to the common other word.

Description

  The present invention belongs to the technical field of natural language processing, and relates to an inter-document relationship extraction apparatus and a program thereof.

  In the technology for extracting the relationship between documents, conventionally, it is determined whether there is an “equivalent” relationship between documents using a word contained in the document, or there is a “transition” relationship between documents. Techniques for determining whether or not are proposed. Non-Patent Document 1 describes a model for specifying “equivalent” between sentences across a document and a model for specifying “transition” between sentences.

Yasunari Miyabe, Daiya Takamura, Manabu Okumura, “Identification of Inter-Document Cross-Sentence Relationships”, 12th Annual Conference of the Language Processing Society, March 2006, pp. 496-499.

  Regarding the relationship between documents, it is desirable to be able to extract not only “equivalent” relationship and “transition” relationship, but also various other relationships. The present invention has been made in consideration of such circumstances, and provides an inter-document relationship extraction apparatus and program capable of extracting relationships other than “equivalent” relationships and “transition” relationships between documents. Is.

  As means for solving the above problems, the inter-document relationship extraction apparatus and program extract a plurality of important keywords from each of two documents and use a word-to-word relationship dictionary automatically acquired from a large collection of documents. To estimate the optimal relationship between keywords. This estimates the relationship between the two documents.

  [1] An inter-document relationship extraction device according to an aspect of the present invention extracts a plurality of keywords of the document from an input document and outputs a score indicating the importance of each of the keywords in the document Based on the relationship between the words read from the inter-word relationship dictionary storage unit that holds data representing the word pairs and the relationship between the words, for the two keyword pairs extracted from the two documents, respectively. When the keyword pair has a direct relationship, or when the keyword pair has a direct relationship with another common word, the keyword pair generation unit outputs the keyword pair as a keyword pair And the relationship between the words read from the inter-word relationship dictionary storage unit with respect to the keyword pair output by the keyword pair generation unit. Therefore, when the keyword pair has a direct relationship, the relationship is output. When the keyword pair has a direct relationship with another common word, the other word and the keyword are output. An inter-keyword relationship extraction unit that outputs a relationship from the keyword included in the pair to the other word.

  [2] Further, according to one aspect of the present invention, in the inter-document relationship extracting apparatus, the inter-keyword relationship extracting unit includes a number of occurrences in the inter-word relationship dictionary storage unit among the relationships of the keyword pairs. It is characterized by outputting the least relationship as the optimum relationship.

  [3] Further, according to one aspect of the present invention, in the inter-document relationship extraction device, the keyword pair generated by the keyword pair generation unit and the relationship output by the keyword relationship extraction unit with respect to the keyword pair And a description generation unit that generates description text data including.

  [4] Further, according to one aspect of the present invention, a keyword extracting unit that extracts a plurality of keywords of the document from an input document and outputs a score indicating the importance of each of the keywords in the document; Based on the relationship between the words read from the inter-word relationship dictionary storage unit that holds data representing the word pair and the relationship between the words, for the two keyword pairs extracted from the two documents, A keyword pair generation unit that outputs a keyword pair as a keyword pair when the keyword pair has a direct relationship, or when the keyword pair has a direct relationship with another common word; The keyword pair output by the keyword pair generation unit is based on the relationship between words read from the inter-word relationship dictionary storage unit. When the keyword pair has a direct relationship, the relationship is output, and when the keyword pair has a direct relationship with another common word, the other word and the keyword pair are output. Is a program for functioning as an inter-keyword relationship extraction unit that outputs a relationship from the keyword included in the other word to the other word.

  According to the present invention, it is possible to estimate and output the relationship between two documents.

1 is a block diagram illustrating a schematic functional configuration of an inter-document relationship extraction device according to an embodiment of the present invention. It is a flowchart which shows the flow of the whole process of the document relationship extraction apparatus by the embodiment. It is a flowchart which shows the procedure of the keyword pair production | generation process by the keyword pair production | generation part by the embodiment. It is a flowchart which shows the procedure of the process in which the relationship extraction part between keywords by the embodiment extracts the relationship between keywords. It is the schematic which shows the example which the relationship extraction apparatus between the documents by the same embodiment output the processing result. It is the schematic which shows the structure of the word relationship dictionary which the word relationship dictionary memory | storage part by the same embodiment memorize | stores. It is the schematic which shows the structure of the word co-occurrence dictionary which the word co-occurrence dictionary memory | storage part by the embodiment memorize | stores.

Next, an embodiment of the present invention will be described with reference to the drawings.
FIG. 1 is a block diagram showing a schematic functional configuration of the inter-document relationship extracting apparatus according to the present embodiment. As shown in the figure, the inter-document relationship extraction apparatus 1 includes a document acquisition unit 10, a keyword extraction unit 11, a keyword pair generation unit 12, an inter-keyword relationship extraction unit 13, an explanation generation unit 14, and an inter-word relationship dictionary. A storage unit 21 and a word co-occurrence dictionary storage unit 22 are included.

The document acquisition unit 10 acquires document data from the outside. The document acquisition unit 10 acquires at least two pieces of document data. In later processing, information representing the relationship between the two documents is extracted. The document data has, for example, a so-called text file format.
The keyword extraction unit 11 extracts a plurality of keywords of the document from the input document, and outputs a score indicating the importance of each keyword in the document. Specifically, the keyword extraction unit 11 extracts a plurality of keywords from each of two document data of the document data acquired by the document acquisition unit 10. Then, the keyword extraction unit 11 outputs keyword set data corresponding to each document.

  When the keyword pair generation unit 12 has two keyword pairs extracted from two documents, the keyword pairs have a direct relationship based on the relationship between words read from the inter-word relationship dictionary storage unit 21. If the keyword pair has a direct relationship with another common word, the keyword pair is output as a keyword pair. That is, the keyword pair generation unit 12 extracts one keyword from each of the keyword sets corresponding to the two document data extracted by the keyword extraction unit 11, and generates keyword pair data between both documents.

  When the keyword pair has a direct relationship based on the relationship between words read from the word relationship dictionary storage unit 21 with respect to the keyword pair output by the keyword pair generation unit 12, the keyword relationship extraction unit 13 In addition to outputting the relationship, if the keyword pair has a direct relationship with another common word, the relationship from the other word and the keyword contained in the keyword pair to the other word is output. To do. That is, the inter-keyword relationship extraction unit 13 performs a process of extracting the relationship between the keywords forming the pair for the keyword pair generated by the keyword pair generation unit 12. Note that the keyword relationship extraction unit 13 refers to the word co-occurrence dictionary storage unit 22 in addition to the word relationship dictionary storage unit 21 during this process.

  The description generation unit 14 generates explanatory text data including the keyword pair extracted by the keyword pair extraction unit and the relationship output by the keyword relationship extraction unit regarding the keyword pair. In other words, the description generation unit 14 performs processing for generating a sentence that explains the relationship between the original two document data by using the relationship extracted by the inter-keyword relationship extraction unit 13 and the keyword.

The inter-word relationship dictionary storage unit 21 holds data representing a word pair and a relationship between the two words. Here, the relationship between words is, for example, a similarity relationship, a causal relationship, an upper / lower relationship, an attribute relationship, and the like. Dictionary data representing the relationship between words is acquired in advance based on text data on the web. In order to automatically acquire the relationship between words based on text data, an inter-word relationship acquisition tool that belongs to the existing technology can be used.
The data in the inter-word relationship dictionary will be described later with reference to FIG.
Instead of providing the inter-word relationship dictionary storage unit 21 in the inter-document relationship extraction device 1, a device having the function of the inter-word relationship dictionary storage unit 21 may be provided as an external device of the inter-document relationship extraction device 1. . In this case, communication between the document relationship extraction apparatus 1 and the word relationship storage unit 21 can be performed via a wired or wireless communication medium. Thereby, the inter-document relationship extracting device 1 acquires necessary data from the inter-word relationship dictionary storage unit 21.

The word co-occurrence dictionary storage unit 22 holds a co-occurrence relationship between words as dictionary data. Here, the co-occurrence relationship data is data representing similarity based on the co-occurrence frequency of two words, and an example is a Dice coefficient. In order to obtain such co-occurrence frequency data, for example, the number of times two words co-occur in the same sentence are counted for text data on the web, and values such as Dice coefficient are calculated based on the count result. Is calculated in advance. In addition, data in a word co-occurrence frequency database that has already been made public can be used.
The data of the word co-occurrence dictionary will be described later with reference to FIG.

Next, an overall processing procedure of the inter-document relationship extracting apparatus 1 will be described.
FIG. 2 is a flowchart showing an overall processing flow of the inter-document relationship extracting apparatus 1. Below, it demonstrates along this flowchart. This flowchart shows a process of inputting two pieces of document data (document 1 and document 2) and extracting a relationship between the documents. It is also possible to input three or more document data and extract the relationship between two of them.

  First, in step S1-1, the keyword extracting unit 11 extracts n keywords (n is a positive integer) from each of the document 1 and the document 2. Document 1 and document 2 are previously acquired by the document acquisition unit 10. The keyword extraction unit 11 extracts n words important for each document as keywords. The process of extracting important keywords from a document can be performed using conventional techniques. The keyword set extracted by the keyword extraction unit 11 is a set of keywords that well represent the characteristics of the original document. For this purpose, the keyword extraction unit 11 extracts the top n keywords from each document, for example, based on the value of TFIDF. TFIDF is the product of the word appearance frequency (Term Frequency) and the inverse document frequency. The keyword extraction unit 11 outputs n keywords for each document, and outputs a weight value indicating the importance of each keyword. As this weight value, the TFIDF value itself may be used, or another weight value whose order relation is consistent with the TFIDF value may be used.

Next, in step S1-2, the keyword pair generation unit 12 generates a keyword pair based on n keywords per document extracted by the keyword extraction unit 11 and the weight value associated with each keyword. Process.
As a result of the processing, the keyword pair generation unit 12 generates N (N is a positive integer) keyword pairs. The value of N can be set in advance. As an example, N = 4 is set in advance. In addition, the keyword pair generation unit 12 calculates a rank score for each keyword pair. The detailed procedure of processing by the keyword pair generation unit 12 will be described later (see FIG. 3).

  Next, in step S <b> 1-3, the inter-keyword relationship extraction unit 13 performs a process of extracting a relationship between keywords for each of the N keyword pairs generated by the keyword pair generation unit 12. At this time, the keyword relationship extraction unit 13 uses the rank score of each keyword pair calculated in the processing of the keyword pair generation unit 12.

  Here, the output by the keyword relationship extraction unit 13 is one of the following two patterns. The first pattern is a case where two keywords constituting a keyword pair have a direct relationship. At this time, the inter-keyword relationship extraction unit 13 outputs the keyword pair and its relationship name. The second pattern is a case where each of two keywords constituting a keyword pair has a direct relationship with a certain common word. At this time, the inter-keyword relationship extraction unit 13 uses the keyword pair and its common A word and a relation name connecting the keyword and the common word are output.

  Based on important (high-ranked) keyword pairs, the relationship extracted and output by the keyword relationship extraction unit 13 represents the relationship between two documents.

  Next, in step S1-4, the description generation unit 14 generates a description corresponding to each keyword pair.

  Specifically, the explanation generation unit 14 outputs a word with a pattern of “XXX connection” for each relationship. A score is output. When a relation name that directly connects two keywords constituting a keyword pair is obtained, the explanation generation unit 14 replaces “XX” in the output pattern “XX connection” with the relation name. When a common word having the same relationship with each of the two keywords constituting the keyword pair can be obtained, the explanation generating unit 14 replaces “XX” in the output pattern “XX connection” with the common word. To do.

  Further, the explanation generating unit 14 generates a sentence serving as an auxiliary explanation by using template data described and stored in advance.

For example, when the output from the inter-keyword relationship extraction unit 13 is a keyword pair and a relationship name that directly connects those keywords, the template used by the explanation generation unit 14 is “<keyword b> is <keyword a>”. <Relationship name>". In this template, <keyword b>, <keyword a>, and <relation name> are parts that are appropriately replaced with specific words or relationship names.
Specifically, for example, when the keyword a is “stroke”, the keyword b is “arteriosclerosis”, and the relation name that directly connects them is “cause”, the explanation generation unit 14 reads “cause connection. Produces the output "Cure is the cause of the stroke."

If the output from the keyword relationship extraction unit 13 is a keyword pair, a common word, and a relationship name between the keyword and the common word, the template used by the explanation generation unit 14 is the relationship name. It depends on.
Specifically, for example, the keyword a is “stroke”, the keyword b is “heart failure”, the common word related to these keywords is “disease”, and the relation name between the keyword and the common word is both In the case of “higher term”, the explanation generation unit 14 outputs “disease connection. Stroke has the same broader term (disease) as heart failure”.
An example of output by the explanation generation unit 14 will be described later with reference to FIG.

Next, processing of main processing units in the apparatus will be described.
FIG. 3 is a flowchart showing details of a process in which the keyword pair generation unit 12 generates a keyword pair. Here, a document pair that is a source of keyword pair generation processing is referred to as a document a and a document b. Then, an input to the keyword pair generation unit 12 for performing a process for generating a keyword pair is a keyword set {w _a1 , w _a2 ,... W _an } extracted from the document a and its weight data, and The keyword set {w _b1 , w _b2 ,... W _bn } extracted from the document b and its weight data. These input data are obtained as a result of the processing of the keyword extraction unit 11.

First, in step S2-1, the keyword pair generation unit 12 extracts one keyword w _ai and w _bj from the keyword set extracted from the document a and the keyword set extracted from the document b, respectively. Candidates (w _ai , w _bj ) are generated. At this time, the method of selecting a keyword from each keyword set is arbitrary.

Next, in step S2-2, the keyword pair generation unit 12 obtains a rank in the document based on the weights assigned to the keywords w _ai and w _bj , and the sum of the ranks is determined as a keyword pair candidate (w _ai , w _bj ) weight rank_score (i, j). That is, rank_score (i, j) = rank _a (i) + rank _b (j). Here, rank _a (i) is the rank of the keyword w _ai in the document a. Rank _b (j) is the rank of the keyword w _bj in the document b. For example, when the keyword w _ai is the keyword having the largest weight in the document a and the keyword w _bj is the keyword having the largest weight in the document b, rank _a (i) = 1 and rank _b ( j) = 1. In this case, rank_score (i, j) = rank _a (i) + rank _b (j) = 1 + 1 = 2.

  In step S2-3, the keyword pair generation unit 12 sets a variable countA for counting the keyword pairs to be generated to 1. For this variable, for example, a predetermined area of the semiconductor memory is allocated.

  Next, in step S2-4, the keyword pair generation unit 12 determines whether all the keyword pair candidates generated in step S2-1 have been processed. When all the processes have been completed (step S2-4: YES), the process of the entire flowchart is terminated. If not all has been processed (step S2-4: NO), the process proceeds to the next step S2-5.

  When the process proceeds to step S2-5, the keyword pair generation unit 12 selects one keyword pair as a processing target from the remaining keyword pairs in the same step. At this time, the keyword pair generation unit 12 selects in ascending order of rank_score (i, j). That is, the keyword pair generation unit 12 selects a keyword pair to be processed as a processing target by giving priority to those having a low rank_score (i, j) value (in other words, giving priority to a keyword pair having a higher rank). . When there are a plurality of keyword pairs having the same rank score value, the inter-keyword relationship extraction unit 13 determines a keyword pair to be processed according to any of the priority orders (a) to (e) listed below. . The priority order is (a) the order of the rank value in the original document of any one of the two keywords constituting the keyword pair, in other words, each of the two keywords constituting the keyword pair. The order of the smallest value of the rank in the original document. (B) Order in which the absolute value of the difference in rank in each original document of the two keywords constituting the keyword pair is small. (C) Order determined randomly by pseudo-random numbers or the like. (D) A combination of (a) and (c) above. (E) A combination of (b) and (c) above.

  Next, in step S2-6, the keyword pair generation unit 12 determines whether both of the keyword pairs to be processed have a direct relationship or whether there is a relationship with a common word with both of the keyword pairs. To do. If any of these conditions is met (step S2-6: YES), the process proceeds to the next step S2-7. If the keyword pair to be processed does not meet any of the conditions (step S2-6: YES), the process returns to step S2-4 to process the next keyword pair candidate. In addition, in order to determine whether a word and a word have a relationship in this step, the keyword pair production | generation part 12 refers to the word relationship dictionary memory | storage part 21. FIG. The inter-word relationship dictionary storage unit 21 stores information representing what kind of relationship a certain word has with another word. The relationship is, for example, a similarity relationship, a causal relationship, an upper / lower relationship, an attribute relationship, or the like.

For example, the inter-word relationship dictionary storage unit 21 includes data representing relationships such as the following (1) and (2).
(1) Stroke-[Relation: causal]-Arteriosclerosis (2) Myocardial infarction-[Relation: causal]-Arteriosclerosis In other words, "stroke" and "arteriosclerosis" have a "causal relationship" In addition, “myocardial infarction” and “arteriosclerosis” have a “causal relationship”. At this time, it is as follows. That is, for example, “stroke” and “arteriosclerosis” have a direct relationship. Further, for example, “stroke” and “myocardial infarction” have a relationship (causal relationship) with a common word “arteriosclerosis”, respectively.

  When the process proceeds to step S2-7, the keyword pair generation unit 12 adds the keyword pair candidate currently being processed to the output as one of the keyword pairs. In other words, when the keyword pair that is a candidate has a direct relationship or a relationship with a common word, the keyword pair generation unit 12 sets the pair as one of the keyword pairs. Generate.

In step S2-8, the keyword pair generation unit 12 updates this variable by adding 1 to the value of the variable countA.
As described above, this variable countA is for counting the keyword pairs to be generated, and adding 1 here corresponds to the fact that one keyword pair is generated in step S2-7 above. Yes.
Next, in step S2-9, the keyword pair generation unit 12 determines whether or not the value of the variable countA is greater than N. That is, it is determined whether or not the inequality of countA> N holds. The value of N is as described above. When the value of the variable countA is greater than N (step S2-9: YES), the process of the entire flowchart is terminated. At that time, the keyword pair generation unit 12 has output N keyword pairs. If the value of the variable countA is N or less (step S2-9: NO), the process returns to step S2-4 to process the next keyword pair candidate.

  Through the series of processes described above, the keyword pair generation unit 12 gives priority to a keyword pair having a higher rank from among the keyword pair candidates, and has a predetermined relationship (determination in step S2-6 is YES). N keyword pairs satisfying the relationship) are output. However, when there are less than N such keyword pair candidates, the keyword pair generation unit 12 outputs less than N keyword pairs (when the determination result in step S2-4 is YES). Further, the keyword pair generation unit 12 outputs the value of the rank score rank_score (i, j) corresponding to each of the output keyword pairs.

  FIG. 4 is a flowchart showing details of the process in which the keyword relationship extraction unit 13 extracts the relationship between keywords. As a premise of this processing, the keyword relationship extraction unit 13 acquires the keyword pair set output by the keyword pair generation unit 12 and the rank score of the keyword pair. Hereinafter, it demonstrates along a flowchart.

  First, in step S3-1, the inter-keyword relationship extraction unit 13 sets a variable countB for counting the number of output relationships to 1. For this variable, for example, a predetermined area of the semiconductor memory is allocated.

  Next, in step S3-2, the inter-keyword relationship extraction unit 13 extracts one keyword pair as a processing target from the acquired keyword pairs. At this time, the inter-keyword relationship extraction unit 13 selects and extracts keyword pairs from the set of unprocessed keyword pairs in ascending order of rank score values. That is, the keyword relationship extraction unit 13 processes keyword pairs in descending order of rank.

  Next, in step S3-3, the inter-keyword relationship extraction unit 13 refers to the above-described inter-word relationship dictionary storage unit 21 to determine whether the keyword pair to be processed has a direct relationship. If the keyword pair has a direct relationship (step S3-3: YES), the process proceeds to step S3-4. If the keyword pair is not directly related (step S3-3: NO), the process proceeds to step S3-5.

  Next, in the case of proceeding to step S3-4, in the same step, the inter-keyword relationship extraction unit 13 performs ranking of relation names regarding the keyword pairs to be processed, and selects an optimum relation name. If the target keyword pair has only one relationship, the relationship name is the optimum relationship name. On the other hand, there are cases where the targeted keyword pairs have a plurality of relationships. As an example, there are two relationships between the word “Terminator 2” and the word “James Cameron”. The first relationship is the relationship between the movie work and the director. The second relationship is between the film director and the screenwriter (writer). These relationships are stored in the inter-word relationship dictionary storage unit 21 as entries in the inter-word relationship dictionary. Here, when a plurality of relation names exist in a specific keyword pair, the inter-keyword relation extraction unit 13 selects an optimum relation name that appears in the entire inter-word relation dictionary from among the relation names. Select as relationship name.

  However, a relation name having a smaller number of appearances than the preset numerical value L may be regarded as an error and the relation name may not be selected. L is a positive number, for example, L = 10. The reason why relation names having a small number of appearances are excluded in this way is because a case where such relation names are included in the inter-word relation dictionary due to an error is taken into consideration.

Next, when the process proceeds to step S3-5, since the target keyword pair has no direct relationship, each of the two keywords included in the pair has a relationship with a certain common word. Is. There may be a plurality of such common words. In this step, the keyword relationship extraction unit 13 selects an optimal relationship name. For example, the keyword “stroke” and the keyword “myocardial infarction” have the following relationship with other words.
(1) Stroke-[Relation: Causal]-Arteriosclerosis (2) Myocardial infarction-[Relation: Causal]-Atherosclerosis (3) Stroke-[Relation: Higher and lower relations]-Sickness (4) Myocardial infarction-[ Relationship: Higher-Lower Relationship]-Disease At this time, the keyword pair “stroke” and “myocardial infarction” have a common word “arteriosclerosis” having a relationship of “causal relationship”. The keyword pair has a common word “disease” having a relationship of “upper and lower relations”.

Since the inter-word relationships (1) to (4) above are registered in the inter-word relationship dictionary, the inter-keyword relationship extraction unit 13 has a common relationship between the keyword “stroke” and the keyword “myocardial infarction”. Extract "arteriosclerosis" and "disease" as the words.
Then, the inter-keyword relationship extraction unit 13 ranks the relationship names for the target keyword pairs and selects the most appropriate relationship name.

  For example, the keyword relationship extraction unit 13 calculates the sum of the similarity between the keyword and the common word from the document a and the similarity between the keyword and the common word from the document b. That is, the keyword relationship extraction unit 13 performs calculation represented by the following expression, and obtains two score values of Score (stroke, myocardial infarction | arteriosclerosis) and Score (stroke, myocardial infarction | disease).

Score (stroke, myocardial infarction | arteriosclerosis) =
disce (stroke, arteriosclerosis) + disce (myocardial infarction, arteriosclerosis)
Score (stroke, myocardial infarction | disease) =
dice (stroke, illness) + dice (myocardial infarction, illness) ・ ・ ・ Equation 2
The former of these two expressions represents the score of the relation name when “arteriosclerosis” is a common word for the word pair “stroke” and “myocardial infarction”. The latter represents the score of the relation name when “disease” is a common word for the word pair “stroke” and “myocardial infarction”. Further, dice (word 1, word 2) is a Dice coefficient between word 1 and word 2.

And as a result of having performed the calculation by the above formulas 1 and 2, for example,
Score (stroke, myocardial infarction | arteriosclerosis)> Score (stroke, myocardial infarction | disease)
In such a case, the inter-keyword relationship extraction unit 13 ranks “arteriosclerosis” having a higher score value as a common word higher than “disease”. Even when there are three or more common words, the common word with the highest score is ranked at the top. That is, the inter-keyword relationship extraction unit 13 ranks common words in descending order of score values.

  Next, the inter-keyword relationship extraction unit 13 specifies a relationship name between the keyword and the common word. In order to specify the optimum relationship name, the keyword relationship extraction unit 13 determines that the more appropriate the relationship name is, the smaller the number of appearances in the word relationship dictionary is, similar to the processing in step S3-4. That is, the inter-keyword relationship extraction unit 13 selects the highest-ranked relationship name related to the highest-ranked common word. Here, as an exception, relation names having fewer than L occurrences in the inter-word relation dictionary may be excluded from selection.

In the process of step S3-5, the inter-keyword relationship extraction unit 13 refers to the word co-occurrence dictionary storage unit 22. The word co-occurrence dictionary storage unit 22 holds similarity data based on word co-occurrence frequencies. An example of the similarity data is a Dice coefficient. When calculating the scores shown in the above formulas 1 and 2, the inter-keyword relationship extraction unit 13 reads and uses the data of the Dace coefficient from the word co-occurrence dictionary storage unit 22.
The configuration of the word co-occurrence dictionary storage unit 22 will be described later with reference to FIG.

  After step S3-4 or S3-5, the process proceeds to step S3-7. In step S3-7, the inter-keyword relationship extraction unit 13 determines whether or not all keyword pairs have been processed. If all keyword pairs have been processed (step S3-7: YES), the processing of the entire flowchart is terminated. If there is a remaining keyword pair (step S3-7: NO), the process proceeds to the next step S3-8.

  Next, in step S3-8, the inter-keyword relationship extraction unit 13 adds the number of relationships processed in step S3-4 or S3-5 to the variable countB.

  In step S3-9, the inter-keyword relationship extraction unit 13 determines whether or not the value of the variable countB is greater than or equal to a preset numerical value M. When countB ≧ M is satisfied (step S3-9: YES), the process of the entire flowchart is terminated. If countB <M (step S3-9: NO), the process returns to step S3-2 to process the next keyword pair.

In other words, as a result of the processing shown in this flowchart, the inter-keyword relationship extraction unit 13 outputs a relationship name directly connecting the keyword pair and those keywords (in the case of the processing in step S3-4), or is related to the keyword pair. A common word and a relation name that connects the keyword and the common word are output (in the case of the process in step S3-5). The inter-keyword relationship extraction unit 13 outputs them for each keyword pair.
In addition, when all the given keyword pairs have been processed, the inter-keyword relationship extraction unit 13 outputs all the relationship names corresponding to those keyword pairs, and ends the processing shown in this flowchart (step S1). S3-7: YES). Further, the inter-keyword relationship extraction unit 13 ends the processing shown in this flowchart even if the given keyword pair remains when the number of processed relationships becomes (M−1) or more (step S3−). 9: YES)

  FIG. 5 is a schematic diagram illustrating an output example of a processing result by the inter-document relationship extraction device 1. As illustrated, in this example, first, the extracted top ten relationships (from relationship 1 to relationship 10) are output in descending order of their scores. Thereafter, a pair of input summary sentences (input summary sentence 1 and input summary sentence 2) and a set of keywords (keyword 1 and keyword 2) extracted from each input sentence are output as reference information. In each of relations 1 to 10, the wording of the pattern “XX connection” generated by the explanation generation unit 14, the score, and an auxiliary explanation are output. In the illustrated example, in the relationship 1, the word “cause connection” is output using the relationship name “cause result”. In relation 2, the word “disease connection” is output using the common word “disease” (here, the broader term).

FIG. 6 is a schematic diagram showing the configuration of the inter-word relationship dictionary stored in the inter-word relationship dictionary storage unit 21. As shown in the figure, the inter-word relationship dictionary is data representing a relationship between a certain word (word 1) and another word (word 2). In the illustrated example, the first line of data indicates that word 1 “stroke” and word 2 “arteriosclerosis” have a causal relationship. The second line of the data indicates that word 1 “myocardial infarction” and word 2 “arteriosclerosis” have a causal relationship. In this “causal relationship” example, word 1 and word 2 are non-commutative, word 1 represents the result, and word 2 represents the cause. The third line of the data indicates that word 1 “stroke” and word 2 “disease” have an upper-lower relationship. The fourth line of the data indicates that word 1 “myocardial infarction” and word 2 “disease” have an upper-lower relationship. In this “higher-lower relationship” as well, word 1 and word 2 are non-commutative, word 1 represents a lower concept, and word 2 represents a higher concept. As one embodiment, the inter-word relationship dictionary storage unit 21 stores this data as a table based on a relational model. As types of relationships between words, there can be various types such as “similarity relationships” and “attribute relationships” in addition to “causal relationships” and “upper and lower relationships”.
The inter-word relation dictionary also has a relation name for generating an explanatory sentence as a data item. This is data held for the purpose of use when the explanation generation unit 14 assigns a relation name to a template to generate an explanation. In the illustrated example, the inter-word relationship dictionary holds “cause” as a relationship name for generating an explanatory sentence corresponding to “causal relationship”. The inter-word relation dictionary holds “higher word” as a relation name for generating a description corresponding to “higher and lower relation”.

  Although only four lines of data are shown in the figure, actually, a huge number of lines of data are included in the inter-word relation dictionary. In addition, it can be said that about the word pair which is not entered in this word relationship dictionary, both words do not have a direct relationship.

Note that the inter-word relationship dictionary data may be created manually in advance, or may be automatically extracted from a large amount of text using existing technology.
The existing technique for extracting the relationship between words is described in the following reference, for example. In this technique, for example, word pairs having a specific relationship are extracted using machine learning based on encyclopedia data that can be acquired through the Internet.
[References: Higher-lower relationship extraction tool Version1.0, URL: http://alaginrc.nict.go.jp/hyponymy/, download date: December 27, 2013]

  FIG. 7 is a schematic diagram showing the configuration of the word co-occurrence dictionary stored in the word co-occurrence dictionary storage unit 22. As illustrated, the inter-word relationship dictionary is data representing a numerical value of the similarity between a certain word (word 1) and another word (word 2). Here, the value of the Dice coefficient is used as the similarity. The Dice coefficient can be automatically obtained by performing statistical processing based on a large amount of document data. The word co-occurrence dictionary storage unit 22 stores this data as a table based on a relational model, but may be expressed in other data formats.

In addition, about the existing database about word co-occurrence frequency, it describes in the following references, for example. In order to construct this database, the score for co-occurrence is calculated by counting the number of times two word pairs co-occur based on the text of the web page. The score is a co-occurrence frequency, a Dice coefficient, and a discounting mutual information amount. There are various conditions for co-occurrence. For example, the frequency is counted under the conditions of co-occurrence in a document, co-occurrence in four adjacent sentences, and co-occurrence in one sentence.
[Reference: (A-5) Word co-occurrence frequency database, URL: https://alaginrc.nict.go.jp/resources/nictmastar/li-resource-info/li-resource-outline.html, download date Date: December 27, 2013]

  Note that the function of the inter-document relationship extracting apparatus in the above-described embodiment may be realized by a computer. In that case, a program for realizing this function may be recorded on a computer-readable recording medium, and the program recorded on this recording medium may be read into a computer system and executed. Here, the “computer system” includes an OS and hardware such as peripheral devices. The “computer-readable recording medium” refers to a storage device such as a flexible disk, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. Furthermore, a “computer-readable recording medium” dynamically holds a program for a short time, like a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line. In this case, a volatile memory inside a computer system serving as a server or a client in that case may be included, and a program that holds a program for a certain period of time. The program may be a program for realizing a part of the above-described functions, or may be a program that can realize the above-described functions in combination with a program already recorded in a computer system.

  According to the present embodiment, since a relationship between documents can be extracted, another document having a specific relationship with a certain document can be specified. Therefore, other documents related to a certain document can be recommended. Moreover, since the relationship name is specified, the reason for recommendation can be specified. When a document is associated with a program such as a television, another program associated with a program can be recommended. When a document is associated with a product for sale, other products associated with the product can be recommended. In addition, the reason for the recommendation (relationship name) can be explicitly output.

(Modification) As a modification of the above embodiment, the inter-document relationship extraction apparatus 1 according to the following mode may be configured. That is, in the above embodiment, the keyword pair generation unit 12 first performs a process of generating N keyword pairs (the process described with reference to FIG. 3), and thereafter, a keyword relationship between the generated keyword pair sets. The extraction unit 13 performed a process of extracting the relationship between keywords (the process described with reference to FIG. 4). On the other hand, in the present modification, each time the keyword pair generation unit 12 generates one keyword pair in the order of rank_score (i, j), the inter-keyword relationship extraction unit 13 selects the keyword pair as a target. Extract relationships.
Specifically, the inter-keyword relationship extraction unit 13 performs a process corresponding to steps S3-3, S3-4, and S3-5 in FIG. 4 at the position of the process in step S2-7 in FIG. At the same time, the termination condition of the inter-document relationship extracting apparatus 1 as a whole is judged and controlled. That is, whether the number of keyword pairs to be processed (the number counted by the variable countA) exceeds the upper limit N, and whether the number of relations to be output (the number counted by the variable countB) is equal to or greater than the upper limit M. Is determined, and when either upper limit is reached, the entire process is terminated.

  The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes designs and the like that do not depart from the gist of the present invention.

  Since the present invention can extract a relationship between documents, another document having a specific relationship with a certain document can be specified. Therefore, it can be used to search for other documents having a specific relationship with a certain document. Further, for example, when a program such as a television broadcast, a radio broadcast, or an internet distribution is associated with a document, it can be used to search for another program having a specific relationship with a certain program. Therefore, the present invention can be used for content recommendation in a content providing service. For example, when a product sold on a mail order site or the like is associated with a document, it can be used to search for another product having a specific relationship with a certain product. Therefore, the present invention can be used for product recommendation on a mail order site or the like.

DESCRIPTION OF SYMBOLS 1 Inter-document relation extraction apparatus 10 Document acquisition part 11 Keyword extraction part 12 Keyword pair production | generation part 13 Keyword relation extraction part 14 Explanation production | generation part 21 Inter-word relation dictionary storage part 22 Word co-occurrence dictionary storage part

Claims (4)

A keyword extracting unit that extracts a plurality of keywords of the document from the input document and outputs a score indicating the importance of each of the keywords in the document;
Based on the relationship between the words read from the inter-word relationship dictionary storage unit that holds data representing the word pair and the relationship between the words, for the two keyword pairs extracted from the two documents, A keyword pair generation unit that outputs a keyword pair as a keyword pair when the keyword pair has a direct relationship, or when the keyword pair has a direct relationship with another common word;
Regarding the keyword pair output by the keyword pair generation unit, when the keyword pair has a direct relationship based on the relationship between words read from the inter-word relationship dictionary storage unit, the relationship is output, When the keyword pair has a direct relationship with another common word, the relationship between the other word and the keyword contained in the keyword pair is output from the keyword to the other word. And
An inter-document relationship extracting apparatus comprising: The inter-keyword relationship extraction unit outputs, as an optimal relationship, a relationship having the smallest number of occurrences in the inter-word relationship dictionary storage unit among the relationships of the keyword pair.
The inter-document relationship extracting device according to claim 1, wherein An explanation generating unit that generates explanatory text data including the keyword pair generated by the keyword pair generating unit and the relationship output by the inter-keyword relationship extracting unit with respect to the keyword pair;
The inter-document relationship extracting apparatus according to claim 1, further comprising: Computer
A keyword extracting unit that extracts a plurality of keywords of the document from the input document and outputs a score indicating the importance of each of the keywords in the document;
Based on the relationship between the words read from the inter-word relationship dictionary storage unit that holds data representing the word pair and the relationship between the words, for the two keyword pairs extracted from the two documents, A keyword pair generation unit that outputs a keyword pair as a keyword pair when the keyword pair has a direct relationship, or when the keyword pair has a direct relationship with another common word;
Regarding the keyword pair output by the keyword pair generation unit, when the keyword pair has a direct relationship based on the relationship between words read from the inter-word relationship dictionary storage unit, the relationship is output, When the keyword pair has a direct relationship with another common word, the relationship between the other word and the keyword contained in the keyword pair is output from the keyword to the other word. Part,
Program to function as.

Images