WO2016035273A1 - テキスト処理システム、テキスト処理方法、及び、コンピュータ・プログラムが記録された記憶媒体 - Google Patents
テキスト処理システム、テキスト処理方法、及び、コンピュータ・プログラムが記録された記憶媒体 Download PDFInfo
- Publication number
- WO2016035273A1 WO2016035273A1 PCT/JP2015/004155 JP2015004155W WO2016035273A1 WO 2016035273 A1 WO2016035273 A1 WO 2016035273A1 JP 2015004155 W JP2015004155 W JP 2015004155W WO 2016035273 A1 WO2016035273 A1 WO 2016035273A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- sentence
- partial structure
- common partial
- common
- tree
- Prior art date
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/268—Morphological analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/20—Natural language analysis
- G06F40/205—Parsing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/30—Semantic analysis
Definitions
- the present invention relates to a text processing system that can recognize an implication relationship between a plurality of texts (sentences).
- Text entailment recognition is a technique for determining whether or not one sentence (target sentence) includes the meaning represented by the other sentence (hypothesis sentence) for two sentences (sentences).
- text implication recognition is considered to be a technique for recognizing (determining) an implication relationship between two or more sentences.
- Non-Patent Document 1 discloses various techniques (methods) proposed in connection with text entailment recognition. Many of the methods disclosed in Non-Patent Document 1 use the coverage of characters or words when determining the implication relationship between texts.
- the coverage is the ratio of elements (for example, words and phrases) that are common to the target sentence in the hypothetical sentence.
- the coverage is information (feature amount) generally used for determination in text implication recognition because it represents the possibility of implication in the vocabulary.
- Non-Patent Document 2 discloses an example of a technique for determining an implication relationship between a target sentence and a hypothetical sentence.
- the technique disclosed in Non-Patent Document 2 converts the target sentence and the hypothesis sentence into a tree structure representing a dependency structure.
- Such a technique determines an implication relationship between the target sentence and the hypothetical sentence based on the ratio of the subtree in common with the tree structure representing the target sentence in the converted tree structure representing the hypothetical sentence.
- Non-Patent Document 3 discloses another example of a technique for determining an implication relationship between a target sentence and a hypothetical sentence.
- the technique disclosed in Non-Patent Document 3 converts the target sentence and the hypothetical sentence into a tree structure representing a dependency structure.
- Such a technique determines an implication relationship between the target sentence and the hypothetical sentence based on the number of editing operations that occur when the tree structure representing the target sentence is transformed into a tree structure representing the hypothetical sentence.
- the number of editing operations represents the number of operations for editing the tree structure, such as insertion, deletion, replacement, or movement of a partial tree of each node constituting the tree structure.
- Non-Patent Document 3 is a feature amount represented by the number of editing operations (that is, the degree (degree) of difference between a tree structure representing a hypothetical sentence and a tree structure representing a target sentence). Is used to determine the implication relationship between the target sentence and the hypothetical sentence.
- Non-Patent Document 1 many of the techniques disclosed in Non-Patent Document 1 are techniques for determining an implication relationship by using a coverage as one of feature quantities.
- the technique disclosed in Non-Patent Document 2 or Patent Document 3 is a node or subtree that is included in common in a tree structure that represents a target sentence and a tree structure that represents a hypothetical sentence, or a node or subtree that is not common. This is a technique for determining an implication relationship using these as feature amounts.
- nodes and subtrees in a tree structure may be collectively referred to as “partial structures”.
- Patent Document 1 discloses a technique for generating a new text based on a plurality of previously collected texts.
- the technique disclosed in Patent Document 1 collects collective data in which a text and an intention represented by the text are paired. Such technology stratifies the intentions based on the relationship between the plural intentions.
- Such a technique generates a new text by using a plurality of texts associated with the hierarchized intention and combining matching and non-matching parts of morphemes constituting the text.
- Patent Document 2 discloses a technique for generating a rule used for classification of structured documents such as XML (Extensible Markup Language).
- the technique disclosed in Patent Document 2 defines feature values related to a variable portion (elements and attributes of a structured document) in the schema of the structured document.
- the technique disclosed in Patent Document 2 generates rules used for classification of structured documents based on feature values acquired from a plurality of structured documents. Such a technique determines similarity between structured documents based on the generated rules.
- Patent Document 3 discloses a technique for determining the similarity of description data (documents or source code) described based on a certain law.
- the technique disclosed in Patent Document 3 converts a plurality of description data described based on a predetermined rule (such as grammar) into a description format such as a parse tree.
- a predetermined rule such as grammar
- Such a technique decomposes the parse tree into subtrees by pruning at a specific level.
- Such a technique determines the similarity between the description data by determining the similarity with respect to the combination of the corresponding subtrees among the plurality of description data.
- Patent Document 4 discloses a technique for extracting synonymous expressions from pairs of sentences similar to each other.
- the technique disclosed in Patent Document 4 performs dependency analysis for each similar sentence pair. Based on the result, such a technique extracts a common expression included in common with each other and a difference expression included only in individual sentences.
- the technique disclosed in Patent Literature 4 extracts synonymous expressions based on the similarity of relative positions where the common expressions are arranged in each sentence and the similarity of relative positions between the different expressions and the common expressions. .
- the technology related to text entailment recognition described above has the following problems.
- the technique for executing implication recognition based on the coverage implies that the target sentence implies a hypothetical sentence having a high coverage with respect to the target sentence. The tendency to judge is shown. In such a case, it is difficult for such a technique to correctly determine an implication relationship (non-implication) between the target sentence and a hypothetical sentence.
- Non-Patent Document 2 and Non-Patent Document 3 do not sufficiently take into account the structural differences between sentences as described above. Specifically, these related techniques, for example, detect differences in the sentence structure between the target sentence and the hypothetical sentence related to the partial structure (subtree, etc.) that is commonly included in the target sentence and the hypothetical sentence. Not enough consideration.
- Patent Document 1 uses the inclusion relation of the intention represented by the text on the premise that information indicating the upper or lower order or the same level of the intention represented by the text is given in advance, and thereby creates a new text. It only discloses the technology to generate.
- Patent Document 2 is a technique for classifying a structured document in a structured document such as XML based on a feature amount related to a document structure definition (schema). That is, the technique disclosed in Patent Document 2 is not a technique that can be directly applied to the determination of the implication relationship regarding a document.
- Patent Document 3 is directed to a document described according to a predetermined grammar, but a natural language may not always be described according to an accurate grammar. Further, when natural language is converted into a tree structure, even if the contents of subtrees in a plurality of documents are similar, considering the structure of the entire document, the documents may not have an implication relationship.
- Patent Document 4 merely discloses a technique for extracting synonyms using document data that is already known to have a similar relationship. That is, the technique disclosed in Patent Document 4 is not a technique that can solve the above-described problem relating to the determination of the implication relationship between documents.
- the present invention has been made in view of the above circumstances.
- the main object of the present invention is to provide a text processing system or the like that can determine an implication relationship between a plurality of sentences by using information representing the structure of the sentence.
- the present invention for example, by using information representing the structure of a sentence, it is possible to appropriately determine an implication relationship between sentences with a high coverage ratio, and to specify a portion (structure) that affects the implication relation in each sentence
- One of the main purposes is to provide a possible text processing system.
- a text processing system comprises the following arrangement. That is, the text processing system according to one aspect of the present invention extracts a common partial structure, which is a structure of the same type part common to the structure representing the first sentence and the structure representing the second sentence.
- a common partial structure extracting means a feature quantity based on a dependency between one or more common partial structures in the first and second sentences, and the common partial structure in the first and second sentences
- the feature amount extracting means for extracting at least one of the feature amounts based on the dependency relationship between the partial structures different from the common partial structure, and the extracted feature amounts are used for the first feature.
- a text processing method includes: a determination unit that determines an implication relationship between a sentence and the second sentence. That is, in the text processing method according to one aspect of the present invention, the information processing apparatus has a structure of the same type part common to the structure representing the first sentence and the structure representing the second sentence. Extracting a common part structure, a feature quantity based on a dependency between one or more common part structures in the first and second sentences, and the common part structure in the first and second sentences; By extracting at least one of the feature quantities based on the dependency relationship between the common partial structure and the partial structure different from the common partial structure, and using the extracted feature quantity, the first sentence and the second sentence Determine implications between sentences.
- the text processing apparatus has the following configuration. That is, the text processing apparatus according to one aspect of the present invention extracts a common partial structure that is a structure of the same type of parts common to the structure representing the first sentence and the structure representing the second sentence.
- a common partial structure extracting means a feature quantity based on a dependency between one or more common partial structures in the first and second sentences, and the common partial structure in the first and second sentences.
- the feature amount extracting means for extracting at least one of the feature amounts based on the dependency relationship between the partial structures different from the common partial structure, and the extracted feature amounts are used for the first feature.
- determining means for determining an implication relationship between the sentence and the second sentence as a single device.
- Another object of the present invention is to provide a computer program for realizing the text processing system having the above-described configuration and the corresponding text processing method by a computer, and a computer-readable storage medium in which the computer program is recorded. Achieved.
- an implication relationship between a plurality of sentences by using information representing the structure of the sentence.
- an implication relationship between sentences with a high coverage can be appropriately determined, and a portion (structure) that affects the implication relation in each sentence can be specified.
- FIG. 1 is a block diagram illustrating a functional configuration of a text entailment recognition apparatus according to the first embodiment of the present invention.
- FIG. 2 is a flowchart illustrating the operation of the text entailment recognition apparatus according to the first embodiment of the invention.
- FIG. 3 is a flowchart illustrating an operation of extracting a common partial structure in the text entailment recognition apparatus according to the first embodiment of the present invention.
- FIG. 4 is a diagram showing a specific example of the common partial structure extracted in the text entailment recognition apparatus according to the first embodiment of the present invention.
- FIG. 5 is a block diagram illustrating a functional configuration of a text entailment recognition apparatus according to the second embodiment of the present invention.
- FIG. 1 is a block diagram illustrating a functional configuration of a text entailment recognition apparatus according to the first embodiment of the present invention.
- FIG. 2 is a flowchart illustrating the operation of the text entailment recognition apparatus according to the first embodiment of the invention.
- FIG. 6 is a diagram showing a specific example of a partial structure that is treated as the same type in the text implication recognition device according to the second embodiment of the present invention.
- FIG. 7 is a diagram showing a specific example of the common partial structure extracted in the text entailment recognition apparatus according to the second embodiment of the present invention.
- FIG. 8 is a flowchart illustrating an operation of extracting a common partial structure in the text entailment recognition apparatus according to the second embodiment of the present invention.
- FIG. 9 is a block diagram illustrating a functional configuration of a text processing system according to the third embodiment of the invention.
- FIG. 10 is a diagram (1/2) illustrating a specific example in which a specific sentence is expressed by a tree structure in the first embodiment of the present invention.
- FIG. 11 is a diagram (2/2) illustrating a specific example in which a specific sentence is expressed by a tree structure in the first embodiment of the present invention.
- FIG. 12 is a block diagram illustrating a hardware configuration of an information processing apparatus capable of realizing a text entailment recognition apparatus according to each embodiment of the invention of the present application or a component constituting the text processing system.
- the “sentence” may be a single sentence or a compound sentence.
- two or more physically separated devices are communicably connected using a wired or wireless communication or any communication network that combines them. May be configured.
- the meaning of a sentence is determined on the basis of vocabulary elements described in the sentence and a structure composed of relationships between vocabulary elements.
- Non-Patent Document 1 captures the presence or absence (commonality) of common vocabulary elements in a plurality of sentences, while the structure of the sentences. It is difficult to catch the difference in For this reason, as in the technique disclosed in Non-Patent Document 1, many of the methods using coverage in implication recognition have implications for hypothetical sentences that are not implied by the target sentence even though the coverage is high. It may not be judged correctly.
- the relationship between the common partial structures included in the target sentence is different from the relationship between the common partial structures included in the hypothetical sentence. It can be one of the factors. More specifically, for example, when there are multiple common partial structures in the target sentence and the hypothetical sentence, the relationship between the common partial structures in the target sentence is different from the relationship between the common partial structures in the hypothetical sentence. Can be the factor.
- one of the factors is that the partial structures (eg, subtrees) that are descendants of the common partial structure are different (mismatch).
- the dependency relationship represents, for example, a relationship between nodes (or between sub-trees) when a sentence is represented by a tree structure (hereinafter, the same applies in the present application).
- the dependency relationship related to the common partial structure described above may include, for example, a connection relationship in which a specific common partial structure and another common partial structure are directly connected.
- the dependency relationship regarding the common partial structure may include, for example, a connection relationship in which an arbitrary partial structure is interposed between a specific common partial structure and another common partial structure.
- Non-patent document 2 and Non-patent document 3 does not consider the dependency relationship regarding the common partial structure included in the target sentence and the hypothetical sentence.
- the dependency relationship related to the common partial structure can be a factor that affects the implication relationship between sentences. That is, taking into account the dependency relationship related to the common partial structure is effective for appropriate determination of the implication relationship between the target sentence and the hypothetical sentence.
- the target sentence (T1) is “arugula found in supermarkets in recent years tastes like sesame with green-yellow vegetables that have been made smaller in spinach”. Further, it is assumed that the hypothetical sentence (H1) is “the arugula tastes like spinach”.
- the original morpheme representing the meaning of the clause can be listed as the following representative morpheme string “T1-M”. That is, the representative morpheme string “T1-M” is “super, see, arugula, spinach, small, do, vegetables, sesame, taste”. Similarly, the representative morpheme string “H1-M” for the hypothesis sentence H1 is “arugula, spinach, taste”.
- the representative morpheme string “H1-M” all elements are common to any element of the representative morpheme string “T1-M”. That is, the representative morpheme string “T1-M” includes all the elements of the representative morpheme string “H1-M”.
- the hypothetical sentence H1 is completely covered with the target sentence T1.
- the coverage in the representative morpheme of the target sentence T1 and the hypothesis sentence H1 is the maximum (coverage: “1”), but the target sentence T1 does not imply the hypothesis sentence H1.
- Non-Patent Document 1 mainly determines the implication relationship between sentences based on the coverage rate, the implication relation between the target sentence T1 exemplified above and the hypothetical sentence H1. It may be difficult to determine correctly.
- each non-patent document specifies the structure (part) of the sentence that affects the implication relationship between the target sentence and the hypothesis sentence. Explain that this is difficult.
- the tree obtained by converting the target sentence T1 and the hypothetical sentence H1 into a tree structure composed of dependency of each phrase unit based on the Japanese grammar is referred to as a tree T1 and a tree H1.
- the nodes (nodes in the tree structure) constituting the tree T1 and the tree H1 are given morphemes that represent the respective clauses of the respective sentences (T1, H1) as labels.
- each node included in a node string obtained by scanning the tree structure in a pre-order is assigned to the node. It is expressed using the labeled label.
- a subtree composed of descendants (child nodes) including the node with a certain node at the head (subtree root) is represented.
- the tree T1 represented by using the character string by the above method is "(Suru (Ruccola (see (super))) (Taste (sesame (vegetable (spin (spinach)) (small)))))”) Is represented (see FIG. 10).
- the root node of the tree T1 is “(Yes)”, and the child nodes of the root node are “(Arugula)” and “(Taste)” in order from the left.
- the node “(taste)” includes six descendant nodes including nodes constituting the node.
- the tree H1 represented by using a character string is represented as “(S (Ruccola) (Taste (Spinach)))” (see FIG. 11).
- Non-Patent Document 2 an implication relationship between the target sentence T1 and the hypothetical sentence H1 is obtained by using a ratio of the common subtree between the tree T1 and the tree H1 in the tree H1. Identify.
- the non-patent document 2 is based on the hypothesis that the larger the proportion occupied by the common subtree, the more often the target sentence implies a hypothetical sentence.
- Non-Patent Document 2 does not sufficiently take into account the structure of a part different from the common subtree in the target sentence and the hypothetical sentence. If it demonstrates using the said specific example, since the tree H1 is comprised only by the partial tree of the tree T1, the ratio for which a common partial tree occupies in the tree H1 is the largest. However, the target sentence T1 does not imply a hypothetical sentence H1. Thus, the method disclosed in Non-Patent Document 2 may not be able to correctly determine the implication relationship between T1 and H1.
- Non-Patent Document 2 focuses only on the ratio of the common partial structure, it is possible to specify a portion (structure) that affects the implication relationship in the target sentence or hypothesis sentence. Have difficulty.
- the method disclosed in Non-Patent Document 3 uses the following feature quantity as an input of a discriminator used for implication determination by machine learning. That is, the feature amount is a feature amount representing an operation (editing operation) that occurs when the tree structure representing the target sentence is converted into the tree structure representing the hypothetical sentence.
- the tree structure editing operation includes, for example, node replacement, deletion, insertion, and subtree movement.
- the editing operation that occurs when converting the tree T1 to the tree H1 is, for example, four nodes (“(do)”, “(ruccola)”, “(flavor)”, “(spinach)”) in the tree T1. Includes substitution.
- the editing operation is, for example, from the tree T1 to “(see)”, “(super)”, “(sesame)”, “(vegetable)”, “(do)”, “(small)”. Includes operations to delete nodes.
- Non-Patent Document 3 focuses on a common partial structure obtained by moving a partial tree, replacing a node, or the like in a tree structure representing a target sentence and a hypothetical sentence.
- the non-patent document 3 includes a portion (for example, “(sesame (vegetables (to be small)))” in T1) and other portions (for example, No distinction is made between “(see (super))” in T1. For this reason, it is difficult for Non-Patent Document 3 to identify an element that greatly affects the implication relationship for a specific target sentence and a hypothetical sentence.
- Non-Patent Document 2 and Non-Patent Document 3 described above are based on the relationship between the common partial structures in the target sentence and the common partial structures in the hypothetical sentence.
- the difference is not fully considered.
- the difference is that the path between the common sub-trees “(S (Ruccola) (Taste (Spinach)))” and “(Spinach)” in each of the tree T1 and the tree H1. Appears in subtrees that exist in
- Non-Patent Document 2 and Non-Patent Document 3 sufficiently consider the difference between the descendant nodes of the common partial structures in the target sentence and the descendant nodes of the common partial structures in the hypothetical sentence. Not done.
- the difference is in the descendant nodes of the common sub-trees “(S (Ruccola) (Taste (Spinach)))” and “(Spinach)” in each of the tree T1 and the tree H1. appear.
- FIG. 1 is a block diagram illustrating a functional configuration of a text entailment recognition apparatus according to the first embodiment of the present invention.
- the text entailment recognition apparatus 100 is an apparatus that can receive a target sentence and a hypothesis sentence and can determine whether the target sentence implies a hypothesis sentence.
- the text entailment recognition apparatus 100 includes a syntax analysis unit 101, a common partial structure extraction unit 102, a feature amount extraction unit 103, and a determination unit 104.
- the text entailment recognition device 100 in this embodiment may be connected to the input device 105 and the output device 106 so as to be able to communicate with each other using any communication means (such as a communication network or a communication line).
- the syntax analysis unit 101 analyzes the dependency structure of the input hypothesis sentence and target sentence. As a result, the syntax analysis unit 101 converts the hypothesis sentence and the target sentence into a tree structure representing a dependency structure. In this case, for example, the syntax analysis unit 101 may perform morphological analysis, syntax analysis, or the like on the input hypothesis text and target text using a known technique.
- the “tree structure” is a structure in which one or more nodes (nodes) are connected by branches (links, edges). Each node in the tree structure may have a descendant node or an ancestor node.
- a node having a descendant node is an ancestor node when viewed from the descendant node.
- a node having an ancestor node is a descendant node when viewed from the ancestor node.
- Such a tree structure may have a root node that does not have an ancestor node. Note that a part of the tree structure constituted by any one or more nodes constituting the tree structure may be referred to as a partial tree.
- Such a “tree structure” can be realized using a known general data structure or the like.
- the dependency structure of the hypothesis sentence and the target sentence is a structure representing a relationship between elements (for example, morphemes, words, phrases, etc.) obtained by decomposing each sentence.
- Japanese sentence based on a Japanese grammar is an example of analyzing a Japanese sentence based on a Japanese grammar, but the present embodiment is not limited to this.
- any language other than Japanese can be used as long as it is a language that can analyze a sentence structure (such as syntax and dependency relationship).
- the syntax analysis unit 101 represents, for example, a sentence-by-phrase dependency structure, such as “The arugula that has been found in supermarkets in recent years tastes like sesame with green-yellow vegetables that have been made smaller in spinach.” Convert to a tree structure. Then, the syntax analysis unit 101 assigns the original form of the morpheme that semantically represents the phrase to each node of the tree structure as a label. In this case, the above-described example sentence is converted into “(suru (lucula (see (super))) (taste (sesame (vegetable (spin (spinach)) (small)))))))))))”.
- the common partial structure extraction unit 102 extracts a common partial structure (common partial structure) between the tree structure representing the target sentence converted by the syntax analysis unit 101 and the tree structure representing the hypothetical sentence.
- the common partial structure may be a partial tree in the tree structure representing each sentence.
- the common partial structure extraction unit 102 selects, for example, the common partial structures in order from the common partial structure having the largest number of nodes so as not to overlap.
- the common partial structure extraction unit 102 for example, a tree structure “(S (Ruccola) (taste))” representing a target sentence and a tree structure “(S (Ruccola) ( "(Sol (Ruccola) (Taste))” is extracted as a common partial structure.
- the feature quantity extraction unit 103 can determine an implication relationship between the target sentence and the hypothetical sentence using the tree structure representing the target sentence, the tree structure representing the hypothetical sentence, and the common partial structure thereof. To extract.
- the feature amount in the present embodiment is information regarding a common partial structure in a structure (for example, a tree structure) representing a target sentence or a hypothetical sentence.
- the information related to the common partial structure includes information representing the dependency relationship between one or more common partial structures in the structure representing the target sentence or the hypothetical sentence, and the dependency relationship between the common partial structure and another partial structure. And at least one of the information to be expressed.
- the information representing the dependency relationship between the common partial structures may include, for example, information representing a connection relationship between the common partial structure and another partial structure existing between one or more common partial structures.
- the information indicating the dependency relationship between the common partial structures includes information on the other partial structures themselves (number of nodes included in the tree structure, node attributes, etc.) existing between one or more common partial structures. But you can.
- the information representing the dependency relationship between the common partial structure and the other partial structure may include, for example, information representing the connection relationship between the common partial structure and the other partial structure. Further, the information representing the dependency relationship between the common partial structure and the other partial structure may include information on the other partial structure itself (number of nodes included in the tree structure, node attributes, and the like).
- the feature amount extraction unit 103 may extract, for example, a difference (match or mismatch) in the dependency relationship between the common partial structures in the target sentence and the hypothesis sentence as the feature amount.
- the feature amount extraction unit 103 may extract, for example, the number of nodes and the attributes of the nodes in the partial structure formed between the partial structures existing between the common partial structures and the descendants of the common partial structures as the feature amounts .
- the tree structure representing the target sentence is “(S (Ruccola) (Taste (Sesame))”, and the tree structure representing the hypothetical sentence “(S (Ruccola) (Taste (Spinach)))” ”.
- the common partial structures in the tree structure representing the target sentence and the tree structure representing the hypothetical sentence are “(do (rukkola))” and “(taste)”.
- the feature amount extraction unit 103 uses, for example, a tree structure that represents the target sentence, a tree structure that represents the hypothetical sentence, and a common partial structure in the tree structure that represents the target sentence and the tree structure that represents the hypothetical sentence. Extract features.
- the feature quantity extraction unit 103 extracts, for example, a feature quantity that “the number of mismatches in the dependency relationship in the common partial structure is 0”.
- the feature quantity extraction unit 103 extracts a feature quantity such as “the total number of nodes in the partial structure between the common partial structures in the target sentence is 0”, for example.
- the feature quantity extraction unit 103 extracts, for example, a feature quantity that “the total number of nodes in the partial structure between the common partial structures in the hypothetical sentence is zero”.
- the feature quantity extraction unit 103 extracts a feature quantity such as “the number of predicate nodes in the partial structure between the common partial structures in the target sentence is 0”, for example.
- the “predicate node” indicates, for example, a node in which a predicate is included in a clause that is an attribute of the node.
- the feature quantity extraction unit 103 extracts, for example, a feature quantity that “the number of predicate nodes in the partial structure between the common partial structures in the hypothesis sentence is zero”.
- the feature quantity extraction unit 103 extracts a feature quantity, for example, “the number of mismatches between partial structures consisting of descendants of the common partial structure is one”.
- the feature amount extraction unit 103 extracts, for example, a feature amount “the number of differences between (spinach) and (sesame) is 1 in the descendants of the common partial structure”.
- the feature amount extraction unit 103 is not limited to the above, and may extract other arbitrary feature amounts.
- the determination unit 104 determines whether the target sentence implies a hypothetical sentence based on the feature amount extracted (output) by the feature amount extraction unit 103 and a predetermined identification rule.
- the determination unit 104 uses, for example, a well-known pattern recognition technique such as a support vector machine, based on the feature vector composed of the feature amount extracted from the target sentence and the hypothetical sentence, and the identification model. The implication relationship between the sentence and the hypothesis sentence may be determined.
- a well-known pattern recognition technique such as a support vector machine
- the determination unit 104 uses the feature vector composed of the feature amount extracted from the target sentence and the hypothesis sentence and the identification hyperplane based on the identification model, And implications between hypothesis sentences can be determined. More specifically, for example, the determination unit 104 determines that the target sentence implies the hypothesis sentence when the feature vector exists on the implication side in the hyperspace divided by the identification hyperplane. Also good.
- the input device 105 is an arbitrary device that can input a target sentence and a hypothetical sentence to the text implication recognition apparatus 100.
- the input device 105 may be, for example, an input device itself such as a keyboard and a mouse, or may be an information processing device to which these input devices are connected. In this case, the input device 105 may transmit the target sentence, the hypothesis sentence, and the like input via the input device to the text entailment recognition apparatus 100.
- the input device 105 is not limited to the above, and may be an arbitrary database holding a target sentence and a hypothetical sentence, a file server, or the like.
- the output device 106 is an arbitrary device that can output the result of the implication recognition in the text implication recognition device 100.
- the output device 106 may be an output device itself such as a display or a printer, or may be an information processing device to which these output devices are connected. In this case, the output device 106 may accept (receive) an implication recognition result from the text implication recognition device 100 and output the result to the output device.
- the communication means for connecting the input device 105, the output device 106, and the text entailment recognition device 100 may employ well-known techniques as appropriate, and will not be described in detail.
- the syntax analysis unit 101 parses each of the target sentence and the hypothetical sentence, and converts it into a tree structure that can represent the dependency (step S201).
- a tree structure capable of representing such a dependency relationship may be referred to as a “dependency structure”.
- the syntax analysis unit 101 performs morphological analysis on the input target sentence and hypothesis sentence, and executes syntax analysis using the result.
- the syntax analysis unit 101 converts the input sentence into a tree structure based on the dependency relationship between the clauses.
- the syntax analysis unit 101 assigns a label to each unit (node constituting the tree structure) of the dependency structure, and generates a labeled tree structure (step S202).
- the syntax analysis unit 101 divides a sentence “Arugula tastes like spinach” into a phrase “Arugula tastes like spinach.” Then, the syntax analysis unit 101 assigns, to each clause, a morpheme that semantically represents the clause as a label. Next, the syntax analysis unit 101 converts the sentence into a tree structure of “(do (rukkola) (taste (spinach))” using the dependency relationship between the divided clauses and the label of the clause. . At this time, the syntax analysis unit 101 sets each clause (or a morpheme representing the relevant clause) divided as described above as an attribute of each node constituting the tree structure.
- the common partial structure extracting unit 102 uses the tree structure representing the target sentence generated in step S202 and the tree structure representing the hypothetical sentence, and the same type of partial structure candidate common to them (hereinafter, “common partial structure”). May be referred to as “candidate”). Then, the common partial structure extraction unit 102 extracts the common partial structure from the detected zero or more common partial structure candidates from the one having the largest number of nodes so as not to overlap (step S203).
- step S203 will be described in detail with reference to FIG.
- FIG. 3 is a flowchart showing a specific example of the operation of extracting the common partial structure in the text entailment recognition apparatus 100 according to the first embodiment of the present invention.
- the common partial structure extraction unit 102 first initializes an array that stores the extracted common partial structure as an element (step S301).
- the common partial structure extraction unit 102 selects nodes one by one from all the nodes constituting the labeled tree structure representing the target sentence, and repeats the processing of steps S303 to S307 below (steps S302 to S307). Step S308).
- the common partial structure extraction unit 102 searches for a node having the same label as the node selected in step S301 from the labeled tree structure representing the hypothetical sentence (step S303).
- the common partial structure extraction unit 102 returns to step S302 and continues the process, Select a node.
- the common partial structure extraction unit 102 sets the node selected in step S301 as the root node from the target sentence.
- a partial structure for example, a partial tree
- the common partial structure extraction unit 102 extracts the partial structure (in the hypothesis sentence) having the node detected in step S303 as a root node (in the hypothesis sentence) and the partial structure copied from the target sentence in step S305 from the root node. Compare by hierarchy. Then, the common partial structure extraction unit 102 deletes a different (inconsistent) node in each partial structure from the partial structure copied in step S305 (step S306). More specifically, the common partial structure extraction unit 102 uses a node and its descendant that do not match between the partial structure whose root node is the node detected in step S303 and the partial structure copied in step S305. Extract a subtree. Then, the common partial structure extraction unit 102 deletes the extracted partial tree from the partial structure copied in step S305.
- the common partial structure extraction unit 102 creates the partial structure (common partial structure candidate) obtained in step S301 as a result of deleting the mismatched nodes and the like from the partial structure copied in step S305 (step S306). (Step S307).
- the common partial structure extraction unit 102 returns to step S301 and selects the next node from the tree structure representing the target sentence (step S301).
- the common partial structure extraction unit 102 selects one of the other elements among all the elements (common partial structure candidates) stored in the array created in step S301.
- An element that overlaps a part (for example, a subtree or a node) is deleted (step S309).
- the common partial structure extraction unit 102 determines that the root node of the common partial structure candidate stored in the specific element of the array is the root node of the common partial structure candidate stored in the other element of the array. If it is included, the specific element is deleted from the array.
- the extracted common partial structure is stored in an array, but the present embodiment is not limited to this.
- the common partial structure extraction unit 102 may hold the extracted common partial structure using an arbitrary data structure.
- FIG. 4 is a diagram showing a specific example of the common partial structure extracted in the text entailment recognition apparatus according to the first embodiment of the present invention.
- the target sentence T1 is “arugula found in a supermarket in recent years tastes like sesame with green-yellow vegetables that are made by reducing spinach”. Further, it is assumed that the hypothetical sentence H1 is “the arugula tastes like spinach”.
- the tree T1 representing the target sentence T1 is “(Sure (Arugula (see (super))) (Taste (sesame (vegetable (spinach) (small))))”).
- the tree H1 representing the hypothetical sentence H1 is “(do (ruccola) (taste (spinach))”.
- the common partial structure extraction unit 102 extracts common partial structure candidates in order from the largest number of nodes included in the common common structure included in the tree T1 and the tree H1. As a result, the partial structure C1 “(S (Ruccola) (taste))” and the partial structure C2 “(Spinach)” are extracted as a common partial structure.
- the feature amount extraction unit 103 uses the common partial structure extracted by the common partial structure extraction unit 102 and the tree structure obtained by converting the target sentence and the hypothetical sentence.
- the amount is extracted (step S204).
- the feature amount is used when determining an implication relationship between the target sentence and the hypothetical sentence.
- the feature amount extraction unit 103 may extract the feature amount based on the dependency relationship between one or more common partial structures in the target sentence and the hypothesis sentence.
- the feature amount extraction unit 103 may extract the feature amount based on the dependency between the common partial structure in the target sentence and the hypothetical sentence and another partial structure different from the common partial structure.
- the feature amount extraction unit 103 is based on the dependency relationship between the one or more common partial structures in the target sentence and the hypothetical sentence, and the dependency relationship between the common partial structure and the other partial structures.
- the feature amount is extracted based on at least one of them.
- the dependency relationship between one or more common partial structures may include a dependency relationship between a common partial structure in each sentence and a partial structure interposed between the common partial structures.
- the dependency relationship between one or more common partial structures may include the structure of another partial structure itself interposed between the common partial structures in each sentence, the attributes of nodes included in the other partial structures, and the like. Good.
- the dependency between the common partial structure and another partial structure different from the common partial structure is that the descendants (subtrees or nodes) of the common partial structure that do not intervene between the common partial structures in each sentence.
- a structure, an attribute, etc. may be included.
- the feature quantity extraction unit 103 detects a difference between the relations between the common part structures included in the target sentence and the relations between the common parts included in the hypothesis sentence. It may be output.
- the feature quantity extraction unit 103 may extract feature quantities from partial structures existing between the common partial structures.
- the feature amount extraction unit 103 may extract a feature amount from a partial structure that is a descendant of each common partial structure.
- the target sentence is the target sentence T1 and the hypothesis sentence is the hypothesis sentence H1.
- the subtree (hereinafter referred to as “partial tree A1”) constituted by the path between the subtree C1 and the subtree C2 in the tree T1 is “(sesame (vegetable))”.
- the tree H1 there is no other partial structure (subtree) between the subtree C1 and the subtree C2.
- a subtree existing in a path between a subtree C1 (partial structure C1) and a subtree C2 (partial structure C2) in the tree T1 (hereinafter referred to as “subtree C2”).
- (Referred to as “partial tree X1”) may be “(sesame (vegetables (to (small))))”.
- the feature quantity extraction unit 103 extracts the feature quantity using, for example, the following subtrees. That is, the partial tree used for extracting the feature amount may be the partial tree A1. In addition, the subtree used for extracting the feature quantity may be a subtree that is a descendant of the subtree C1 or the subtree C2 in the tree H1 and is not between the common partial structures. Further, the subtree used for extracting the feature amount may be a subtree that is a descendant of the subtree C1 or the subtree C2 in the tree T1 and is not between the common partial structures.
- the feature amount is, for example, one or more of the number of nodes included in the subtree, labels set in each node constituting the subtree, and clause components set as attributes in each node. May be included.
- the feature amount may include the number of nodes from the root node to the subtree C1 or the subtree C2 for the tree T1 or the tree H1.
- the feature amount relates to a dependency relationship between the subtree C1 and the subtree C2 in the tree T1, or a difference (match or mismatch) in the dependency relationship between the subtree C1 and the subtree C2 in the tree H1. Information may be included.
- the feature quantity extraction unit 103 is based on the dependency relationship (connection relationship) between A1 and C1 (or C2) existing between the common partial structures (between C1 and C2) in the tree T1. An arbitrary feature amount may be extracted. In addition, the feature quantity extraction unit 103 may extract an arbitrary feature quantity from the structure of A1 itself, the attributes of the nodes constituting A1, and the like.
- the feature quantity extraction unit 103 is a descendant of the subtree C1 or the subtree C2 in the tree T1, and a dependency relationship (connection relationship) between each subtree that is not between the common partial structures and C1 (or C2). Based on the above, an arbitrary feature amount may be extracted.
- the subtrees that are descendants of the subtree C1 or the subtree C2 in the tree T1 and that are not between the common partial structures are, for example, (see (super)) and (small).
- the feature amount extraction unit 103 may extract feature amounts for all combinations thereof. Further, the feature amount extraction unit 103 may extract feature amounts for combinations based on the dependency structure of the common partial structure in the tree T1.
- the determination unit 104 checks whether or not the feature vector composed of the feature amount extracted by the feature amount extraction unit 103 in step S204 satisfies a specific identification condition (step S205).
- step S205 If the identification condition is satisfied in step S205 (YES in step S205), the determination unit 104 determines that the target sentence implies a hypothetical sentence (step S206).
- step S205 If the identification condition is not satisfied in step S205 (NO in step S205), the determination unit 104 determines that the target sentence does not imply a hypothetical sentence (step S207).
- the determination unit 104 may determine an implication relationship between the target sentence and the hypothetical sentence using a support vector machine as a determination condition in steps S205 to S207, for example. In this case, for example, when the feature vector extracted in step S204 belongs to the implication side in the space divided by the identification hyperplane, the determination unit 104 determines that the target sentence implies a hypothetical sentence. The determination unit 104 determines that the target sentence does not imply a hypothetical sentence when the feature vector extracted in step S204 belongs to the other space (not the implication side) among the spaces divided by the identification hyperplane. . Note that the determination unit 104 may use a discriminator other than the support vector machine.
- the text entailment recognition apparatus 100 can correctly determine the implication relationship when the target sentence does not imply a hypothesis sentence with a high coverage rate.
- a description will be given of a method for extracting a feature quantity that identifies a part that affects an implication relationship between sentences by the text implication recognition apparatus 100 according to the present embodiment.
- the common partial structure of the tree T1 representing the target sentence T1 and the tree H1 representing the hypothetical sentence H1 is the partial tree C1 and the partial tree C2.
- the feature quantity extraction unit 103 refers to the subtree A1 and extracts a feature quantity from the attributes of the nodes constituting the subtree A1. More specifically, the feature amount extraction unit 103 determines, for example, a feature amount such as “there is one case particle“ de ”” from the phrase “green-yellow vegetable” that is an attribute of the node “(vegetable)”. To extract. That is, in the target sentence T1, a case particle appears in a partial structure (specifically, a clause that is an attribute of the partial structure) existing between the common partial structures (that is, between C1 and C2). In this case, the relationship between the common partial structure that is a descendant of the partial structure and the common partial structure that is an ancestor of the partial structure changes.
- the determination unit 104 sets the above-mentioned relationship by setting a condition that “if one or more case particles“ de ”are included in subtrees between common partial structures, it is determined as non-entailed” in the identification condition. It is possible to capture changes.
- the determination unit 104 can correctly determine that the target sentence T1 is non-implication of the hypothesis sentence H1. That is, based on the feature amount extracted by the feature amount extraction unit 103 (for example, “a case particle appears in the subtree existing between the common partial structures C1 and C2”) and the identification condition, the determination unit 104 Can determine the implication relationship between the target sentence T1 and the hypothesis sentence H1.
- the identification condition is not limited to the presence or absence of the case particle “de”.
- Such an identification condition may be, for example, a condition for designating a position at which an object (node or node attribute or the like) to be extracted as a feature amount appears in a sentence using a relationship with a common partial structure.
- Such an identification condition may be, for example, a condition for designating a relative position between a target position to be extracted as a feature quantity and the common partial structure.
- identification condition is an example of interpreting a Japanese sentence based on a Japanese grammar, but the present embodiment is not limited to this.
- Such an identification condition may be appropriately determined based on, for example, a grammar related to the language for each language of the target sentence. That is, the operation of the determination unit 104 described above can be applied regardless of language (for example, applicable to other languages regardless of Japanese).
- these identification conditions may not be given explicitly. That is, in the text implication recognition apparatus 100 according to the present embodiment, the determination unit 104 can learn the identification condition by creating an identification model using well-known machine learning.
- the text entailment recognition apparatus 100 uses the target sentence and the hypothetical sentence to which a label (“entailment” or “non-entailment”) is assigned, and a feature amount used for identification processing (entailment determination) for those sentences. It is possible to generate an identification model by preparing it as teacher data and executing well-known machine learning. Note that the identification model may be generated in advance by the user of the text implication recognition device 100 or the like and set for the text implication recognition device 100, for example. By adopting such a method using machine learning, it is possible to generate an identification model regardless of the language of the target sentence (that is, not limited to a specific language such as Japanese). It is.
- the determination unit 104 can determine whether or not the identification condition is satisfied in step S205 based on the identification model. That is, the determination unit 104 can identify an implication relationship for a set of unlabeled target sentences and hypothesis sentences using an identification model learned in advance.
- learning of the identification condition may affect the implication relationship between the target sentence and the hypothetical sentence included in a certain corpus.
- a feature quantity is specified. For example, whether the feature quantity affects the implication relation based on the result of determining the implication relation between the target sentence and the hypothetical sentence using an identification model generated by performing machine learning using a certain feature quantity. It is possible to specify whether or not.
- the text entailment recognition apparatus 100 can specify a partial structure that affects the implication relationship based on the feature amount.
- the feature amount extraction unit 103 pays attention to the common partial structure included in the tree structure representing the target sentence and the tree structure representing the hypothetical sentence, and features from the relationship between the common partial structures. Extract the amount.
- the feature amount extraction unit 103 extracts feature amounts based on the relationship between the descendant nodes of the common partial structure (for example, the difference between the descendant nodes, etc.) in each tree structure representing the target sentence and the hypothetical sentence.
- the determination unit 104 in the present embodiment identifies an implication relationship between the target sentence and the hypothetical sentence based on these feature quantities and the identification conditions related to these feature quantities. For this reason, the determination unit 104, for example, corrects an implication for a hypothesis sentence that has a high coverage with respect to the target sentence but is not implied by the target sentence, based on a feature amount such as a degree of difference regarding a common partial structure between the sentences. The relationship can be determined.
- the text entailment recognition apparatus 100 extracts the feature amount extracted from the relationship between the common partial structures in each tree structure representing the target sentence and the hypothetical sentence and the difference between the descendant nodes of the common partial structure. It is possible to distinguish the feature amount.
- the text entailment recognition apparatus 100 determines whether the relationship between the common partial structures in the target sentence and the hypothetical sentence affects the implication relation, or the descendant nodes of each common partial structure are in the implication relation. It is possible to distinguish whether it affects. Therefore, the text implication recognition device 100 according to the present embodiment can specify a partial structure that affects the implication relationship.
- the common partial structure extraction unit 102 in the present embodiment selects a common partial structure from each tree structure obtained by converting the target sentence and the hypothetical sentence, the common partial structure extraction unit 102 overlaps in descending order of the number of nodes included in the common partial structure. Choose not to. For example, it is assumed that a plurality of nodes having a common label appear in a tree structure representing the target sentence and a tree structure representing the hypothetical sentence (for example, when the coverage is high). In such a case, the text entailment recognition apparatus 100 according to the present embodiment can suppress an increase in the number of combinations of common partial structures due to the subdivision of the common partial structures. Thereby, the text entailment recognition apparatus 100 according to the present embodiment can ignore the relationship between the common partial structures that does not affect the implication relationship.
- the text entailment recognition apparatus 100 in the present embodiment it is possible to appropriately determine the implication relationship between the target sentence and the hypothetical sentence by using the feature amount that is information representing the structure of the sentence. . More specifically, the text implication recognition device 100 according to the present embodiment can appropriately determine an implication relationship between sentences with a high coverage, for example. In addition, the text implication recognition device 100 according to the present embodiment can specify, for example, a portion that affects an implication relationship in a structure representing each sentence.
- the text entailment recognition device 100 in the present embodiment described above may be configured as a single device (information processing device or the like) including all the components.
- the text entailment recognition apparatus 100 uses a device (a physical information processing device, a virtual information processing device, or the like) in which one or more components are physically or logically separated. It may be configured as a realized system. In this case, a plurality of such devices may be communicably connected using any communication network that is wired or wireless or a combination thereof. When the plurality of devices are configured by virtual information processing devices or the like, the communication network may be a virtual communication network.
- FIG. 5 is a block diagram illustrating a functional configuration of the text implication recognition device 100 according to this embodiment.
- the basic configuration of the text entailment recognition apparatus 100 in the present embodiment is the same as that in the first embodiment. More specifically, the common partial structure extraction method by the common partial structure extraction unit 102 in the present embodiment is different from that in the first embodiment. Other components may be the same as those in the first embodiment.
- the text entailment recognition apparatus 100 in this embodiment is connected to the isomorphic information holding unit 107 so as to be communicable.
- the common partial structure extraction unit 102 may be communicably connected to the same-type information holding unit 107.
- the isomorphic information holding unit 107 is a storage device that holds the isomorphic information dictionary 107a.
- the isomorphic information holding unit 107 may be a storage device itself that provides an arbitrary database, a file system, or the like, or may be an information processing device provided with the storage device. Such a storage device may be realized by appropriately adopting a known technique such as a hard disk drive (HDD) or a nonvolatile semiconductor storage device.
- the isomorphic information holding unit 107 may be configured as an element independent of the text implication recognition device 100 (for example, FIG. 5) or may be configured as a part of the text implication recognition device 100.
- the isomorphic information dictionary 107a holds information that associates a specific partial structure (partial tree) with another partial structure that is treated as the same as the specific partial structure. Specifically, the isomorphic information dictionary 107a holds one or more pairs of a specific partial structure and another partial structure treated as the same as the specific partial structure.
- the common partial structure extraction unit 102 can replace these partial structures with each other. That is, when a specific partial structure and another partial structure are registered as a pair in the isomorphic information dictionary 107a, the common partial structure extraction unit 102 treats these partial structures as virtual isomorphous partial structures. It is possible. In the isomorphic information dictionary 107a, a pair of partial structures treated as the same may be registered in advance.
- the common partial structure extraction unit 102 in the present embodiment refers to information on partial structures (pairs) that are registered in the isomorphic information dictionary 107a and are treated as the same when extracting the common partial structure.
- the partial structure pair P1 “(kake (curry) (rice)), (curry and rice)” is registered in the isomorphic information dictionary 107a.
- the target sentence T2 is “I want to eat curry with rice” and the hypothetical sentence H2 is “I want to eat curry rice.”
- the tree T2 in which the target sentence T2 is represented by a tree structure is “(eat (powder (curry) (rice))”.
- the tree H2 in which the hypothetical sentence H2 is represented by a tree structure is “(eat (curry and rice))”.
- the common partial structure extracted by the common partial structure extraction unit 102 in the first embodiment is only “(eat)”.
- the common partial structure extraction unit 102 uses the information in the isomorphic information dictionary 107a to make the partial structure pair P1 “(kake (curry) (rice)), (curry and rice)” the same. Treat as a partial structure.
- the text entailment recognition apparatus 100 in the present embodiment executes steps S201 and S202 in the flowchart illustrated in FIG. 2 as in the first embodiment. Since these processes may be the same as those in the first embodiment, detailed description thereof is omitted.
- the common partial structure extraction unit 102 detects a candidate of the same type and a common partial structure from the tree structure generated in step S202.
- the extraction process of the common partial structure by the common partial structure extraction unit 102 will be described with reference to the flowchart illustrated in FIG.
- the common partial structure extraction unit 102 in the present embodiment refers to the isomorphic information dictionary 107a stored in the isomorphic information holding unit 107 when specifying a partial structure having the same type. As described above, in the isomorphic information dictionary 107a stored in the isomorphic information holding unit 107, one or more pairs of partial structures that are regarded as identical to a certain partial structure are registered.
- the common partial structure extraction unit 102 searches the isomorphic information dictionary 107a stored in the isomorphic information holding unit 107 for a partial structure having each node of the tree structure representing a hypothetical sentence as a root node (step S801). Then, the common partial structure extraction unit 102 extracts a partial structure treated as the same as the partial structure in the hypothesis sentence from the isomorphic information dictionary 107a. In this case, the common partial structure extraction unit 102 may extract a list of partial structures that are treated as being the same as a certain partial structure included in the hypothesis sentence.
- the common partial structure extraction unit 102 replaces the partial structure included in the tree structure representing the hypothesis sentence with the elements included in the list for all elements (subtrees) included in the list. Extract. Then, the common partial structure extraction unit 102 generates a tree structure list in which the partial structures included in the tree structure representing the hypothesis text are replaced for all the extracted combinations (step S802).
- the combination may include both a case where the partial structure included in the tree structure representing the hypothetical text is replaced with another partial structure treated as the same as the partial structure and a case where the partial structure is not replaced.
- the common partial structure extraction unit 102 selects elements one by one from all the elements (tree structures representing hypothetical sentences) included in the tree list generated in step S802 (step S803), and performs the following processing. Repeat (Steps S803 to S805).
- the common partial structure extraction unit 102 extracts a common partial structure between the element selected in step S803 (a tree structure representing a hypothetical sentence) and the target sentence (step S804).
- the specific operation in step S804 may be the same as the common partial structure extraction operation (the flowchart illustrated in FIG. 3) described in the first embodiment. For this reason, detailed description is omitted.
- step S805 When the above processing for all elements included in the list extracted in step S802 is completed (step S805), the common partial structure extraction unit 102 continues the processing from step S806.
- the common partial structure extraction unit 102 refers to the common partial structure extracted in step S804. Then, the common partial structure extraction unit 102 has the smallest number of nodes that are not common partial structures (not included in the common partial structures) among all tree structures representing hypothetical sentences included in the list extracted in step S802. A certain tree structure is selected (step S806).
- the common partial structure extraction unit 102 confirms the common partial structure between the target sentence extracted in step S804 for all tree structures representing hypothetical sentences included in the list extracted in step S802. .
- the common partial structure extraction unit 102 then minimizes the number of nodes other than the common partial structure among the tree structures representing the hypothetical sentences included in the list extracted in step S802 (that is, the number of nodes included in the common partial structure). Is selected).
- the common partial structure extraction unit 102 outputs the tree structure representing the selected hypothetical sentence, the tree structure representing the target sentence, and the selected common partial structure.
- the text entailment recognition apparatus 100 in this embodiment may continue the processing from step S204 illustrated in FIG. 2 after step S806 as in the first embodiment. Since the processing after step S204 may be the same as that in the first embodiment, detailed description thereof is omitted.
- the common partial structure extraction unit 102 uses partial structure pair information registered in the isomorphic information dictionary 107a. As a result, the common partial structure extraction unit 102 replaces the partial structure included in the hypothesis sentence with another partial structure that is treated as the same as the partial structure. That is, the common partial structure extraction unit 102 in this embodiment can transform the tree structure representing the hypothetical sentence by replacing at least a part of the tree structure representing the hypothetical sentence with another partial structure.
- the common partial structure extraction unit 102 can extract the common partial structure even when, for example, a notation fluctuation exists between the target sentence and the hypothetical sentence.
- the text entailment recognition apparatus 100 can extract feature quantities using a unit (partial structure) of an expression that represents the same meaning even if the notation is different, and can make an implication determination using the feature quantities. It is.
- the text entailment recognition apparatus 100 in the present embodiment has the same configuration as that of the first embodiment, the same effects as those of the first embodiment can be obtained.
- FIG. 9 is a block diagram illustrating a functional configuration of a text processing system 900 according to the third embodiment of the invention.
- the text processing system 900 includes a common partial structure extraction unit (common partial structure extraction unit) 901, a feature amount extraction unit (feature amount extraction unit) 902, and a determination unit (determination unit) 903.
- the text processing system 900 according to the present embodiment may be configured by a single device (information processing device or the like) including these components.
- the text processing system 900 according to the present embodiment is realized by using a plurality of devices (physical information processing devices, virtual information processing devices, etc.) in which these components are physically or logically separated. It may be configured as a system. In this case, a plurality of such devices may be communicably connected using any communication network that is wired or wireless or a combination thereof.
- the communication network may be a virtual communication network.
- the common partial structure extraction unit 901 extracts a common partial structure that is a structure of the same type of parts common to the structure representing the first sentence and the structure representing the second sentence.
- the first sentence may be the same as the target sentence in each of the above embodiments, and the second sentence may be the same as the hypothetical sentence in each of the above embodiments.
- the structure representing the first sentence may be a tree structure generated from the result of executing morphological analysis, dependency structure analysis, etc. on the first sentence, as in the above embodiments. .
- the structure representing the second sentence is a tree structure generated from the result of executing morphological analysis, dependency structure analysis, etc. on the second sentence, as in the above embodiments. May be.
- the common partial structure may be a partial tree or a node common to a tree structure representing the first sentence and a tree structure representing the second sentence.
- the common partial structure extraction unit 901 in this embodiment may function as a common partial structure extraction unit that can realize the same function as the common partial structure extraction unit 102 in each of the above embodiments, for example.
- the feature amount extraction unit 902 is different from the feature amount based on the dependency relationship between the one or more common partial structures, the common partial structure, and the common partial structure in each of the first and second sentences. At least one of the feature quantity based on the dependency between the partial structures is extracted.
- the feature amount extraction unit 902 includes a dependency relationship between one or more common partial structures in the first sentence, a dependency relationship between one or more common partial structures in the second sentence, A feature amount based on the above may be extracted.
- the feature quantity extraction unit 902 may extract feature quantities by the following method. That is, the feature quantity extraction unit 902 extracts, for example, a dependency relationship (first dependency relationship) between the common partial structure and a partial structure different from the common partial structure in the first sentence. . Further, the feature amount extraction unit 902 extracts a dependency relationship (second dependency relationship) between the common partial structure and a partial structure different from the common partial structure in the second sentence. The feature amount extraction unit 902 includes a feature amount based on the first dependency relationship and the second dependency relationship (a feature amount based on a difference between the first dependency relationship and the second dependency relationship). , May be extracted.
- the feature quantity extraction unit 902 in this embodiment may function as a feature quantity extraction unit that can realize the same function as the feature quantity extraction unit 103 in each of the above embodiments.
- the determination unit 903 determines an implication relationship between the first sentence and the second sentence by using the extracted feature amount. More specifically, the determination unit 903 determines between the first sentence and the second sentence based on the feature amount extracted by the feature amount extraction unit 902 and a specific identification condition regarding the feature amount. The implication relationship may be determined.
- the determination unit 903 in the present embodiment may function as a determination unit that can realize the same function as the determination unit 104 in each of the above embodiments.
- the text processing system 900 in the present embodiment extracts the common partial structure in each structure representing the first sentence and the second sentence. Then, the text processing system 900 extracts a feature amount from the relationship between the common partial structures or the relationship between the common partial structure and another partial structure.
- the text processing system 900 determines an implication relationship between the first sentence and the second sentence based on, for example, these feature quantities and identification conditions for the feature quantities. That is, the text processing system 900 according to the present embodiment configured as described above can determine an implication relationship between two or more texts (documents), as with the text implication recognition device 100 in each of the above embodiments.
- the text processing system 900 of the present embodiment by using the feature amount representing the structure of the sentence (first sentence and second sentence) that is the object of the implication relation determination, between the sentences. Implication relations can be determined. More specifically, the text processing system 900 in the present embodiment can appropriately determine an implication relationship between sentences with a high coverage. In addition, the text processing system 900 according to the present embodiment can specify a portion that affects the implication relationship in the structure representing each sentence.
- each component of the text processing system 900 may be collectively referred to as “text processing system or the like”.
- each component of the text processing system 900 may be realized by a single device, and a plurality of different components. It may be realized by an apparatus.
- each unit shown in each of the above drawings may be realized as hardware (an integrated circuit or the like on which processing logic is mounted) that is partially or wholly integrated.
- the text processing system or the like may be configured by hardware as illustrated in FIG. 12 and various software programs (computer programs) executed by the hardware.
- the arithmetic device 1201 in FIG. 12 is an arithmetic processing device such as a general-purpose CPU (Central Processing Unit) or a microprocessor.
- the arithmetic device 1201 may read various software programs stored in a nonvolatile storage device 1203, which will be described later, into the storage device 1202, and execute processing according to the software programs.
- the storage device 1202 is a memory device such as a RAM (Random Access Memory) that can be referred to from the arithmetic device 1201, and stores software programs, various data, and the like.
- the storage device 1202 may be a volatile memory device.
- the non-volatile storage device 1203 is a non-volatile storage device such as a ROM (Read Only Memory) or a flash memory using a semiconductor storage device, and may record various software programs and data.
- ROM Read Only Memory
- flash memory using a semiconductor storage device
- the drive device 1204 is, for example, a device that processes reading and writing of data with respect to a storage medium 1205 described later.
- the storage medium 1205 is an arbitrary recording medium capable of recording data, such as an optical disk, a magneto-optical disk, and a semiconductor flash memory.
- the network interface 1206 is an interface device that connects the text processing system and the like to a wired or wireless network and an arbitrary communication network configured by combining them.
- the text implication recognition device 100 or the text processing system 900 in each of the above embodiments is configured by a combination of a plurality of devices configured by hardware illustrated in FIG. They may be connected to each other via an interface 1206 so that they can communicate with each other.
- the input / output interface 1207 is an interface to which an input device that accepts various inputs to the text processing system and an output device that accepts various outputs from the text processing system and the like are connected.
- the input device 105 and the output device 106 in the first and second embodiments may be connected to the text entailment recognition device 100 via the input / output interface 1207.
- the present invention described by taking each of the embodiments described above as an example includes, for example, a text processing system configured by the hardware device illustrated in FIG. 12, and the functions described in each of the above embodiments for the text processing system. May be realized by supplying a software program capable of realizing the above. In this case, the present invention may be achieved by the arithmetic device 1201 executing the software program supplied to the text processing system or the like.
- each unit shown in each of the above drawings can be realized as a software module, which is a function (processing) unit of a software program executed by the hardware described above.
- the division of each software module shown in these drawings is a configuration for convenience of explanation, and various configurations can be assumed for implementation.
- these software modules are stored in the nonvolatile storage device 1203, and the arithmetic device 1201 performs each processing. These software modules may be read out to the storage device 1202 when executing.
- these software modules may be configured to transmit various data to each other by an appropriate method such as shared memory or interprocess communication. With such a configuration, these software modules can be connected so as to communicate with each other.
- each software program may be recorded in the storage medium 1205.
- the text processing system may be configured so that the software program is stored in the non-volatile storage device 1203 through the drive device 1204 as appropriate at the shipping stage or the operation stage.
- the text is used by using an appropriate jig.
- a method of installing in a processing system or the like may be employed.
- the supply method of various software programs may be a generally used method such as a method of downloading from the outside via a communication line such as the Internet.
- the present invention can be understood to be constituted by a code constituting the software program or a computer-readable storage medium in which the code is recorded.
- a common partial structure extracting unit that extracts a common partial structure that is a structure of the same type of part common to the structure that represents the first sentence and the structure that represents the second sentence; The feature quantity based on the dependency between one or more common partial structures in the first and second sentences, and the common partial structure in the first and second sentences are different from the common partial structure.
- Feature quantity extraction means for extracting at least one of the feature quantities based on the dependency relationship with the partial structure; Determining means for determining an implication relationship between the first sentence and the second sentence by using the extracted feature value;
- a text processing system comprising:
- the common partial structure extracting means is There are one or more common partial structure candidates that are subtrees of the same type included in the first tree structure that represents the first sentence and the second tree structure that represents the second sentence. If there, By selecting from the one or more common partial structure candidates, based on the number of nodes included in the common partial structure candidate, the common partial structure candidate that does not overlap with the other common partial structure candidates, The text processing system according to attachment 1, wherein the common partial structure is extracted.
- the feature amount extraction means includes: When there are a plurality of the common partial structures, the dependency between the common partial structures in the structure representing the first sentence and the dependence between the common partial structures in the structure representing the second sentence The text processing system according to appendix 2, wherein the feature quantity including at least information representing the relationship is extracted.
- the feature amount extraction means includes: In the case where there are a plurality of the common partial structures, in the first tree structure and the second tree structure, among the plurality of common partial structures, between the specific common partial structure and the other common partial structures.
- the feature amount extraction means includes: From the partial structure that is a descendant of the specific common partial structure and does not exist between the specific common partial structure and the other common partial structure in the first tree structure and the second tree structure, Extracting the feature quantity based on at least one of the number of nodes and the attributes of the nodes included in the partial structure; The text processing system according to any one of appendix 2 to appendix 4.
- (Appendix 6) For one or more partial structures, it further comprises isomorphic information holding means for holding isomorphic information related to other partial structures that are isomorphic to the partial structure,
- the common partial structure extracting means is Based on the isomorphic information held by the isomorphic information holding means, at least one of the partial structure included in the first tree structure and the partial structure included in the second tree structure is the same type as the partial structure. Replace with other substructures, Extracting the common partial structure based on the first tree structure and the second tree structure after the replacement;
- the text processing system according to any one of appendix 2 to appendix 5.
- the common partial structure extracting means is Based on the isomorphic information held by the isomorphic information holding means, at least one of the partial structure included in the first tree structure and the partial structure included in the second tree structure is the same type as the partial structure.
- the common partial structure candidates are extracted based on the first tree structure and the second tree structure after the replacement, Among the extracted candidates for the common partial structure, the common partial structure candidate that minimizes the number of nodes that are not candidates for the common partial structure included in the second tree structure after the replacement is used as the common partial structure candidate. Extract as a substructure, The text processing system according to attachment 6.
- Information processing device Based on the structure representing the first sentence and the structure representing the second sentence, a common partial structure that is the same type of structure common to them is extracted, The feature quantity based on the dependency between one or more common partial structures in the first and second sentences, and the common partial structure in the first and second sentences are different from the common partial structure. Extract at least one of the features based on the dependency between the substructures, By using the extracted feature quantity, an implication relationship between the first sentence and the second sentence is determined. Text processing method.
- a common partial structure extracting unit that extracts a common partial structure that is a structure of the same type of part common to the structure that represents the first sentence and the structure that represents the second sentence; The feature quantity based on the dependency between one or more common partial structures in the first and second sentences, and the common partial structure in the first and second sentences are different from the common partial structure.
- Feature quantity extraction means for extracting at least one of the feature quantities based on the dependency relationship with the partial structure; Determining means for determining an implication relationship between the first sentence and the second sentence by using the extracted feature value;
- a text processing device comprising a single device.
- the common partial structure extraction unit is When there are one or more candidates for the common partial structure included in the first tree structure and the second tree structure, From the one or more common partial structure candidates, the common partial structure candidates that do not overlap with the other common partial structure candidates in descending order of the number of nodes included in the common partial structure candidates based on a predetermined criterion To extract the selected candidate for the common partial structure as the common partial structure, The text processing system according to attachment 2.
- the feature quantity extraction unit When there are multiple common partial structures, The information including at least information indicating whether or not the dependency relationship between the common partial structures in the structure representing the first sentence matches the dependency relationship between the common partial structures in the structure representing the second sentence.
- the common partial structure extracting means is There are one or more common partial structure candidates that are subtrees of the same type included in the first tree structure that represents the first sentence and the second tree structure that represents the second sentence. If there, By selecting from the one or more common partial structure candidates, the common partial structure candidate that has the maximum number of nodes included in the common partial structure candidate and does not overlap with the other common partial structure candidates.
- the text processing system according to attachment 2 wherein the common partial structure is extracted.
- the present invention can be applied to, for example, an information search apparatus that searches for other sentences implying a specific sentence from a corpus, a computer program for realizing the information search apparatus using an information processing apparatus, and the like.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Machine Translation (AREA)
Abstract
Description
本発明の一態様に係るテキスト処理方法は、以下のとおりである。即ち、本発明の一態様に係るテキスト処理方法は、情報処理装置が、第1の文を表す構造と第2の文を表す構造とに基づいて、それらに共通する同型な部分の構造である共通部分構造を抽出し、上記第1及び第2の文における1以上の上記共通部分構造間の依存関係に基づいた特徴量、及び、上記第1及び第2の文における上記共通部分構造と、上記共通部分構造とは異なる部分構造との間の依存関係に基づいた特徴量の、少なくともいずれかを抽出し、抽出された上記特徴量を用いることにより、上記第1の文と上記第2の文との間の含意関係を判定する。
次に、本願発明を実施する第1の形態について図面を参照して詳細に説明する。
次に、本発明の第2実施形態について詳細に説明する。以下の説明においては、本実施形態に係る特徴的な部分を中心に説明する。上述した第1の実施形態と同様な構成についての重複する説明は省略する。
次に、本発明の第3の実施形態について説明する。図9は、本発明の第3の実施形態にかかるテキスト処理システム900の機能的な構成を例示するブロック図である。
以下、上記説明した各実施形態を実現可能なハードウェア及びソフトウェア・プログラムの構成について説明する。以下においては、特に、上記各実施形態におけるテキスト含意認識装置100、テキスト処理システム900の構成について説明する。以下、テキスト含意認識装置100と、テキスト処理システム900の各構成要素とをまとめて「テキスト処理システム等」と称する場合がある。なお、以下においては、テキスト処理システム900の各構成要素(共通部分構造抽出部901、特徴量抽出部902、及び、判定部903)は、単一の装置により実現されてもよく、複数の異なる装置により実現されてもよい。
第1の文を表す構造と第2の文を表す構造とに基づいて、それらに共通する同型な部分の構造である共通部分構造を抽出する共通部分構造抽出手段と、
上記第1及び第2の文における1以上の上記共通部分構造間の依存関係に基づいた特徴量、及び、上記第1及び第2の文における上記共通部分構造と、上記共通部分構造とは異なる部分構造との間の依存関係に基づいた特徴量の、少なくともいずれかを抽出する特徴量抽出手段と、
抽出された上記特徴量を用いることにより、上記第1の文と上記第2の文との間の含意関係を判定する判定手段と、
を備えるテキスト処理システム。
上記共通部分構造抽出手段は、
上記第1の文を表す構造である第1の木構造、及び、上記第2の文を表す構造である第2の木構造に含まれる同型な部分木である共通部分構造の候補が1以上存在する場合、
1以上の上記共通部分構造の候補から、当該共通部分構造の候補に含まれるノードの数に基づいて、他の上記共通部分構造の候補と重複しない上記共通部分構造の候補を選択することにより、上記共通部分構造を抽出する
付記1に記載のテキスト処理システム。
上記特徴量抽出手段は、
上記共通部分構造が複数存在する場合、上記第1の文を表す構造における上記共通部分構造間の依存関係と、上記第2の文を表す構造における上記共通部分構造間の依存関係との間の関係を表す情報を少なくとも含む上記特徴量を抽出する
付記2に記載のテキスト処理システム。
上記特徴量抽出手段は、
上記共通部分構造が複数存在する場合、上記第1の木構造及び上記第2の木構造において、複数の上記共通部分構造のうち、特定の上記共通部分構造と他の上記共通部分構造との間に存在する部分構造に含まれるノードの数及び当該ノードの属性の少なくとも一方に基づいて上記特徴量を抽出する
付記2または付記3に記載のテキスト処理システム。
上記特徴量抽出手段は、
上記第1の木構造及び上記第2の木構造において、特定の上記共通部分構造の子孫であり、特定の上記共通部分構造と、他の上記共通部分構造との間に存在しない部分構造から、当該部分構造に含まれるノードの数及びノードの属性の少なくとも一方に基づいて上記特徴量を抽出する、
付記2乃至付記4の何れかに記載のテキスト処理システム。
1以上の部分構造について、当該部分構造と同型である他の部分構造に関する同型情報を保持する同型情報保持手段を更に備え、
上記共通部分構造抽出手段は、
上記同型情報保持手段が保持する上記同型情報に基づいて、上記第1の木構造に含まれる部分構造及び上記第2の木構造に含まれる部分構造の少なくとも一方を、当該部分構造と同型である他の部分構造に置換し、
当該置換後の上記第1の木構造と上記第2の木構造とに基づいて、上記共通部分構造を抽出する、
付記2乃至付記5のいずれかに記載のテキスト処理システム。
上記共通部分構造抽出手段は、
上記同型情報保持手段が保持する上記同型情報に基づいて、上記第1の木構造に含まれる部分構造及び上記第2の木構造に含まれる部分構造の少なくとも一方を、当該部分構造と同型である他の部分構造に置換する全ての組合せについて、当該置換後の上記第1の木構造と上記第2の木構造とに基づいて上記共通部分構造の候補を抽出し、
上記抽出した上記共通部分構造の候補のうち、上記置換後の上記第2の木構造に含まれる上記共通部分構造の候補ではないノードの数が最小となる上記共通部分構造の候補を、上記共通部分構造として抽出する、
付記6に記載のテキスト処理システム。
情報処理装置が、
第1の文を表す構造と第2の文を表す構造とに基づいて、それらに共通する同型な部分の構造である共通部分構造を抽出し、
上記第1及び第2の文における1以上の上記共通部分構造間の依存関係に基づいた特徴量、及び、上記第1及び第2の文における上記共通部分構造と、上記共通部分構造とは異なる部分構造との間の依存関係に基づいた特徴量の、少なくともいずれかを抽出し、
抽出された上記特徴量を用いることにより、上記第1の文と上記第2の文との間の含意関係を判定する、
テキスト処理方法。
第1の文を表す構造と第2の文を表す構造とに基づいて、それらに共通する同型な部分の構造である共通部分構造を抽出する処理と、
上記第1及び第2の文における1以上の上記共通部分構造間の依存関係に基づいた特徴量、及び、上記第1及び第2の文における上記共通部分構造と、上記共通部分構造とは異なる部分構造との間の依存関係に基づいた特徴量の、少なくともいずれかを抽出する処理と、
抽出された上記特徴量を用いることにより、上記第1の文と上記第2の文との間の含意関係を判定する処理と、をコンピュータに実行させる、
コンピュータ・プログラム。
第1の文を表す構造と第2の文を表す構造とに基づいて、それらに共通する同型な部分の構造である共通部分構造を抽出する共通部分構造抽出手段と、
上記第1及び第2の文における1以上の上記共通部分構造間の依存関係に基づいた特徴量、及び、上記第1及び第2の文における上記共通部分構造と、上記共通部分構造とは異なる部分構造との間の依存関係に基づいた特徴量の、少なくともいずれかを抽出する特徴量抽出手段と、
抽出された上記特徴量を用いることにより、上記第1の文と上記第2の文との間の含意関係を判定する判定手段と、
を単一の装置として備えるテキスト処理装置。
上記共通部分構造抽出部は、
上記第1の木構造、及び、上記第2の木構造に含まれる上記共通部分構造の候補が1以上存在する場合、
上記1以上の共通部分構造の候補から、当該共通部分構造の候補に含まれるノードの数が所定の基準に基づいて多い順に、他の上記共通部分構造の候補と重複しない上記共通部分構造の候補を選択することにより、当該選択した上記共通部分構造の候補を上記共通部分構造として抽出する、
付記2に記載のテキスト処理システム。
上記特徴量抽出部は、
上記共通部分構造が複数存在する場合に、
上記第1の文を表す構造における上記共通部分構造間の依存関係と、上記第2の文を表す構造における上記共通部分構造間の依存関係とが一致するか否かを表す情報を少なくとも含む上記特徴量を抽出する
付記3に記載のテキスト処理システム。
上記共通部分構造抽出手段は、
上記第1の文を表す構造である第1の木構造、及び、上記第2の文を表す構造である第2の木構造に含まれる同型な部分木である共通部分構造の候補が1以上存在する場合、
1以上の上記共通部分構造の候補から、当該共通部分構造の候補に含まれるノードの数が最大となる、他の上記共通部分構造の候補と重複しない上記共通部分構造の候補を選択することにより、上記共通部分構造を抽出する
付記2に記載のテキスト処理システム。
101 構文解析部
102 共通部分構造抽出部
103 特徴量抽出部
104 判定部
105 入力装置
106 出力装置
107 同型情報保持部
900 テキスト処理システム
901 共通部分構造抽出部
902 特徴量抽出部
903 判定部
1201 演算装置
1202 記憶装置
1203 不揮発性記憶装置
1204 ドライブ装置
1205 記憶媒体
1206 ネットワークインタフェース
1207 入出力インタフェース
Claims (10)
- 第1の文を表す構造と第2の文を表す構造とに基づいて、それらに共通する同型な部分の構造である共通部分構造を抽出する共通部分構造抽出手段と、
前記第1及び第2の文における1以上の前記共通部分構造間の依存関係に基づいた特徴量、及び、前記第1及び第2の文における前記共通部分構造と、前記共通部分構造とは異なる部分構造との間の依存関係に基づいた特徴量の、少なくともいずれかを抽出する特徴量抽出手段と、
抽出された前記特徴量を用いることにより、前記第1の文と前記第2の文との間の含意関係を判定する判定手段と、
を備えるテキスト処理システム。 - 前記共通部分構造抽出手段は、
前記第1の文を表す構造である第1の木構造、及び、前記第2の文を表す構造である第2の木構造に含まれる同型な部分木である共通部分構造の候補が1以上存在する場合、
1以上の前記共通部分構造の候補から、当該共通部分構造の候補に含まれるノードの数に基づいて、他の前記共通部分構造の候補と重複しない前記共通部分構造の候補を選択することにより、前記共通部分構造を抽出する
請求項1に記載のテキスト処理システム。 - 前記共通部分構造抽出手段は、
前記第1の文を表す構造である第1の木構造、及び、前記第2の文を表す構造である第2の木構造に含まれる同型な部分木である共通部分構造の候補が1以上存在する場合、
1以上の前記共通部分構造の候補から、当該共通部分構造の候補に含まれるノードの数が最大となる、他の前記共通部分構造の候補と重複しない前記共通部分構造の候補を選択することにより、前記共通部分構造を抽出する
請求項2に記載のテキスト処理システム。 - 前記特徴量抽出手段は、
前記共通部分構造が複数存在する場合、前記第1の文を表す構造における前記共通部分構造間の依存関係と、前記第2の文を表す構造における前記共通部分構造間の依存関係との間の関係を表す情報を少なくとも含む前記特徴量を抽出する
請求項2または請求項3に記載のテキスト処理システム。 - 前記特徴量抽出手段は、
前記共通部分構造が複数存在する場合、前記第1の木構造及び前記第2の木構造において、複数の前記共通部分構造のうち、特定の前記共通部分構造と他の前記共通部分構造との間に存在する部分構造に含まれるノードの数及び当該ノードの属性の少なくとも一方に基づいて前記特徴量を抽出する
請求項2乃至請求項4のいずれかに記載のテキスト処理システム。 - 前記特徴量抽出手段は、
前記第1の木構造及び前記第2の木構造において、特定の前記共通部分構造の子孫であり、特定の前記共通部分構造と、他の前記共通部分構造との間に存在しない部分構造から、当該部分構造に含まれるノードの数及びノードの属性の少なくとも一方に基づいて前記特徴量を抽出する、
請求項2乃至請求項4の何れかに記載のテキスト処理システム。 - 1以上の部分構造について、当該部分構造と同型である他の部分構造に関する同型情報を保持する同型情報保持手段を更に備え、
前記共通部分構造抽出手段は、
前記同型情報保持手段が保持する前記同型情報に基づいて、前記第1の木構造に含まれる部分構造及び前記第2の木構造に含まれる部分構造の少なくとも一方を、当該部分構造と同型である他の部分構造に置換し、
当該置換後の前記第1の木構造と前記第2の木構造とに基づいて、前記共通部分構造を抽出する、
請求項2乃至請求項6のいずれかに記載のテキスト処理システム。 - 前記共通部分構造抽出手段は、
前記同型情報保持手段が保持する前記同型情報に基づいて、前記第1の木構造に含まれる部分構造及び前記第2の木構造に含まれる部分構造の少なくとも一方を、当該部分構造と同型である他の部分構造に置換する全ての組合せについて、当該置換後の前記第1の木構造と前記第2の木構造とに基づいて前記共通部分構造の候補を抽出し、
前記抽出した前記共通部分構造の候補のうち、前記置換後の前記第2の木構造に含まれる前記共通部分構造の候補ではないノードの数が最小となる前記共通部分構造の候補を、前記共通部分構造として抽出する、
請求項7に記載のテキスト処理システム。 - 情報処理装置が、
第1の文を表す構造と第2の文を表す構造とに基づいて、それらに共通する同型な部分の構造である共通部分構造を抽出し、
前記第1及び第2の文における1以上の前記共通部分構造間の依存関係に基づいた特徴量、及び、前記第1及び第2の文における前記共通部分構造と、前記共通部分構造とは異なる部分構造との間の依存関係に基づいた特徴量の、少なくともいずれかを抽出し、
抽出された前記特徴量を用いることにより、前記第1の文と前記第2の文との間の含意関係を判定する、
テキスト処理方法。 - 第1の文を表す構造と第2の文を表す構造とに基づいて、それらに共通する同型な部分の構造である共通部分構造を抽出する処理と、
前記第1及び第2の文における1以上の前記共通部分構造間の依存関係に基づいた特徴量、及び、前記第1及び第2の文における前記共通部分構造と、前記共通部分構造とは異なる部分構造との間の依存関係に基づいた特徴量の、少なくともいずれかを抽出する処理と、
抽出された前記特徴量を用いることにより、前記第1の文と前記第2の文との間の含意関係を判定する処理と、をコンピュータに実行させるコンピュータ・プログラムが記録された
記憶媒体。
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US15/506,293 US10339223B2 (en) | 2014-09-05 | 2015-08-20 | Text processing system, text processing method and storage medium storing computer program |
JP2016546291A JP6614152B2 (ja) | 2014-09-05 | 2015-08-20 | テキスト処理システム、テキスト処理方法、及び、コンピュータ・プログラム |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2014-180828 | 2014-09-05 | ||
JP2014180828 | 2014-09-05 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2016035273A1 true WO2016035273A1 (ja) | 2016-03-10 |
Family
ID=55439362
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2015/004155 WO2016035273A1 (ja) | 2014-09-05 | 2015-08-20 | テキスト処理システム、テキスト処理方法、及び、コンピュータ・プログラムが記録された記憶媒体 |
Country Status (3)
Country | Link |
---|---|
US (1) | US10339223B2 (ja) |
JP (1) | JP6614152B2 (ja) |
WO (1) | WO2016035273A1 (ja) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2019053878A1 (ja) * | 2017-09-15 | 2019-03-21 | 日本電気株式会社 | 知識生成装置、知識生成方法、及びコンピュータ読み取り可能な記録媒体 |
Families Citing this family (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2017163346A1 (ja) * | 2016-03-23 | 2017-09-28 | 株式会社野村総合研究所 | 文章解析システム及びプログラム |
CN107688614B (zh) * | 2017-08-04 | 2018-08-10 | 平安科技(深圳)有限公司 | 意图获取方法、电子装置及计算机可读存储介质 |
US11195006B2 (en) * | 2018-12-06 | 2021-12-07 | International Business Machines Corporation | Multi-modal document feature extraction |
JP7472587B2 (ja) * | 2020-03-26 | 2024-04-23 | 富士通株式会社 | エンコーディングプログラム、情報処理装置およびエンコーディング方法 |
Citations (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2013058118A1 (ja) * | 2011-10-20 | 2013-04-25 | 日本電気株式会社 | テキスト含意判定装置、テキスト含意判定方法、及びコンピュータ読み取り可能な記録媒体 |
Family Cites Families (12)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN1934570B (zh) * | 2004-03-18 | 2012-05-16 | 日本电气株式会社 | 文本挖掘装置和其方法 |
JP4992715B2 (ja) * | 2005-08-04 | 2012-08-08 | 日本電気株式会社 | データ処理装置、データ処理方法、データ処理プログラム |
US8244730B2 (en) * | 2006-05-30 | 2012-08-14 | Honda Motor Co., Ltd. | Learning syntactic patterns for automatic discovery of causal relations from text |
US8738359B2 (en) * | 2006-10-18 | 2014-05-27 | Honda Motor Co., Ltd. | Scalable knowledge extraction |
US20080177531A1 (en) * | 2007-01-19 | 2008-07-24 | Oki Electric Industry Co., Ltd. | Language processing apparatus, language processing method, and computer program |
JP5076575B2 (ja) | 2007-03-19 | 2012-11-21 | 日本電気株式会社 | 同義表現抽出システム、同義表現抽出方法、及び同義表現抽出プログラム |
JP2009099030A (ja) | 2007-10-18 | 2009-05-07 | Nec Corp | 処理内容判定装置および処理内容判定方法 |
US8335754B2 (en) * | 2009-03-06 | 2012-12-18 | Tagged, Inc. | Representing a document using a semantic structure |
US8874432B2 (en) * | 2010-04-28 | 2014-10-28 | Nec Laboratories America, Inc. | Systems and methods for semi-supervised relationship extraction |
JP5496853B2 (ja) | 2010-10-29 | 2014-05-21 | インターナショナル・ビジネス・マシーンズ・コーポレーション | 構造化文書を分類するためのルールを生成するための方法、並びにそのコンピュータ・プログラム及びコンピュータ |
JP5637888B2 (ja) | 2011-02-09 | 2014-12-10 | 三菱電機株式会社 | 同一意図テキスト生成装置、意図推定装置および同一意図テキスト生成方法 |
KR20150129134A (ko) * | 2014-05-08 | 2015-11-19 | 한국전자통신연구원 | 질의 응답 시스템 및 그 방법 |
-
2015
- 2015-08-20 JP JP2016546291A patent/JP6614152B2/ja active Active
- 2015-08-20 US US15/506,293 patent/US10339223B2/en active Active
- 2015-08-20 WO PCT/JP2015/004155 patent/WO2016035273A1/ja active Application Filing
Patent Citations (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2013058118A1 (ja) * | 2011-10-20 | 2013-04-25 | 日本電気株式会社 | テキスト含意判定装置、テキスト含意判定方法、及びコンピュータ読み取り可能な記録媒体 |
Non-Patent Citations (2)
Title |
---|
M. W. CHANG ET AL.: "Discriminative Learning over Constrained Latent Representations", HUMAN LANGUAGE TECHNOLOGIES: THE 2010 ANNUAL CONFERENCE OF THE NORTH AMERICAN CHAPTER OF THE ACL, 30 June 2010 (2010-06-30), pages 429 - 437 * |
MASAAKI TSUCHIDA ET AL.: "Big Data Katsuyo no Tameno Text Bunseki Gijutsu", NEC TECHNICAL JOURNAL, vol. 65, no. 2, 1 September 2012 (2012-09-01), pages 77 - 80 * |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2019053878A1 (ja) * | 2017-09-15 | 2019-03-21 | 日本電気株式会社 | 知識生成装置、知識生成方法、及びコンピュータ読み取り可能な記録媒体 |
JPWO2019053878A1 (ja) * | 2017-09-15 | 2020-10-15 | 日本電気株式会社 | 知識生成装置、知識生成方法、及びプログラム |
Also Published As
Publication number | Publication date |
---|---|
JP6614152B2 (ja) | 2019-12-04 |
US10339223B2 (en) | 2019-07-02 |
US20170255611A1 (en) | 2017-09-07 |
JPWO2016035273A1 (ja) | 2017-06-15 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10706236B1 (en) | Applied artificial intelligence technology for using natural language processing and concept expression templates to train a natural language generation system | |
US9087236B2 (en) | Automated recognition of process modeling semantics in flow diagrams | |
JP6614152B2 (ja) | テキスト処理システム、テキスト処理方法、及び、コンピュータ・プログラム | |
US10839155B2 (en) | Text analysis of morphemes by syntax dependency relationship with determination rules | |
RU2610241C2 (ru) | Способ и система синтеза текста на основе извлеченной информации в виде rdf-графа с использованием шаблонов | |
US20170286103A1 (en) | Identifying and correlating semantic bias for code evaluation | |
Li et al. | Global greedy dependency parsing | |
Le et al. | Interactive program synthesis | |
KR20150084706A (ko) | 온톨로지의 지식 학습 장치 및 그의 방법 | |
CN114239546A (zh) | 一种基于语法树剪枝的翻译机测试方法 | |
CN108519963B (zh) | 一种将流程模型自动转换为多语言文本的方法 | |
Makhambetov et al. | Data-driven morphological analysis and disambiguation for kazakh | |
JP6952967B2 (ja) | 自動翻訳装置 | |
Wax | Automated grammar engineering for verbal morphology | |
Cui et al. | Simple question answering over knowledge graph enhanced by question pattern classification | |
JP2005181928A (ja) | 機械学習システム及び機械学習方法、並びにコンピュータ・プログラム | |
CN115066674A (zh) | 使用源代码元素的数值数组表示来评估源代码的方法 | |
Kramer et al. | Improvement of a naive Bayes sentiment classifier using MRS-based features | |
KR102621758B1 (ko) | 표로부터 지식 데이터를 추출하기 위한 시스템 및 방법 | |
US20210097104A1 (en) | Painting label generation method and electronic device | |
CN114528459A (zh) | 一种基于语义的网页信息抽取方法及系统 | |
KR102203895B1 (ko) | 임베딩 기반의 인과 관계 탐지 시스템과 방법 및 이를 실행하기 위한 프로그램이 기록된 기록매체 | |
Chitra et al. | Genetic algorithm based feature selection for paraphrase recognition | |
Chen et al. | Improving Shift‐Reduce Phrase‐Structure Parsing with Constituent Boundary Information | |
JP4933118B2 (ja) | 文章区間抽出装置及びプログラム |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 15838061 Country of ref document: EP Kind code of ref document: A1 |
|
ENP | Entry into the national phase |
Ref document number: 2016546291 Country of ref document: JP Kind code of ref document: A |
|
WWE | Wipo information: entry into national phase |
Ref document number: 15506293 Country of ref document: US |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 15838061 Country of ref document: EP Kind code of ref document: A1 |