JP4815934B2 - Text mining device, text mining method, text mining program - Google Patents

Description

  The present invention relates to text mining, and more particularly, to a text mining device, a text mining method, and a text mining program that can perform mining by equating synonymous expressions having different dependency structure trees.

The text mining device is configured to extract a pattern of words that frequently appear in the text in order to extract useful knowledge from a large amount of text. An example of a conventional text mining device is described in Patent Document 1.
The text mining device described in Patent Document 1 includes a language analysis device that analyzes a syntax structure of each sentence in a text and constructs a syntax tree, and a pattern extraction device that finds a frequent pattern from the syntax tree. And syntactic patterns of words that frequently appear in the text.

On the other hand, Non-Patent Document 1 describes an example of a system that transforms a dependency structure tree and converts it into another dependency structure tree corresponding to a synonymous expression. The dependency structure processing system described in Non-Patent Document 1 refers to a predetermined conversion rule (matching pattern / rewrite pattern) and converts another dependency structure tree that matches the matching pattern according to the rewrite pattern. Convert to
In the conversion rule, the relationship of corresponding nodes between the dependency structure tree before conversion and the dependency structure tree after conversion is described using variables. In Non-Patent Document 1, for example, a dependency structure tree corresponding to the expression “N1 is caused to V2 by N2” is converted into another dependency structure tree corresponding to a synonymous expression “N2 causes V1 to N1”. An example conversion rule for is shown. In this example, N1 and N2 are variables corresponding to nouns, and V is a variable corresponding to a verb.

JP 2001-84250 A Yuya Iwakura and four others, General Purpose Dependent Structure Processing Module KURALANG, Proceedings of the 9th Annual Conference of the Language Processing Society, March 18, 2003, pp. 687-690

  However, the conventional text mining device has a problem in that mining cannot be performed by equating a dependency structure tree of synonymous expressions.

Conventional text mining devices have different synonymous expressions with different dependency structure trees, that is, different dependency structure trees even though they represent the same semantic content (words used and dependency relationships between the words are different) ) Mining cannot be done with the same expression. This is because the conventional text mining apparatus does not consider any synonymous expressions with different dependency structure trees.
As a result, when there are multiple expressions corresponding to the synonymous meaning contents, it must be determined whether or not to extract each expression, and there is a risk that characteristic meaning contents may not be extracted. .

For example, the expression EX1 “reducing the character to be displayed” and the expression EX2 “displaying with a small character” both express the same meaning content “reducing the size of the character used for display”. Synonymous expression. However, the dependency structure tree DT-EX1 corresponding to the expression EX1 shown in FIG. 1A is different from the dependency structure tree DT-EX2 corresponding to the expression EX2 shown in FIG.
As an example, here is an example of a dependency structure tree in which a clause is a node, an independent word belonging to a clause is converted to a terminal form, a node label is used, and the dependency relationship between clauses is a branch. ing. The same applies to the following examples.

  Since the conventional text mining device cannot identify these expressions, for example, there are 23 texts in which the meaning content “reducing the size of characters used for display” is described using the expression EX1. In the case where there are 34 texts described using the expression EX2, it may be recognized that there are 57 texts including the meaning content “reducing the size of characters used for display” in total. Can not. As a result, for example, when mining to extract expressions that appear 50 times or more, the meaning of “reducing the size of characters used for display” is described in 57 texts. Cannot be extracted as characteristic meaning content.

  The conventional text mining device converts the dependency structure tree in advance so as to unify the synonymous expressions into one expression, and performs the conversion rule between the dependency structure trees even if the dependency structure tree after conversion is mined. Takes time to create The reason for this is that in the conversion rules between the dependency structure trees, it is clear which node should be connected to each node in the dependency structure tree after conversion. This is because it needs to be described.

  For example, when describing a rule for converting the dependency structure tree DT-EX1 (FIG. 1A) corresponding to the expression EX1 to the dependency structure tree DT-EX2 (FIG. 1B) corresponding to the expression EX2, DT -The node joined to the node "display" in EX1 and the node joined to the node "Yes" are both converted to be joined to the node "display" in DT-EX2, and DT -It is necessary to clearly describe, for example, that a node joined to the node "small" in EX1 is converted to be joined to the node "small" in DT-EX2. Otherwise, in the dependency structure tree DT-S1 (FIG. 2) corresponding to the sentence S1 “A method for reducing the size of characters for displaying mail as much as possible” is a portion that matches the dependency structure tree DT-EX1 (FIG. 2). 3 is converted to the dependency structure tree DT-EX2 (FIG. 1B), the node “mail”, the node “as much as possible” and the node “examine” are joined to which node in the converted dependency structure tree. It is not possible to decide what to do, and the dependency structure tree cannot be converted.

  Even if the conventional text mining device converts the dependency structure tree in advance so that the synonymous expressions are unified into one expression and performs mining on the converted dependency structure tree, the expression to be regarded as synonymous can be obtained. If the nodes are not completely the same in meaning and cannot correspond to the nodes between the dependency structure trees, the dependency structure trees corresponding to those expressions cannot be mined in advance. The reason for this is that if it is not clear which node should be joined to each node in the dependency structure tree before conversion, it is necessary to convert between the dependency structure trees. This is because it cannot be done.

For example, the dependency structure tree DT-EX3 corresponding to the expression EX3 “increasing the number of lines to be displayed” is as illustrated in FIG. At this time, each node of the dependency structure tree DT-EX1 corresponding to the expression EX1 “reducing displayed characters” shown in FIG. 1A and each node of the dependency structure tree DT-EX3 of FIG. Even if it is desired that the expressions EX1 and EX3 are regarded as synonymous, it is not possible to perform conversion between corresponding dependency structure trees.
Actually, the dependency structure tree DT-S2 (FIG. 4B) corresponding to the sentence S2 “double the number of lines for displaying the mail” conforms to the dependency structure tree DT-EX3 (FIG. 4A). Even if an attempt is made to convert the portion to be converted into the dependency structure tree DT-EX1 (FIG. 1 (a)), in the expression EX1 “reducing the displayed character”, there is a destination related to the phrase “double” in the sentence S2. The node “double” in the dependency structure tree DT-S2 cannot be joined to any node of the converted dependency structure tree.

  In a conventional text mining device, if a dependency structure tree is converted in advance so that synonymous expressions are unified into one expression, and the dependency structure tree after conversion is mined, an expression that is not characteristic in nature is obtained. There is a possibility that an expression that is originally extracted or a characteristic expression is not extracted by mistake. The reason is that there is a side effect that a node that did not originally exist is newly generated or a node that originally existed is deleted by conversion of the dependency structure tree.

For example, the dependency structure tree DT-EX2 (FIG. 1B) corresponding to the expression EX2 “display with small characters” is changed to the dependency structure tree DT-EX1 (FIG. 1B) corresponding to the expression EX1 “reducing display characters”. In the case of conversion to 1 (a)), the node “small” and “node” that did not originally exist are newly generated.
When such dependency structure tree conversion is performed, for example, 21 texts using the expression EX1 “reducing displayed characters” and texts using the expression EX2 “display with small characters” are used. In addition to 18 cases, it is assumed that there are 16 texts that are different from the expression EX1 and the expression EX2 such as “reduce the size of the image to be displayed”, but use the expression including “decrease”. After the conversion, a total of 55 subtrees composed of the node “small” and the node “do” appear in all the dependency structure trees.
As a result, for example, when mining is performed so as to extract a subtree that appears 50 or more times, only 37 subtrees that are composed of the nodes “small” and “do” appear originally. Regardless, it is extracted as a characteristic subtree.

On the other hand, for example, the dependency structure tree DT-EX1 (FIG. 1A) corresponding to the expression EX1 “reducing displayed characters” is changed to the dependency structure tree DT− corresponding to the expression EX2 “displaying with small characters”. In the case of conversion to EX2 (FIG. 1B), the node “small” and the node “yes” that were originally present are deleted.
When such dependency structure tree conversion is performed, for example, 34 texts using the expression EX1 “reducing displayed characters” and texts using the expression EX2 “display with small characters” are used. In addition to 13 cases, there are 19 texts that are different from the expression EX1 and the expression EX2 as in “reducing the size of an image to be displayed”, but use an expression including “decrease”. Then, after the conversion, only a total of 19 subtrees constituted by the node “small” and the node “do” appear in the entire dependency structure tree.
As a result, for example, when mining is performed so as to extract subtrees that appear 50 times or more, 53 subtrees originally composed of the nodes “small” and “do” have appeared. Nevertheless, it is no longer extracted as a characteristic subtree.

  Accordingly, an object of the present invention is to provide a text mining device, a text mining method, and a text mining program that can perform mining while equating synonymous expressions having different dependency structure trees.

In the text mining device according to the present invention, the synonym expression identifying means includes a dependency structure tree (in the dependency structure tree of the sentence to be text mined) (target sentence dependency structure tree) of the expressions registered in the synonym expression dictionary. It is identified whether or not a subtree (matching subtree) that matches the synonymous expression dependency structure tree is included.
The node replacement means replaces the matching subtree with a special node (synonymous expression node) indicating the group to which the synonym expression belongs, and the feature subtree extraction means extracts the feature subtree from the target sentence dependency structure tree after replacement ( Claim 1).

According to the text mining device, the node substituting unit replaces the matching subtree included in the target sentence dependent structure tree with the synonymous expression node, and the feature subtree extracting unit extracts the feature part from the target sentence dependent structural tree after the replacement. Extract trees.
Therefore, a characteristic subtree can be extracted by equating synonymous expressions with different dependency structure trees.

In the text mining device according to the present invention, the synonymous expression identifying means identifies whether or not the matching subtree is included in the target sentence dependent structure tree.
The node addition means adds a synonymous expression node to the target sentence dependency structure tree. The node addition means adds a dependency branch from the synonymous expression node to a node having a dependency branch from a node in the matching subtree at a node outside the matching subtree, and matches at a node outside the matching subtree A dependency branch to a synonymous expression node is added from a node having a dependency branch to a node in the subtree. The feature subtree extracting means extracts the feature subtree from the target sentence dependency structure tree (dependency structure bundle) after the addition of the synonymous expression node and the dependency branch.

According to the text mining device, the node adding means adds a synonymous expression node corresponding to the matching subtree included in the target sentence dependency structure tree to the target sentence dependency structure tree, and determines the relationship between the matching subtree and external nodes. A dependency structure bundle is generated by adding a dependency branch equivalent to the dependency branch shown to the synonymous expression node. The feature subtree extracting means extracts a feature subtree from the dependency structure bundle after the addition of nodes and dependency branches.
Therefore, a characteristic subtree can be extracted by equating synonymous expressions with different dependency structure trees. Furthermore, since the existing nodes remain as they are in the dependency structure bundle that is the target of the feature subtree extraction, adverse effects caused by unifying synonymous expressions can be suppressed to a low level.

In the text mining device, the exclusive relation branch adding means generates a dependency structure bundle in which the synonymous expression nodes and the nodes included in the matching subtree are connected by the exclusive relation branches, and the feature subtree extracting means is the exclusive relation branch. A feature subtree may be extracted from a subtree that does not include a connected node.
In this way, a subtree that does not make sense as an extraction result including both nodes and synonymous expression nodes included in the matching subtree is not extracted as a feature subtree.

The text mining device further includes an extraction result output unit, and the extraction result output unit is configured to output a representation of a synonym expression node node label included in the feature subtree and representing an expression belonging to the synonym expression group indicated by the label. The shape of the feature part tree may be output in a form that can be visually recognized by a human.
In this way, the label of the synonym expression node included in the feature structure displayed on the output device is not a symbol or the like that does not make sense to the user, but an expression that represents the synonym expression group. You can understand the meaning.

  In the text mining device, the output expression may be an expression listed first in the synonym expression group indicated by the label of the synonym expression node (claim 5).

  In the text mining device, the output expression may be an expression having the shortest length in the synonym expression group indicated by the synonym expression node label.

  In the text mining device, the output expression may be an expression that appears most frequently in the sentence set that is the target of the text mining in the synonym expression group indicated by the synonym expression node label.

  In the text mining device, the output expression may be an expression designated in advance as an expression for output in the synonym expression group indicated by the label of the synonym expression node (claim 8).

  In the text mining device, the output expression may be an expression defined in advance corresponding to the synonym expression group indicated by the synonym expression node label, separately from the expressions included in the synonym expression group. ).

In the text mining method according to the present invention, the computer of the text mining device generates a target sentence-dependent structure tree from the text database to be mined, reads the expressions stored in the synonym expression dictionary from the storage device, and synonymous expressions. Generates a dependency structure tree, compares the target sentence dependency structure tree with the synonymous expression dependency structure tree to identify whether the matching subtree is included in the target sentence dependency structure tree, and is distinguished from a normal node label an identifier generating the synonymous expression nodes for a label that, all nodes in the matching subtree replaced with synonymous expression node, executes a process of extracting a characteristic partial tree from a sentence dependency structure tree after being substituted to (claim 10).

According to the text mining method, the matching subtree included in the target sentence dependency structure tree is replaced with the synonym expression node, and the feature subtree is extracted from the target sentence dependency tree after replacement.
Therefore, a characteristic subtree can be extracted by equating synonymous expressions with different dependency structure trees.

In the text mining method according to the present invention, the computer of the text mining device generates a target sentence-dependent structure tree from the text database to be mined, reads the expressions stored in the synonym expression dictionary from the storage device, and synonymous expressions. Generates a dependency structure tree, compares the target sentence dependency structure tree with the synonymous expression dependency structure tree to identify whether the matching subtree is included in the target sentence dependency structure tree, and is distinguished from a normal node label A synonymous expression node with the identifier as a label is generated and added to the target sentence dependency structure tree. At this time, add a dependency branch from the synonymous expression node to a node that has a dependency branch from a node in the matching subtree at a node outside the matching subtree, and match at a node outside the matching subtree A dependency branch to a synonymous expression node is added from a node having a dependency branch to a node in the subtree. And you may make it perform the process which extracts a feature subtree from the object sentence dependence structure tree (dependence structure bundle) after the addition of a synonymous expression node and the addition of a dependency branch (claim 11).

According to the text mining method, a synonymous expression node corresponding to a matching subtree included in the target sentence dependency structure tree is added to the target sentence dependency structure tree, and a dependency branch indicating a relationship between the matching subtree and an external node is obtained. An equivalent dependency branch is added to the synonym expression node to generate a dependency structure bundle. Then, a feature subtree is extracted from the dependency structure bundle.
Therefore, a characteristic subtree can be extracted by equating synonymous expressions with different dependency structure trees. Furthermore, since the existing node is left as it is in the target sentence dependency structure tree that is the target of the feature subtree extraction, adverse effects caused by unifying synonymous expressions can be reduced.

In the text mining method, after a synonymous expression node is added to the target sentence dependency structure tree, the synonymous expression node and an exclusive relation branch connecting step of connecting each node included in the matching subtree with an exclusive relation branch, The operation contents of the exclusive relation branch connection step are executed by the computer of the text mining apparatus, and in the feature subtree extraction step, the computer of the text mining apparatus extracts nodes connected by the exclusive relation branch of the dependency structure bundle. The feature subtree may be extracted from a subtree that is not included (claim 12).
In this way, a subtree that does not make sense as an extraction result including both nodes and synonymous expression nodes included in the matching subtree is not extracted as a feature subtree.

  In the text mining program of the present invention, a function for generating a target sentence dependency structure tree from a text database to be mined and a expression stored in the synonym expression dictionary are read from the storage device to generate a synonym expression dependency structure tree. An identifier that distinguishes a function from a normal node label and a function that identifies whether a matching subtree is included in the target sentence dependency structure tree by comparing the target sentence dependency structure tree with the synonymous expression dependency structure tree A function that generates synonymous nodes with labels, a function that replaces all nodes included in the matching subtree with synonymous nodes, and a function that extracts feature subtrees from the target sentence dependency structure tree after replacement Are executed by a computer (claim 13).

According to the text mining program, the matching subtree included in the target sentence dependency structure tree is replaced with the synonymous expression node, and the feature subtree is extracted from the target sentence dependency tree after replacement.
Therefore, it is possible to extract a characteristic partial tree by operating a computer as a text mining device and identifying synonymous expressions having different dependency structure trees.

  In the text mining program according to the present invention, a function for generating a target sentence dependency structure tree from a text database to be mined and a expression stored in the synonym expression dictionary are read from the storage device to generate a synonym expression dependency structure tree. Are distinguished from normal node labels and the function that identifies whether the matching subtree is included in the target sentence dependency structure tree by comparing the target sentence dependency structure tree with the synonymous expression dependency structure tree. The computer executes a function of generating a synonymous expression node having the identifier as a label and adding it to the target sentence dependency structure tree. At this time, add a dependency branch from the synonymous expression node to a node that has a dependency branch from a node in the matching subtree at a node outside the matching subtree, and match at a node outside the matching subtree A dependency branch to a synonym expression node is added from a node having a dependency branch from a node in the subtree. Then, a feature subtree is extracted from the target sentence dependency structure tree (dependency structure bundle) after the addition of the synonymous expression nodes and the dependency branches.

According to the text mining program, a synonymous expression node corresponding to a matching subtree included in the target sentence dependency structure tree is added to the target sentence dependency structure tree, and a dependency branch indicating a relationship between the matching subtree and an external node is obtained. An equivalent dependency branch is added to the synonym expression node to generate a dependency structure bundle. A feature subtree is extracted from the dependency structure bundle.
Therefore, it is possible to extract a characteristic partial tree by operating a computer as a text mining device and identifying synonymous expressions having different dependency structure trees. Furthermore, since the existing node is left as it is in the target sentence dependency structure tree that is the target of the feature subtree extraction, adverse effects caused by unifying synonymous expressions can be reduced.

In the above text mining program, after a synonymous expression node is added to the target sentence dependency structure tree, a function to generate a dependency structure bundle in which each node included in the synonymous expression node and the matching subtree is connected by an exclusive relation branch is added to the computer. The feature subtree may be extracted from subtrees that do not include nodes connected by exclusive relation branches of the dependency structure bundle.
In this way, a subtree that does not make sense as an extraction result including both nodes and synonymous expression nodes included in the matching subtree is not extracted as a feature subtree.

According to the present invention, the node replacement unit replaces the matching subtree included in the target sentence dependency structure tree with the synonymous expression node, and the feature subtree extraction unit extracts the feature subtree from the target sentence dependency tree after replacement. To do.
Therefore, a characteristic subtree can be extracted by equating synonymous expressions with different dependency structure trees.

The configuration and operation of the text mining device 10 according to the first embodiment of the present invention will be described with reference to the drawings.
(Configuration of text mining device 10)
FIG. 5 is a schematic functional block diagram of the text mining apparatus 10.
The text mining device 10 includes an input device 1 such as a keyboard and a mouse, a data processing device 20 that operates under program control, a storage device 30 such as a hard disk that stores information, and an output device 4 such as a display device.

The storage device 30 includes a synonym expression dictionary storage unit 31 and a text set storage unit 32.
The synonym expression dictionary storage unit 31 stores in advance a synonym expression dictionary defined by grouping expressions that are considered synonymous when performing text mining into expressions that are considered synonymous. The text set storage unit 32 stores in advance text to be mined.
In the synonym expression dictionary, synonym expressions belonging to the same synonym expression group are identified with each other when performing text mining.

  FIG. 6 shows an example of the synonym expression dictionary. In this example, at the time of text mining, the expression EX1 “reducing displayed characters”, the expression EX2 “displaying with small characters”, and the expression EX3 “increasing the number of lines to be displayed” are identified (synonymous expression group G1). Furthermore, this is a definition example of the synonym expression dictionary when the expression EX4 “send image by e-mail” and the expression EX5 “attach image to e-mail” are to be identified (synonym expression group G2).

The data processing device 20 includes language analysis means 21, synonymous expression identification means 22, synonymous expression node generation means 23, node replacement means 24, feature subtree extraction means 25, and extraction result output means 26.
The language analysis means 21 constructs a dependency structure tree corresponding to all expressions stored in the synonym expression dictionary storage unit 31 and all sentences in the text stored in the text set storage unit 32.
The dependency structure tree expresses a sentence as a tree structure with a sentence component as a node, a dependency relationship between the sentence components (dependency relationship) as a branch. Each node is distinguished by a label attached to the node.

For example, a morpheme may be adopted as a component of a sentence corresponding to a node, and a dependency structure tree may be constructed in which a prototype of each morpheme is used as a node label and a dependency relationship between morphemes is a branch. As in the example of 1 (a), adopt a clause, build a dependency structure tree with the independence word belonging to the clause converted to a terminal form as a node label, and the dependency relationship between clauses as branches It may be.
In order to construct a dependency structure tree, for example, a generally known method can be used, such as dividing a sentence into morpheme units by performing morphological analysis and obtaining a relationship between morphemes by performing syntax analysis. .
The dependency structure tree constructed by the language analysis means 21 may be held in a temporary storage device such as a DRAM (Dynamic Random Access Memory) (not shown) or may be held in the storage device 30. .

  The synonym expression identifying unit 22 uses a dependency structure tree (target sentence dependency structure tree) corresponding to each sentence in the text stored in the text set storage unit 32 and an expression stored in the synonym expression dictionary storage unit 31. Match the corresponding dependency structure tree (synonymous expression dependency structure tree). As a result, the part where the expression in the synonym expression dictionary is used in the text is specified, and which part in the synonym expression dictionary is used at which part in the text.

The synonym expression node generation means 23 corresponds to the expression in the synonym expression dictionary in the part where the expression in the synonym expression dictionary is used in each sentence in the text, that is, the dependency structure tree corresponding to each sentence in the text. A new node (synonymous expression node) is generated by associating the dependency structure tree (matching subtree) with each of the locations where it is included as a subtree.
An identifier indicating the synonymous expression group to which the expression used at the location belongs is assigned as a label to the node to be generated. That is, a common label is assigned to nodes generated for locations where expressions belonging to the same synonymous expression group are used. Also, the label is a special label different from the node label that originally existed in the dependency structure tree constructed by the language analysis means 21.

  The node replacement means 24 is a synonymous expression node generation means 23 for all the nodes included in the subtree in the dependency structure tree corresponding to each sentence in the text. Replace with the synonym node generated in association with the location. This replacement process deletes a branch in which both the start point and the end point are replaced with the same node. Leave the branch where only one of the start and end nodes is replaced.

An example of processing by the node replacement means 24 is shown in FIG. This corresponds to the expression EX1 “reducing the displayed characters” in the dependency structure tree DT-S1 (FIG. 2) corresponding to the sentence S1 “The method of reducing the displayed characters as much as possible by WEB”. This is an example of replacing all nodes included in the subtree at the location where the dependency structure tree DT-EX1 is included as a subtree (PT1 in FIG. 3) with a new node “G1”. .
In this example, the four nodes “display”, “character”, “small”, and “to” are replaced by the node “G1”. The branches BR1, BR2, and BR3 that have joined these four nodes are deleted. The branch BR4 that has joined the node “mail” and the node “display” is left as it is in the form of joining the node “mail” and the node “G1”. Further, the branch BR5 that has joined the node “as much as possible” and the node “small” is left as it is in the form of joining the node “as possible” and the node “G1”. Further, the branch BR6 that has joined the node “Yes” and the node “Method” is left as it is in the form of joining the node “G1” and the node “Method”.

The characteristic subtree extracting unit 25 extracts a characteristic subtree from the dependency structure tree corresponding to each sentence in the text.
A general data mining method is used to determine whether a subtree is characteristic. For example, a subtree that appears more than a predetermined threshold in all the dependency structure trees can be extracted as a characteristic subtree. In addition, when there are more than a predetermined threshold number of texts in which one or more sentences in which a subtree appears in the corresponding dependency structure tree, the subtree may be extracted as a characteristic subtree. Is possible. In addition, a subtree having a correlation greater than or equal to a predetermined threshold is determined as to whether or not the text belongs to a predetermined set and whether or not a subtree appears in the dependency structure tree corresponding to each sentence in the text. You may make it extract as a characteristic subtree.

  Criteria for determining whether or not a subtree is characteristic (for example, a threshold for the number of appearances, a set of texts for which correlation is obtained, a correlation threshold, etc.) are input by the user through the input device 1. It may be. In addition, a plurality of methods for determining whether or not it is characteristic may be prepared so that the user can select it. In addition, the user may be able to specify the condition of the subtree to be extracted and the text condition for extracting the subtree.

  The extraction result output means 26 outputs the extracted partial tree shape to the output device 4 in a form that the user can visually recognize. At this time, the nodes that originally existed in the dependency structure tree constructed by the language analysis unit 21 are output as they are, and the nodes generated by the synonym expression node generation unit 23 are referred to the synonym expression dictionary storage unit 31. Then, the label is replaced with an expression corresponding to the corresponding synonym expression group and output. Thereby, the special label given by the synonymous expression node generation means 23 can be corrected to a state in which the user can understand.

For the expression used for label replacement, for example, the first expression in the synonym expression group, the expression with the shortest length, and the expression that appears most frequently in the text are automatically selected from the synonym expression dictionary. Can do.
In the synonym expression dictionary, expressions representing the synonym expression groups may be marked in advance and used.
In the synonym expression dictionary, expressions used for label replacement may be stored separately from expressions regarded as synonymous.

Next, the operation of the text mining device 10 will be described in detail.
FIG. 8 is a flowchart showing the operation of the text mining apparatus.
First, the language analysis unit 21 analyzes all expressions stored in the synonym expression dictionary storage unit 31 and constructs a synonym expression dependency structure tree (steps A1 and A2 in FIG. 8).
Next, the language analysis unit 21 analyzes one sentence in the text stored in the text set storage unit 32 and constructs a target sentence dependency structure tree (step A3).
Subsequently, the synonym expression identifying unit 22 compares the target sentence dependency structure tree constructed in step A3 with the synonym expression dependency structure tree constructed in step A1, and the expression recorded in the synonym expression dictionary in the target sentence. Is included (step A4 and step A5).

  If the target sentence dependency structure tree includes a portion where the dependency structure tree (matching subtree) corresponding to the expression in the synonym expression dictionary is included as a subtree, the synonym expression node generation means 23 performs the expression. A special node (synonymous expression node) whose label is an identifier corresponding to the synonym expression group to which the symbol belongs is generated in association with the location (step A6). Further, the node replacement means 24 replaces all nodes included in the matching subtree with synonymous expression nodes (step A7).

  The synonym expression identification unit 22 determines whether or not the collation with all the synonym expression dependency structure trees constructed in step A1 has been completed (step A8). If a synonymous expression dependent structural tree that has not yet been collated remains, the synonymous expression identifying means 22, the synonymous expression node generating means 23, and the node replacing means 24 repeat the operations from step A4 to A7. At this time, after the second round, processing is performed on the structure after the first round processing is performed.

  Further, the language analysis means 21 determines whether or not the analysis has been completed for all sentences in the text stored in the text set storage unit 32 (step A9). If a sentence that has not been analyzed still remains, the language analysis means 21, the synonym expression identification means 22, the synonym expression node generation means 23, and the node replacement means 24 repeat the operations from step A3 to A8.

When the processing so far is completed for all sentences, the feature subtree extracting unit 25 extracts a characteristic subtree from the dependency structure tree corresponding to each sentence (step A10).
Finally, the extraction result output unit 26 outputs the extraction results to the output device 4 in order. First, the synonym expression dictionary storage unit 31 is referred to, and in the extraction result, the label of the synonym expression node generated in step A6 is replaced with a corresponding expression (expression for output) (step A10). Subsequently, the extraction result is output through the output device 4 (step A11). This operation is repeated for all extraction results (step A12).

  In the present embodiment, for the text stored in the text set storage unit 32, the language analyzing unit 21 constructs a dependency structure tree for each sentence, and synonymous expression identifying unit 22 and synonymous expression node generating unit 23. In the above description, the node replacement unit 24 sequentially processes the dependency structure tree. However, the language analysis unit 21 collectively processes the dependency structure tree for all sentences in the text stored in the text set storage unit 32. And the synonymous expression identifying means 22, the synonymous expression node generating means 23, and the node replacing means 24 may each process all the dependency structure trees at once.

Next, a specific operation example of the text mining device 10 will be described.
In this actual operation example, as the dependency structure tree, a tree structure is used in which a clause is a node, an independent word belonging to the clause is changed to a terminal form, a node label is used, and a dependency relationship between clauses is a branch. .
The synonym expression dictionary storage unit 31 stores in advance the contents shown in FIG.
In addition, the text set storage unit 32 stores in advance text to be subjected to text mining. In FIG. 15, a rectangle 303 schematically shows one text, for example, an electronically recorded content of an inquiry sent to a call center. One text includes one or more sentences. The text set storage unit 32 stores a plurality of such texts.

First, the language analysis means 21 analyzes each expression in the synonym expression dictionary storage unit 31 to construct a synonym expression dependency structure tree.
In the present embodiment, each expression is divided into morphemes by morphological analysis, and the dependency relation between clauses is obtained by syntactic analysis to construct a synonymous expression dependent structure tree.
By this processing, the dependency structure tree DT-EX1 (FIG. 1A) from the expression EX1 in FIG. 6, the dependency structure tree DT-EX2 (FIG. 1B) from the expression EX2, and the dependency structure tree DT from the expression EX3 are displayed. -EX3 (FIG. 4 (a)) is constructed from the expression EX4 to the dependency structure tree DT-EX4 (FIG. 16 (a)), and from the expression EX5 to the dependency structure tree DT-EX5 (FIG. 16 (b)). The

  When the construction of the synonymous expression dependency structure tree is completed for all the expressions in the synonym expression dictionary storage unit 31, the language analysis unit 21 subsequently analyzes each sentence included in the text in the text set storage unit 32. Then, the target sentence dependency structure tree is constructed, the synonym expression identifying unit 22 identifies whether or not the matching subtree is included in the target sentence dependency structure tree, and the synonym expression node generating unit 23 includes the matching subtree. A synonymous expression node is generated in association with the location, and the node replacement means 24 performs a process of replacing all the nodes included in the matching subtree with the synonymous expression node.

Here, the example of the said process with respect to sentence S1 in a text "The method to make the character which displays an e-mail as small as possible was investigated by WEB."
First, the language analysis means 21 analyzes this sentence and constructs a target sentence dependency structure tree. As a result, the dependency structure tree DT-S1 shown in FIG. 2 is constructed.
Next, the synonym expression identifying means 22 is synonymous expression dependency structure tree DT-EX1 (FIG. 1 (a)), DT-EX2 (FIG. 1 (b)), DT-EX3 (FIG. 4 (a)), DT- Each of EX4 (FIG. 16 (a)) and DT-EX5 (FIG. 16 (b)) is compared with the dependency structure tree DT-S1 (FIG. 2) corresponding to the sentence S1, and the dependency structure tree DT- It is identified in which part in S1 the dependency structure tree corresponding to which expression is included as a subtree.

As a result of the collation, the dependency structure tree DT-S1 includes the dependency structure tree DT-EX1 as a subtree PT1 as shown in FIG. 3, but the dependency structure trees DT-EX2, DT-EX3, DT are included. -It is identified that EX4 and DT-EX5 are not included.
The synonym expression node generation means 23 generates a synonym expression node in association with the subtree PT1 in FIG. An identifier representing a synonym expression group to which an expression corresponding to the adapted dependency structure tree belongs is assigned as a label to the synonym expression node.

  The dependency structure tree DT-EX1 corresponds to the expression EX1, and since the expression EX1 belongs to the synonym expression group G1, a label “G1” is given here (FIG. 17). This label is a special label that is different from the node label that originally existed in the dependency structure tree constructed by the language analysis means 21. In the drawing, “G1” is underlined to indicate that it is a synonym expression node generated by the synonym expression node generation means 23.

  Next, the node replacement unit 24 replaces all nodes included in the subtree PT1 of the dependency structure tree DT-S1 with the node “G1” generated by the synonymous expression node generation unit 23 (FIG. 7). The node replacement unit 24 deletes the branches BR1 to BR3 in which both the start point and the end point are replaced with the same node, and leaves the branches BR4 to BR6 in which only one of the start point and the end point is replaced. As a result, the dependency structure tree DT-S1 is converted into the dependency structure tree DT-S1R (FIG. 18A).

  The language analysis unit 21, the synonym expression identification unit 22, the synonym expression node generation unit 23, and the node replacement unit 24 operate in this manner, so that the sentence S1 “A method for minimizing the character for displaying the mail is examined by WEB. Dependent structure tree DT-S1 (FIG. 2) is constructed from “”, and finally converted to dependency structure tree DT-S1R (FIG. 18A). The dependency structure tree DT-S1R generated in this way is a target from which the feature subtree extraction unit 25 extracts a characteristic subtree.

  By the same process, from the sentence S2 “double the number of lines for displaying mail”, the dependency structure tree DT-S2 (FIG. 4B) is constructed, and finally the dependency structure tree DT-S2R ( FIG. 18B) is generated. Furthermore, from the sentence S3 “display mail on another screen with small characters”, the dependency structure tree DT-S3 (FIG. 19A) is constructed, and finally the dependency structure tree DT-S3R (FIG. 19 ( b)) is generated.

  When the dependency structure tree DT-S1R (FIG. 18A), the dependency structure tree DT-S2R (FIG. 18B), and the dependency structure tree DT-S3R (FIG. 19B) are compared, these dependency structure trees are compared. Each include the node “G1”. This is used in the expression EX1 used in the sentence S1, the expression EX2 used in the sentence S2, and the sentence S3 by the processing of the synonym expression identifying means 22, the synonymous expression node generating means 23, and the node replacing means 24. This shows that the difference in the expression EX3 is absorbed, and both are now represented as a single node “G1” in the dependency structure tree.

  Also, from the sentence S4 “send the photographed image to two people by mail”, the dependency structure tree DT-S4 (FIG. 20A) is constructed, and finally the dependency structure tree DT-S4R (FIG. 20B). )) Is generated. From the sentence S5 “Attach image taken by camera to mail”, dependency structure tree DT-S5 (FIG. 21A) is constructed, and finally dependency structure tree DT-S5R (FIG. 21B). Is generated.

  Comparing the dependency structure tree DT-S4R (FIG. 20B) and the dependency structure tree DT-S5R (FIG. 21B), these dependency structure trees all include the node “G2”. This is because the difference between the expression EX4 used in the sentence S4 and the expression EX5 used in the sentence S5 is absorbed by the processing of the synonym expression identifying unit 22, the synonym expression node generating unit 23, and the node replacing unit 24. Is also represented as a single node “G2” in the dependency structure tree.

When the language analysis unit 21, the synonym expression identification unit 22, the synonym expression node generation unit 23, and the node replacement unit 24 repeat the processing to generate a dependency structure tree for all sentences in each text in the text set storage unit 32, The characteristic subtree extracting unit 25 extracts a characteristic subtree for all generated dependency structure trees.
Here, it is assumed that a subtree that appears 50 times or more is extracted as a characteristic subtree in all dependent structure trees after node replacement. In this case, for example, the feature subtree extracting unit 25 lists all types of subtrees included in the dependency structure tree, counts the number of appearances of each, and extracts subtrees with the number of appearances of 50 or more. Can do.

When the extraction of the characteristic partial tree is completed, the extraction result output unit 26 outputs the extraction results in order. At this time, the nodes that originally existed in the dependency structure tree constructed by the language analysis unit 21 are output as they are, and the labels generated by the synonym expression node generation unit 23 are labeled with the corresponding synonym expression groups. Replace with the expression according to the output.
In this operation example, it is assumed that the node label generated by the synonym expression node generation unit 23 is replaced using the first expression in the corresponding synonym expression group.

An example in which the extraction result includes the dependency structure tree DT-R1 shown in FIG. 22A and the extraction result output unit 26 outputs the dependency structure tree DT-R1 will be described.
In the dependency structure tree DT-R 1, the node “mail” is a node that originally existed in the dependency structure tree constructed by the language analysis unit 21, but the node “G 1” is generated by the synonym expression node generation unit 23. The generated node. Since the first expression in the synonym expression group G1 (see FIG. 6) is the expression EX1 “reducing the displayed character”, the label of the node “G1” is replaced by this expression EX1, and FIG. The result is output as follows.

The part corresponding to the expression EX1 in the sentence S1, the part corresponding to the expression EX3 in the sentence S2, and the part corresponding to the expression EX2 in the sentence S3 are all represented and identified by the same node “G1”. In this state, characteristic subtrees are extracted. Further, both the part corresponding to the expression EX4 in the sentence S4 and the part corresponding to the expression EX5 in the sentence S5 are represented by the same node “G2”, and these are also characteristic parts in the state of being identified. Tree extraction is performed. In addition, since the label of the added node is replaced with an appropriate expression corresponding to the expression identified by each node at the time of output, the user can easily perform the identification even if the expression is identified. Understand the results.
In addition, these nodes “G1” and “G2” are different from the nodes in the original dependency structure tree in which the independent words belonging to the phrase are converted to the final form, and thus the original dependency structure. A subtree that can be distinguished from nodes in the tree and is not inherently characteristic is not erroneously extracted.

As described above, the text mining device 10 replaces the matching subtree of the target sentence dependent structure tree with the synonymous expression node generated by the synonymous expression node generating means 22 before the feature structure is extracted. The subtree extracting means 25 can extract characteristic subtrees by equating synonymous expressions.
Note that the structure after replacement is also a tree structure, and the nodes are not increased by the replacement, so that the cost for extracting a characteristic subtree does not increase compared to before replacement.
In addition, when unifying the structure of the part corresponding to the synonymous expression in the target sentence dependency structure tree, it is not converted to another dependency structure tree, but replaced by a single node called a synonym expression node. Synonymous expressions can be identified without describing conversion rules between structural trees. It is also possible to identify synonymous expressions that cannot be converted between corresponding dependency structure trees.
Furthermore, since the synonymous expression node added by the node replacement means is a special node different from the existing nodes, it does not affect the extraction of the subtree consisting only of the existing nodes. As a result, side effects caused by unifying synonymous expressions can be suppressed.

  Next, the configuration and operation of the text mining device 11 according to the second embodiment of the present invention will be described with reference to the drawings. Since many elements of the text mining device 11 are common to the text mining device 10, the common elements are denoted by the same reference numerals and description thereof is omitted.

(Configuration of text mining device 11)
FIG. 9 is a schematic functional block diagram of the text mining apparatus 11.
The text mining device 11 is different from the data processing device 20 of the text mining device 10 in FIG. 5 in the configuration of the data processing device 29. The data processing device 29 replaces the node replacement unit 24 and the feature subtree extraction unit 25 of FIG. 5 with a bundle forming unit (node addition unit and exclusive branch addition unit) 27 and a feature subtree extraction unit (feature subtree). Extraction means) 28.

The bundle constructing unit 27 adds a synonymous expression node when a matching subtree is included in the target sentence dependency structure tree. The bundle forming unit 27 further adds a branch equivalent to the case where all the nodes included in the matching subtree are replaced with the synonymous expression nodes to the dependency structure tree.
That is, for a matching subtree, if there is a branch in the original dependency structure tree that starts from a node included in the matching subtree and ends in a node not included in the matching subtree, the end point from the synonymous expression node Add a branch to to the dependency tree. If a branch starting from a node not included in the matching subtree and ending in a node included in the matching subtree exists in the original dependency structure tree, a branch from the starting point to the synonym expression node is determined. Add to dependency tree.

At this time, the bundle forming unit 27 associates each node included in the matching subtree and the synonymous expression node with each other by an exclusive relationship branch and associates them as nodes having an exclusive relationship.
The dependency structure tree is converted from a tree structure to a bundle structure by the processing of the bundle forming unit 27. Hereinafter, the converted structure is called a dependency structure bundle.

An example of processing by the bundle forming unit 27 is shown in FIG. This corresponds to the expression EX1 “reducing the displayed characters” in the dependency structure tree DT-S1 (FIG. 2) corresponding to the sentence S1 “The method of reducing the displayed characters as much as possible by WEB”. This is an example in which a branch is added when the node “G1” is generated in association with a location (PT1 in FIG. 10) where the dependency structure tree DT-EX1 is included as a subtree.
In this example, since there is a branch BR6 starting from the node “Yes” included in PT1 and ending in the node “method” outside PT1, the branch BR7 from the node “G1” to the node “method” is added. The
Further, a branch BR4 whose starting point is the node “mail” outside PT1 and whose end point is the inner node “display”, and a branch BR5 whose starting point is “possible” outside node PT1 and whose inner node “small” is the end point. Therefore, the branch BR8 and the branch BR9 from the node “mail” and the node “as much as possible” to the node “G1” are added.
Also, the four nodes “display”, “character”, “small”, “do” in PT1 and the node “G1” are connected to each other by the exclusive relationship branches BR10 to BR13 and are associated as exclusive nodes. In FIG. 10, the exclusive relationship is indicated by a dotted line.

Here, nodes that are mutually exclusive means a node that cannot be interpreted as both appearing simultaneously. The synonym expression node generated by the synonym expression node generation means 23 for the matching subtree is a node corresponding to the fact that the whole subtree is grouped and the expression in the synonym expression dictionary appears. It is.
On the other hand, each node in the matching subtree is a node corresponding to the interpretation that the constituent elements constituting the expression appear individually.
Accordingly, since it cannot be interpreted that both appear simultaneously, the bundle forming unit 27 associates both as nodes that are mutually exclusive.

  If there is already a synonymous expression node that is in an exclusive relationship with the nodes included in the matching subtree, the synonymous expression node is also associated with the newly generated synonymous expression node as a node that is mutually exclusive. . This is because the expressions represented by both have the same node as a component and cannot be interpreted as both appearing simultaneously.

The bundle feature subtree extraction means 28 extracts a characteristic subtree from the dependency structure tree constructed by the language analysis means 21 and the dependency structure bundle constructed by the bundle construction means 27. When a dependency structure bundle is configured from the dependency structure tree, the original dependency structure tree is not used for extracting a characteristic subtree. Also, subtrees that include a plurality of nodes that are mutually exclusive are not extracted.
For example, in the dependency structure bundle DL-S1 shown in FIG. 11, when the node “display”, the node “character”, the node “small”, and the node “enable” are connected to the node G1 through exclusive relation branches, respectively, The dependency structure tree PT2 shown in FIG. 12 includes the nodes “Yes” and “G1” that are in an exclusive relationship, and therefore the bundle feature subtree extraction unit 18 does not extract them.

  Also in the present embodiment, a general data mining technique can be used to determine whether a certain subtree is characteristic. For example, a subtree that appears more than a predetermined threshold in all the dependency structure trees can be extracted as a characteristic subtree. In addition, when there is a text having one or more sentences in which a certain subtree appears in the corresponding dependency tree, the subtree is extracted as a characteristic subtree when there are more than a predetermined threshold. Is also possible. In addition, a subtree having a correlation greater than or equal to a predetermined threshold is determined as to whether or not the text belongs to a predetermined set and whether or not a subtree appears in the dependency structure tree corresponding to each sentence in the text. Alternatively, it may be extracted as a characteristic subtree.

  Also in the present embodiment, as in the first embodiment, criteria for determining whether or not it is characteristic (for example, a threshold for the number of appearances, a set of texts for which correlation is obtained, a correlation threshold, etc. ) May be input by the user through the input device 1. In addition, a plurality of methods for determining whether or not it is characteristic may be prepared so that the user can select it. In addition, the user may be able to specify the condition of the subtree to be extracted and the text condition for extracting the subtree.

Next, the operation of the text mining device 11 will be described in detail.
FIG. 13 is a flowchart showing the operation of the text mining device 11. The operations of Step A1 to Step A6 are the same as those of the text mining device 10.

In step B1, when the bundle constructing unit 27 includes a matching subtree corresponding to the synonymous expression dependency structure tree in the target sentence dependency structure tree constructed in step A3, a node of the target sentence dependency structure tree, A branch is added between the synonymous expression nodes generated in step A6 to form a dependency structure bundle. That is, if there is a branch in the original dependency structure tree starting from a node included in the matching subtree and ending in a node not included in the matching subtree, the branch from the synonymous expression node to the end point If there is a branch in the original dependency structure tree that starts from a node that is not included in the matching subtree and ends in a node that is included in the matching subtree, from that start point to the synonymous expression node Add a branch.
In step B2, the bundle forming unit 27 associates each node included in the matching subtree and the synonymous expression node as nodes that are mutually exclusive.
In step B3, the bundle feature subtree extracting unit 28 extracts a characteristic subtree from the dependency structure tree or the dependency structure bundle corresponding to each sentence. However, subtrees that include a plurality of nodes that are mutually exclusive are not extracted.

Next, a specific operation example of the text mining device 11 will be described.
Similarly to the operation example of the text mining apparatus 10 in this embodiment, the dependency structure tree is obtained by changing a clause as a node and a self-standing word belonging to the clause as a terminal form as a node label. A tree structure with branches is adopted.
Further, the contents shown in FIG. 23 are stored in the synonym expression dictionary storage unit 31 in advance.
In addition, the text set storage unit 32 stores in advance text to be subjected to text mining with the contents shown in FIG.

Also in the present embodiment, as in the case of the text mining device 10, first, the language analysis unit 21 analyzes each expression in the synonym expression dictionary storage unit 31 and constructs a synonym expression dependency structure tree.
With this processing, the expression EX1 to the dependency structure tree DT-EX1 (FIG. 1A), the expression EX2 to the dependency structure tree DT-EX2 (FIG. 1B), and the expression EX3 to the dependency structure tree DT-EX3 ( In FIG. 4A, a dependency structure tree DT-EX6 (FIG. 24A) is constructed from the expression EX6, and a dependency structure tree DT-EX7 (FIG. 24B) is constructed from the expression EX7.

  Subsequently, the language analysis unit 21 analyzes each sentence included in the text in the text set storage unit 32 to construct a target sentence dependency structure tree, and the synonym expression identification unit 22 corresponds to the expression in the synonym expression dictionary. The matching subtree is identified as being included in the target sentence dependency structure tree, and the synonym expression node generation unit 23 generates a synonym expression node in association with the matching subtree.

Here, first, an explanation will be given by taking as an example the processing for the sentence S1 in the text “The method for making the letters for displaying mail as small as possible was examined by WEB”.
First, the dependency analysis tree DT-S1 (FIG. 2) is constructed by the language analysis means 21. Subsequently, by the synonymous expression identifying means 22, each dependency structure tree DT-EX1 (FIG. 1 (a)), DT-EX2 (FIG. 1 (b)), DT-EX3 (FIG. 4 (a)), DT-EX6 (FIG. 24A) and DT-EX7 (FIG. 24B) are collated with this dependency structure tree DT-S1 in order.
As a result, it is identified that the dependency structure tree DT-EX1 (FIG. 1 (a)) is included as a subtree in the dependency structure tree DT-S1 (FIG. 3). A new synonymous expression node “G1” is generated in association with the portion (FIG. 17). Up to this point, the same processing as in the first embodiment is performed.

Next, the bundle forming unit 27 adds a branch between the synonymous expression node “G1” and the node in the original dependency structure tree DT-S1 to form a dependency structure bundle (FIG. 10). At this time, paying attention to the matching subtree PT1 that matches the dependency structure tree DT-EX1, a branch having a node in the subtree PT1 as a start point and a node outside the subtree PT1 as an end point is included in the dependency structure tree DT-S1. If there is a branch from the node “G1” to its end point, a branch having a node outside the subtree PT1 as a start point and a node in the subtree PT2 as an end point is included in the dependency structure tree DT-S1. If it exists, a branch from the starting point to the node “G1” is added.
In FIG. 10, since there is a branch BR6 starting from the inner node “Yes” and having the outer node “Method” as the end point in the subtree PT1 that matches the left dependency structure tree DT-EX2, the node “G1” exists. A BR7 branch from to the node “method” is added. Also, the branch node BR4 starting from the outer node “mail” of the subtree PT1 and the inner node “display” as the end point, and the outer node “possible” of the subtree PT1 as the starting point, the inner node “ Since there is a branch BR5 whose end point is “small”, the bundle forming unit 27 adds the branch “BR” and the branch BR8 and BR9 from the node “mail” and the node “G1” to the node “G1”, and the dependency structure bundle DL-S1 ( FIG. 11) is constructed.

The bundle forming unit 27 further associates each node included in the subtree matching the dependency structure tree DT-EX1 and the node “G1” as nodes that are mutually exclusive. As a result, the four nodes “Yes”, “Small”, “Character”, “Display” and the node “G1” existing inside the dotted line portion in FIG. 17 are associated with the node “G1” as mutually exclusive nodes. In FIG. 11, the exclusive relationship is indicated by a dotted line.
The dependency structure bundle DL-S1 (FIG. 11) configured in this way is continuously checked against the dependency structure tree corresponding to the expression in the synonym expression dictionary by the synonym expression identification unit 22, Since no dependency structure tree exists, this dependency structure bundle DL-S1 is finally a target from which the bundle feature subtree extraction unit 28 extracts characteristic subtrees.

Next, assuming that there is a text in the text set storage unit 32 that includes the sentence S3 “A method for making a character for displaying mail as small as possible” is described in the text set storage unit 32, the processing for this sentence S3 will be described.
First, the dependency analysis tree DT-S3 (FIG. 19A) is constructed by the language analysis means 21. Subsequently, by the synonymous expression identifying means 22, each dependency structure tree DT-EX1 (FIG. 1 (a)), DT-EX2 (FIG. 1 (b)), DT-EX3 (FIG. 4 (a)), DT-EX6 (FIG. 24A) and DT-EX7 (FIG. 24B) are collated in order with this dependency structure tree DT-S3.
As a result, it is identified that the dependency structure tree DT-EX2 (FIG. 1B) is included as a subtree PT3 in the dependency structure tree DT-S3. Correspondingly, a new node “G1” is generated (FIG. 25A).

Next, the bundle forming unit 27 adds a branch between the synonymous expression node “G1” and the node in the original dependency structure tree DT-S3 to form a dependency structure bundle. At this time, paying attention to the matching subtree PT3 conforming to the dependency structure tree DT-EX2, a branch having a node in the subtree PT3 as a start point and a node outside the subtree PT3 as an end point exists in the dependency structure tree DT-S3. When adding a branch from the node “G1” to its end point, a branch having a node outside the subtree PT3 as a start point and a node inside the subtree PT3 as an end point exists in the dependency structure tree DT-S3. If so, a branch from the starting point to the node “G1” is added.
In FIG. 25 (a), the branch BR14 starting from the outer node “mail” of the subtree PT3 conforming to the dependency structure tree DT-EX2 and ending at the inner node “display” and the subtree PT3 Since there is a branch BR15 having the outer node “screen” as the start point and the inner node “display” as the end point, the bundle forming unit 27 changes the node “mail” and the node “screen” to the node “G1”. Branches BR16 and BR17 are added to form a dependent structure bundle DL-S3A (FIG. 25B).

The bundle forming unit 27 further associates each node included in the subtree PT3 with the node “G1” as nodes that are mutually exclusive. As a result, the three nodes “display”, “character”, “small”, and node “G1” that existed inside the subtree PT3 in FIG. 25A are mutually exclusive by the branches BR18, BR19, BR20. Connected as nodes that are connected and exclusive.
The dependency structure bundle DL-S3A (FIG. 25 (b)) thus configured is continuously checked against the dependency structure tree corresponding to the expression in the synonym expression dictionary by the synonym expression identifying unit 22. It is identified that the dependency structure tree DT-EX6 (FIG. 24A) corresponding to the expression EX6 exists as the subtree PT4 in this dependency structure bundle DL-S3A (FIG. 25B). Since the expression EX6 belongs to the synonym expression group G3, the synonym expression node generation unit 23 generates a new node “G3” in association with the portion (FIG. 26).

  The bundle forming means 27 performs the same processing as when a branch is added between the node in the dependency structure tree DT-S3 and the node “G1”, and the node and node “G3” in the dependency structure tree DL-S3A. To add a branch. According to FIG. 26, a branch BR21 having a node “mail” outside the subtree PT4 conforming to the dependency structure tree DT-EX6 as a start point and an end point “display” inside the subtree PT4 and a node “character” outside the subtree PT4 ”And a branch BR22 having an inner node“ display ”as an end point exists, and the bundle forming unit 27 adds branches BR23 and BR24 from the node“ mail ”and the node“ character ”to the node“ G3 ”. The dependency structure bundle DL-S3B (FIG. 27) is configured.

The bundle forming unit 27 further associates each node included in the subtree conforming to the dependency structure tree DT-EX6 and the node “G3” as nodes that are mutually exclusive. As a result, the three nodes “display”, “screen”, “other”, and node “G3” that existed inside the subtree PT4 in FIG. 26 are associated as nodes that are mutually exclusive, and the exclusive relationship branch They are connected by BR25, BR26, and BR27, respectively.
At this time, since the node “G1” is already associated with the node “display” among the nodes included in the subtree PT4, the node “G1” and the node “G3” are also associated with each other. They are associated as nodes that are mutually exclusive and are connected by an exclusive branch BR28.

  In this embodiment, in the dependency structure tree DT-S3 (FIG. 19A), it is identified that the subtree consisting of the node “small”, the node “character”, and the node “display” matches the dependency structure tree DT-EX2. At the same time, the subtree consisting of the node “different”, the node “screen”, and the node “display” is identified as matching the dependency structure tree DT-EX6. In the present embodiment, since the node generated by the synonymous expression node generation unit 23 does not replace the node of the original dependency structure tree, the single node “display” is expressed as one expression EX2. The part can also be taken as part of the expression EX6.

The dependency structure bundle DL-S3B (FIG. 27) thus configured is continuously checked against the dependency structure tree corresponding to the expression in the synonym expression dictionary by the synonym expression identification unit 22, but is included elsewhere. Since there is no dependency structure tree, this dependency structure bundle DL-S3B is finally a target from which the feature subtree extracting unit 28 extracts characteristic subtrees.
In this way, the node “G1” is newly added to the location where the expressions EX1, EX2 and EX3 are used by the processes of the synonym expression identifying unit 22, the synonym expression node generating unit 23, and the bundle forming unit 27. Added to dependency tree. Expression EX1, expression EX2, and expression EX3 are all represented as a single node “G1”, and their differences are absorbed. Similarly, expressions EX6 and EX7 are also represented as a single node “G3” and their differences are absorbed.

  The language analysis unit 21, synonym expression identification unit 22, synonym expression node generation unit 23, and bundle formation unit 27 repeat the processing, and a dependency structure tree or dependency structure bundle for all sentences in each text in the text set storage unit 32. Is generated, the feature subtree extracting means for bundle 28 extracts a characteristic subtree for the generated dependency structure tree or dependency structure bundle. At this time, when a dependency structure bundle is configured from the dependency structure tree, the original dependency structure tree is not used for extracting a characteristic subtree. Also, subtrees that include a plurality of nodes that are mutually exclusive are not extracted.

Here, it is assumed that a dependent structure tree from which a characteristic subtree is to be extracted and a subtree that appears 50 times or more in total in the dependent structure bundle are extracted as a characteristic subtree. In this case, for example, it is possible to list all types of subtrees included in the dependency structure tree and the dependency structure bundle, count the number of appearances of each, and extract subtrees with the appearance count of 50 or more.
The extraction result output means 26 sequentially outputs the subtrees extracted in this way. At this time, the nodes that originally existed in the dependency structure tree constructed by the language analysis unit 21 are output as they are, and the nodes generated by the synonym expression node generation unit 23 correspond to the corresponding synonym expression groups. Replace the label with the output.

As in the case of the text mining device 10, if the label is replaced using the first expression in the corresponding synonym expression group, for example, an extraction like the dependency structure tree DT-R2 shown in FIG. As a result, the label of the node “G3” is replaced by the expression EX6 “display on another screen” which is the first expression of the synonym expression group G3 (see FIG. 23), and the result is output as shown in FIG. The
Also in this embodiment, as in the case of the text mining device 10, all the parts corresponding to expressions belonging to the same synonym expression group in the synonym expression dictionary are represented by the same node, and these are regarded as being the same. The characteristic sub-tree is extracted at. In addition, since the label of the added node is replaced with an appropriate expression corresponding to the expression identified by each node at the time of output, the user can easily perform the identification even when the expression is identified. Understand the results.
Similar to the first embodiment, the added nodes “G1” and “G3” are different from the nodes in the original dependency structure tree, and thus are not mistakenly regarded as characteristic.

  Further, when the expressions are identified, a part corresponding to the dependency structure tree corresponding to the expression is not deleted, and thus a characteristic subtree can be extracted from the part. For example, in the dependency structure bundle DL-S1 of FIG. 11, the four nodes included in the dependency structure tree DT-EX1 (FIG. 1A) corresponding to the expression EX1 “do” “small” “character” “display”. Remains as it is, and that part is also the target for extracting characteristic subtrees.

  Furthermore, since subtrees that contain nodes that are mutually exclusive are not extracted, leaving both the structure before unification and the structure after unification does not make sense as an extraction result that includes both. Does not extract subtrees. For example, in the dependency structure bundle DL-S1 of FIG. 11, the node “Yes” and the node “G1” are mutually exclusive, so a subtree that does not make sense as an extraction result as shown in FIG. 12 is not extracted.

As described above, in the text mining device 11, when unifying the structure of the part corresponding to the synonymous expression in the target sentence dependent structure tree, the bundle forming unit 27, instead of replacing the matching subtree with the synonymous expression node, A syntactic expression node is added to the target sentence dependency structure tree to generate a bundle structure, and the existing nodes are prevented from being deleted due to the unification of the structure.
For this reason, the tree structure of the portion corresponding to the synonymous expression is not lost, and the feature subtree extracting unit 28 for bundling can extract the characteristic subtree from the portion.
Further, when unifying the structures of the parts corresponding to the synonymous expressions, the bundle forming unit 27 associates the synonymous expression nodes and the nodes in the matching subtree with exclusive relation branches and associates them as nodes in the exclusive relation. The bundling feature subtree extracting means 28 does not extract subtrees including nodes that are mutually exclusive when extracting a characteristic subtree.
For this reason, even if both the structure before unification and the structure after unification are left, the subtree which does not make sense as an extraction result which includes both is not extracted.
Thus, according to the present embodiment, in addition to the effect of the first embodiment, an effect that the side effects caused by unifying synonymous expressions can be further reduced can be obtained.

The text mining device 10 and the text mining device 11 described above can also be realized by a computer and a program that operates the computer.
FIG. 14 is a diagram for explaining such an embodiment.
The computer 40 includes a storage device 30, an output device 4, an input device 1, a CPU (Central Processing Unit) 41, and a main storage device 42. The storage device 1 is a hard disk device, for example, and includes a synonym expression dictionary storage unit 31 that stores a synonym expression dictionary and a text set storage unit 32 that stores a text set to be mined. The main storage device 42 is constituted by a RAM (Random Access Memory), for example, and stores a text mining program 43.
The text mining program 43 stored in the main storage device 42 is read and executed by the CPU 41.
Here, the text mining program 43 is a program that causes a computer to execute the operations described above.
In this way, the CPU 41 functions as the language analysis means 21, synonymous expression identification means 22, synonymous expression node generation means 23, node replacement means 24, feature subtree extraction means 25, and extraction result output means 26. And the computer 40 can be operated as the text mining device 10.
Similarly, the data processing device 29 functions as the CPU 41 as the language analysis unit 21, the synonym expression identification unit 22, the synonym expression node generation unit 23, the bundle formation unit 27, the bundle feature subtree extraction unit 28, and the extraction result output unit 26. The computer 40 can be operated as the text mining device 11 by operating.

FIG. 1A is a diagram illustrating a dependency structure tree corresponding to the expression “reducing the displayed characters”. FIG. 1B is a diagram illustrating a dependency structure tree corresponding to the expression “display with small characters”. It is a figure which shows the dependence structure tree corresponding to the sentence "The method of reducing the character which displays an email was investigated by WEB." It is a figure which shows the part which fits the dependence structure tree of Fig.1 (a) in the dependence structure tree of FIG. FIG. 4A is a diagram illustrating a dependency structure tree corresponding to the expression “increase the number of rows to be displayed”. FIG. 4B is a diagram illustrating a dependency structure tree corresponding to the expression “increasing the number of lines for displaying mail twice”. It is a block diagram which shows the structure of the text mining device which is the 1st Embodiment of this invention. It is a figure which shows an example of a synonym expression dictionary. It is a figure which shows the example which a node replacement means replaces a node with respect to a dependency structure tree. It is a flowchart which shows operation | movement of a text mining apparatus. It is a block diagram which shows the structure of the text mining device which is the 2nd Embodiment of this invention. It is a figure which shows the example which a bundling means adds a branch with respect to a dependence structure tree. It is a figure which shows the dependency structure bundle | flux produced | generated when the bundle | stack formation means adds a branch with respect to a dependency structure tree. It is a figure which shows the example of the subtree which is not extracted from the dependence structure bundle | flux of FIG. 10 is a flowchart showing the operation of the text mining device of FIG. 9. It is a figure which shows embodiment of this invention by a computer and a computer program. It is a figure which shows an example of a text set. FIG. 16A is a diagram showing a dependency structure tree corresponding to the expression “send image by e-mail”. FIG. 16B is a diagram illustrating a dependency structure tree corresponding to the expression “attach an image to an email”. It is a figure which shows the example which a synonymous expression node production | generation means produces | generates a new node with respect to the dependence structure tree of FIG. FIG. 18A is a diagram showing a dependency structure tree generated by the node replacement means replacing nodes in the dependency structure tree shown in FIG. FIG. 18B is a diagram showing the dependency structure tree generated by the node replacement means replacing nodes with respect to the dependency structure tree of FIG. FIG. 19A is a diagram illustrating a dependency structure tree corresponding to the sentence “display mail on a different screen with small characters”. FIG. 19B is a diagram illustrating the dependency structure tree after node replacement is performed on the dependency structure tree of FIG. FIG. 20A is a diagram showing a dependency structure tree corresponding to the sentence “send a photographed image to two people by mail”. FIG. 20B is a diagram illustrating the dependency structure tree after node replacement is performed on the dependency structure tree of FIG. FIG. 21A is a diagram illustrating a dependency structure tree corresponding to the sentence “Attach an image captured by a camera to an email”. FIG. 21B is a diagram illustrating the dependency structure tree after node replacement is performed on the dependency structure tree of FIG. FIG. 22A is a diagram illustrating an example of a dependency structure tree as an extraction result. FIG. 22B is a diagram illustrating an output example of the extraction result. It is a figure which shows another example of a synonym expression dictionary. FIG. 24A is a diagram illustrating a dependency structure tree corresponding to the expression “display on another screen”. FIG. 24B is a diagram illustrating a dependency structure tree corresponding to the expression “divide screens to be displayed”. FIG. 25A is a diagram illustrating an example of generating synonymous expression nodes for the dependency structure tree of FIG. FIG. 25B is a diagram illustrating an example of generating a dependency structure bundle from the dependency structure tree of FIG. FIG. 20 is a diagram illustrating a dependency structure bundle generated by adding a branch to the dependency structure tree of FIG. 19 by the bundle forming unit. FIG. 27 is a diagram showing a dependency structure bundle generated by adding a branch to the dependency structure tree of FIG. FIG. 28A is a diagram illustrating an example of a dependency structure tree as an extraction result. FIG. 28B is a diagram illustrating an output example of the extraction result.

Explanation of symbols

10, 11: Text mining device 21: Language analysis means 22: Synonym expression identification means 23: Synonym expression node generation means 24: Node replacement means 25: Feature subtree extraction means 26: Extraction result output means 27: Bundle construction means 28: Feature subtree extracting means 31 for bundling: synonymous expression dictionary storage unit

Claims (15)

Synonymous expression dictionary storage means for storing a synonymous expression dictionary that defines different expressions with synonymous contents as synonymous expression groups;
A target sentence dependency structure tree that is a dependency structure tree of each sentence included in a sentence set that is a target of text mining is compared with a synonym expression dependency structure tree that is a dependency structure tree of each expression included in the synonym expression dictionary, Synonym expression identifying means for identifying whether or not a matching subtree that is a subtree matching the synonymous expression dependency structure tree is included in the target sentence dependency structure tree;
Synonymous expression node generating means for generating a synonymous expression node having an identifier that indicates the synonymous expression group to which the expression corresponding to the matching subtree belongs and that is distinguished from a normal node label;
Node replacement means for replacing all nodes included in the matching subtree with the synonymous expression nodes;
A text mining device comprising: a feature subtree extracting unit that extracts a feature subtree from the target sentence dependency structure tree after the replacement. Synonymous expression dictionary storage means for storing a synonymous expression dictionary that defines different expressions with synonymous contents as synonymous expression groups;
A target sentence dependency structure tree that is a dependency structure tree of each sentence included in a sentence set that is a target of text mining is compared with a synonym expression dependency structure tree that is a dependency structure tree of each expression included in the synonym expression dictionary, Synonym expression identifying means for identifying whether or not a matching subtree that is a subtree matching the synonymous expression dependency structure tree is included in the target sentence dependency structure tree;
Synonymous expression node generating means for generating a synonymous expression node having an identifier that indicates the synonymous expression group to which the expression corresponding to the matching subtree belongs and that is distinguished from a normal node label;
The synonym expression node is added to the target sentence dependency structure tree, and the synonym expression is obtained from a node that is not included in the matching subtree and has a dependency branch toward the node included in the matching subtree. A dependency branch toward the node is added, from the synonymous expression node toward a node that is not included in the matching subtree and has a dependency branch directed from the node included in the matching subtree. A node adding means for generating a dependency structure bundle by adding a dependency branch;
A text mining device comprising: a feature subtree extracting unit that extracts a feature subtree from the dependency structure bundle. An exclusive branch connecting means for connecting the nodes included in the synonymous expression nodes and the matching subtree with exclusive branches;
3. The text mining apparatus according to claim 2, wherein the feature subtree extracting unit extracts the feature subtree from a subtree not including nodes connected by the exclusive relation branch of the dependency structure bundle.   A label of the synonym expression part node included in the feature subtree is replaced with an output expression representative of an expression belonging to the synonym expression group indicated by the label, and the shape of the feature subtree is visible to humans. 4. The text mining apparatus according to claim 1, further comprising an extraction result output means for outputting.   5. The text mining device according to claim 4, wherein the output expression is an expression first listed in the synonym expression group indicated by a label of the synonym expression node.   The text mining device according to claim 4, wherein the output expression is an expression having the shortest length in the synonym expression group indicated by the label of the synonym expression node.   5. The output expression is an expression that appears most frequently in a sentence set that is a target of the text mining in the synonym expression group indicated by a label of the synonym expression node. Text mining device.   5. The text mining device according to claim 4, wherein the output expression is an expression designated in advance as an expression for output in the synonym expression group indicated by the label of the synonym expression node. . 5. The output expression is an expression defined in advance separately from expressions included in the synonym expression group corresponding to the synonym expression group indicated by the label of the synonym expression node. The text mining device described. In a text mining method executed by a text mining device that analyzes a sentence included in a text database to generate a target sentence dependency structure tree and extracts a feature subtree from the target sentence dependency structure tree,
A language analysis step of reading out expressions stored in a synonym expression dictionary that defines different expressions as synonym expressions as synonym expression groups from a storage device and generating a synonym expression dependency structure tree that is a dependency structure tree of the expression;
A synonym for collating the target sentence dependency structure tree with the synonym expression dependency structure tree and identifying whether or not a matching subtree that is a subtree matching the synonym expression dependency structure tree is included in the target sentence dependency structure tree An expression identification step;
A synonym expression node generation step of generating a synonym expression node having an identifier that indicates the synonym expression group to which the expression corresponding to the matching subtree belongs and is distinguished from a normal node label;
A node replacement step of replacing all nodes included in the matching subtree with the synonymous nodes;
A feature subtree extraction step of extracting a feature subtree from the target sentence-dependent structure tree after the replacement ,
A text mining method , wherein the operation content of each step is executed by a computer of the text mining apparatus . In a text mining method executed by a text mining device that analyzes a sentence included in a text database to generate a target sentence dependency structure tree and extracts a feature subtree from the target sentence dependency structure tree,
A language analysis step of reading out expressions stored in a synonym expression dictionary that defines different expressions as synonym expressions as synonym expression groups from a storage device and generating a synonym expression dependency structure tree that is a dependency structure tree of the expression;
A synonym for collating the target sentence dependency structure tree with the synonym expression dependency structure tree and identifying whether or not a matching subtree that is a subtree matching the synonym expression dependency structure tree is included in the target sentence dependency structure tree An expression identification step;
A synonym expression node generation step of generating a synonym expression node having an identifier that indicates the synonym expression group to which the expression corresponding to the matching subtree belongs and is distinguished from a normal node label;
The synonym expression node is added to the target sentence dependency structure tree, and the synonym expression is obtained from a node that is not included in the matching subtree and has a dependency branch toward the node included in the matching subtree. A dependency branch toward the node is added, from the synonymous expression node toward a node that is not included in the matching subtree and has a dependency branch directed from the node included in the matching subtree. A node addition step for generating a dependency structure bundle by adding a dependency branch;
A feature subtree extraction step for extracting a feature subtree from the dependency structure bundle ,
A text mining method , wherein the operation content of each step is executed by a computer of the text mining apparatus . After a synonym expression node is added to the target sentence dependency structure tree in the node addition step, the synonym expression node and an exclusive relation branch connection step of connecting each node included in the matching subtree with an exclusive relation branch,
The computer of the text mining device executes the operation content of the exclusive relation branch connection step,
12. The feature subtree extraction step, wherein the computer of the text mining apparatus extracts the feature subtree from a subtree that does not include nodes connected by the exclusive relation branch of the dependency structure bundle. The text mining method described in. In a text mining program that causes a computer to execute a function of analyzing a sentence included in a text database and generating a target sentence dependency structure tree and a function of extracting a feature subtree from the target sentence dependency structure tree,
In the computer,
A function of reading expressions stored in a synonym expression dictionary that defines different expressions as synonymous expression groups with synonymous contents from a storage device and generating a synonymous expression dependency structure tree that is a dependency structure tree of the expression;
A function of collating the target sentence dependency structure tree with the synonymous expression dependency structure tree and identifying whether or not a matching subtree that is a subtree matching the synonymous expression dependency structure tree is included in the target sentence dependency structure tree When,
A function of generating a synonymous expression node having an identifier that indicates the synonymous expression group to which the expression corresponding to the matching subtree belongs and is distinguished from a normal node label;
A function of replacing all nodes included in the matching subtree with the synonymous nodes;
A text mining program that executes a function of extracting a feature subtree from the target sentence dependency structure tree after the replacement. In a text mining program that causes a computer to execute a function of analyzing a sentence included in a text database and generating a target sentence dependency structure tree and a function of extracting a feature partial structure from the target sentence dependency structure tree,
In the computer,
A function of reading expressions stored in a synonym expression dictionary that defines different expressions as synonymous expression groups with synonymous contents from a storage device and generating a synonymous expression dependency structure tree that is a dependency structure tree of the expression;
A function of collating the target sentence dependency structure tree with the synonymous expression dependency structure tree and identifying whether or not a matching subtree that is a subtree matching the synonymous expression dependency structure tree is included in the target sentence dependency structure tree When,
A function of generating a synonymous expression node having an identifier that indicates the synonymous expression group to which the expression corresponding to the matching subtree belongs and is distinguished from a normal node label;
The synonym expression node is added to the target sentence dependency structure tree, and the synonym expression is obtained from a node that is not included in the matching subtree and has a dependency branch toward the node included in the matching subtree. A dependency branch toward the node is added, from the synonymous expression node toward a node that is not included in the matching subtree and has a dependency branch directed from the node included in the matching subtree. A function to generate a dependency structure bundle by adding a dependency branch,
A text mining program that executes a function of extracting a feature subtree from the dependency structure bundle. After a synonymous expression node is added to the target sentence dependency structure tree, the computer is caused to execute a function of connecting the synonymous expression node and each node included in the matching subtree with an exclusive relation branch,
15. The text mining according to claim 14, wherein, when extracting the feature subtree, the feature subtree is extracted from a subtree that does not include nodes connected by the exclusive branch of the dependency structure bundle. program.

Images