JP6042264B2 - Grammar rule learning device, method, and program - Google Patents

Description

  The present invention relates to a grammar rule learning apparatus, method, and program, and more particularly, a grammar rule learning apparatus, method, and method for learning a grammar rule based on a sampling method from a corpus of a syntax tree to which syntactic information is assigned. Regarding the program.

  Conventionally, in the natural language processing field, grammatical rules are learned from a syntactic tree corpus to which syntactic information is given based on a probabilistic grammar model. Here, FIG. 5 shows an example of a syntax tree. In the example of FIG. 5, words are assigned to the end nodes of the tree structure, and symbols representing syntactic information are assigned to the other nodes. In FIG. 5, “NP” is a noun phrase, “DT” is a preposition, “N” is a noun, and “JJ” is an adjective symbol. Examples of probabilistic grammar models include models based on context-free grammars, tree replacement grammars, and tree-joint grammars. FIG. 6 shows an example of grammar rules obtained from the syntax tree shown in FIG. 5 according to the tree replacement grammar. Each grammar rule in the tree replacement grammar has a partial structure (subtree) of the parse tree.

  As a method for learning grammatical rules from a syntax tree corpus, a method using a Gibbs sampling method has been proposed (for example, see Non-Patent Document 1). In the Gibbs sampling method, a syntax tree is circulated one by one, and the current subtree assignment constituting the target syntax tree is updated probabilistically. In the Gibbs sampling method, the likelihood of the probabilistic grammar model is calculated every time the subtree assignment is updated, and the subtree assignment with the highest likelihood is output as the final grammar rule.

Trevor Cohn, Sharon Goldwater, and Phil Blunsom, "Inducing compact but accurate tree-substitution grammars," In Proceedings of HLT-NAACL, pages 548-556, Boulder, Colorado, June. Association for Computational Linguistics, 2009.

  However, in the method of learning grammar rules by updating subtrees one by one, such as the Gibbs sampling method described in Non-Patent Document 1, when the data amount of the syntax tree corpus is large, the local optimal solution remains. Therefore, there is a problem that it is impossible to learn a grammar rule having a high likelihood in the probabilistic grammar model.

  The present invention has been made to solve the above problems, and provides a grammar rule learning device, method, and program capable of learning a grammar rule having a high likelihood in a probabilistic grammar model. Objective.

In order to achieve the above object, the grammar rule learning device of the present invention includes the plurality of subtrees and the predetermined in the probabilistic grammar model representing a plurality of syntax trees with a plurality of types of subtrees and predetermined parameters. An initial setting unit that sets initial values of the parameters, and a method of probabilistically updating assignment of subtrees constituting the syntax tree, the plurality of subtrees set by the initial setting unit and the predetermined parameters, Alternatively, a part configured by dividing a plurality of subtrees updated last time and an update unit that updates a predetermined parameter, and a first type subtree included in the plurality of subtrees updated by the update unit A part selected from each tree candidate according to the joint probability of each subtree candidate and the syntax tree set including the subtree candidate in the probabilistic grammar model represented by the predetermined parameter The candidates with the second type of subtree, all of the first type of subtree included in the plurality of syntax trees, it is replaced before Symbol second type of subtree, the predetermined parameters The replacement update unit to be updated and the update of the plurality of subtrees and the predetermined parameter by the update unit until the predetermined end condition is satisfied, and the replacement of the subtree by the replacement update unit and the update of the predetermined parameter And an end determination unit that outputs each of the plurality of subtrees when the end condition is satisfied as a grammatical rule together with a predetermined parameter when the end condition is satisfied.

According to the grammatical rule learning device of the present invention, the initial setting unit includes a plurality of subtrees and predetermined parameters in a probabilistic grammar model in which a plurality of types of subtrees and predetermined parameters represent a plurality of syntax trees. Set the initial value. Then, the update unit probabilistically updates the allocation of subtrees constituting the syntax tree, and a plurality of subtrees and predetermined parameters set by the initial setting unit, or a plurality of subtrees updated last time and Update predetermined parameters. For example, a conventional method such as a Gibbs sampling method can be used to update the plurality of subtrees and the predetermined parameter by the updating unit.

Then, the replacement update unit generates a probabilistic representation represented by a predetermined parameter from each of the subtree candidates configured by dividing the first type subtree included in the plurality of subtrees updated by the update unit. A subtree candidate selected according to the joint probability of each subtree candidate and the syntax tree set including the subtree candidate in the grammar model is set as the second type of subtree, and all the first included in the plurality of syntax trees. The sub-tree of the type is replaced with the sub-tree of the second type, and the predetermined parameter is updated. Then, until the end determination unit satisfies a predetermined end condition, the updating unit repeatedly updates a plurality of subtrees and predetermined parameters, and the replacement updating unit replaces the subtrees and updates predetermined parameters. Are output together with predetermined parameters when the end condition is satisfied as grammar rules.

  In this way, the probabilistic grammar representing a plurality of syntax trees by replacing the first type of subtrees included in the plurality of subtrees constituting the plurality of syntax trees with the second type of subtree at a time. By updating the parameters of the model, the frequency distribution of the subtree can be significantly changed, so that a grammar rule having a high likelihood in the probabilistic grammar model can be learned.

  Further, the replacement update unit includes each subtree candidate in the probabilistic grammar model represented by the predetermined parameter from each of the subtree candidates configured by dividing the first type of subtree. The subtree candidate selected according to the joint probability with the syntax tree set including the subtree candidate can be set as the second type of subtree. Thereby, since the frequency distribution of the subtree can be changed in various ways, a grammar rule with higher likelihood can be learned.

  In the grammatical rule learning method of the present invention, the initial setting unit includes the plurality of subtrees in the probabilistic grammar model in which a plurality of types of subtrees and a predetermined parameter represent a plurality of syntax trees, and the predetermined A step of setting initial values of parameters, and a step of updating the plurality of subtrees and the predetermined parameters set by the initial setting unit or the plurality of subtrees and the predetermined parameters updated last time And the replacement updating unit replaces the first type of subtree included in the plurality of subtrees updated by the updating unit with a second type of subtree different from the first type. Updating the predetermined parameter, and updating the plurality of subtrees and the predetermined parameter by the updating unit until the end determination unit satisfies a predetermined end condition, and The replacement of the subtree by the replacement updating unit and the updating of the predetermined parameter are repeated, and each of the plurality of subtrees when the end condition is satisfied is a grammatical rule, along with the predetermined parameter when the end condition is satisfied And a step of outputting.

  The grammatical rule learning program of the present invention is a program for causing a computer to function as each part constituting the grammatical rule learning device.

  As described above, according to the grammatical rule learning device, method, and program of the present invention, the first type of subtrees included in the plurality of subtrees constituting the plurality of syntax trees can be converted to the second type at a time. By substituting different types of subtrees and updating the parameters of the probabilistic grammar model that represents multiple syntax trees, the frequency distribution of the subtree can be changed significantly, so the grammar rules with high likelihood in the probabilistic grammar model Can be learned.

It is a block diagram which shows the functional structure of the grammar rule learning apparatus which concerns on this Embodiment. It is the schematic for demonstrating substitution of a subtree. It is a flowchart which shows the content of the grammar rule learning process routine in this Embodiment. It is a figure which shows the result of having compared the likelihood of the probabilistic grammar model of the method in this Embodiment, and the conventional method. It is a figure which shows an example of a syntax tree. It is a figure which shows an example of the grammar rule obtained from the syntax tree shown in FIG. 5 according to a tree replacement grammar.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  As shown in FIG. 1, the grammar rule learning apparatus 10 according to the present embodiment receives a syntax tree corpus {t}, which is a set of syntax trees t to which syntactic information is assigned, and inputs a probabilistic grammar model. The optimal grammar rule that maximizes the likelihood, that is, the set of subtrees is output. In this embodiment, a case where a probabilistic grammar model based on a tree replacement grammar (for example, see Non-Patent Document 1) is used as an example is described.

  Here, in the tree replacement grammar, a specific expression of the probabilistic grammar model is given by, for example, the following expression (1).

Where e _t is a set of subtrees e constituting one syntax tree t, {e _t } is a set of subtrees e in the entire syntax tree corpus {t}, and P ({e _t }, {t } | Θ) is a joint probability between a subtree set {e _t } and a syntax tree corpus {t} in a probabilistic grammar model with a parameter θ. I is an index indicating a syntax tree in the syntax tree corpus, j is an index indicating a subtree constituting the syntax tree t, I is a total number of syntax trees included in the syntax tree corpus, and J is a syntax tree. This is the total number of subtrees e constituting t. Further, X represents a symbol of the root node of the subtree e, n _e is the subtree e is the number representing how appeared many times in the syntax tree corpus. Θ _X is a parameter of the tree replacement grammar and is defined for each symbol X of the root node. Therefore, the parameters of the probabilistic grammar model are a set of θ _X , that is, θ = {θ _X }. n _{. , X} is the number of times representing how many times a subtree whose root node is X appears in the syntax tree corpus. P ₀ (e | X) is a probability called a base probability of the subtree e, and is calculated based on, for example, a uniform distribution.

The grammar rule learning device 10 receives a syntax tree corpus {t} as an input, and a grammar that maximizes the likelihood P ({e _t }, {t} | θ) of a probabilistic grammar model as shown in equation (1). The rule {^ e _t } and the parameter ^ θ are output.

  The grammatical rule learning device 10 is constituted by a computer including a CPU, a RAM, and a ROM that stores a grammatical rule learning program for executing a grammatical rule learning processing routine described later. That is, the computer functions as the grammar rule learning device 10 when the CPU executes the grammar rule learning program stored in the ROM. Further, the computer may be configured to include an HDD as storage means.

  As shown in FIG. 1, this computer can be functionally represented by a configuration including an initial setting unit 11, a Gibbs sampling unit 12, a replacement update unit 13, and an end determination unit 14. The initial setting unit 11 and the Gibbs sampling unit 12 are examples of the setting unit of the present invention.

The initial setting unit 11 receives the syntax tree corpus {t} as an input, and sets the initial subtree {e _t } ⁽⁰⁾ and the initial parameter θ ⁽⁰⁾ . The initial subtree {e _t } ⁽⁰⁾ is, for example, a variable in which 0 or 1 is randomly assigned to each node of the syntax tree excluding the terminal node and the root node, and the node to which the variable of 1 is assigned It can be set as a root node of a tree, and a node assigned a variable of 0 can be set as an internal node of the subtree (a node other than the root node of the subtree). The initial parameter θ ⁽⁰⁾ can also be set at random. The initial setting unit 11 outputs the set initial subtree {e _t } ⁽⁰⁾ and the initial parameter θ ⁽⁰⁾ to the Gibbs sampling unit 12.

The Gibbs sampling unit 12 uses the initial subtree {e _t } ⁽⁰⁾ and the initial parameter θ ⁽⁰⁾ , or the partial tree {e _t } ^{(u + 1)} and the parameter θ ^{(u + 1)} output from the end determination unit 14 described later. Is accepted as input. U is a variable representing the number of times the update process is repeated. For example, as disclosed in Non-Patent Document 1, the Gibbs sampling unit 12 circulates each node of the syntax tree in a random order and sets each node as a processing target node. The Gibbs sampling unit 12 probabilistically determines whether the target node is a root node or an internal node of the subtree, and updates the current setting for the target node.

In addition, when the processing for updating the setting with all the nodes of the syntax tree as target nodes is completed, the Gibbs sampling unit 12 updates the parameter θ based on the equation (1). As a method for updating the parameter θ, for example, a method based on the sampling method disclosed in Non-Patent Document 1 can be used. In this method, assuming that the probability distribution P (θ) of the parameter θ follows the gamma distribution Gamma (1.0, 1.0), P ({e _t −}, {t}, θ) = P ({ e _t −}, {t} | θ) × P (θ) is calculated. Then, after fixing all {e _t- }, the updated θ ′ shown in the following equation (2) is searched by the Markov chain Monte Carlo method, and the parameter θ is updated to θ ′.

The Gibbs sampling unit 12 outputs the updated subtree {e _t } ^(u) and the parameter θ ^(u) .

ただし、ｋは｛ｔ｝に含まれる構文木を指すインデックスであり、Ｋは｛ｔ｝に含まれる構文木の総数である。また、Ｐ（ｅ’，ｔ_ｋ｜θ）は、（１）式下段に示す、パラメータをθとする確率的文法モデルにおける部分木ｅ’と構文木ｔとの同時確率である。
このとき、部分木ｅ’の候補として、対象部分木ｅを分割して構成される全ての部分木の組み合わせを考慮する。図２に示すような部分木ｅが対象部分木の場合、対象部分木ｅを分割して構成できる全ての部分木の組み合わせ（図２中のｅ’_（１）〜ｅ’_（４））を部分木ｅ’の候補とする。ただし、部分木ｅと部分木ｅ’とが同じ部分木になる場合もある。図２の例では、部分木ｅ’_（４）が対象部分木ｅと同じ部分木である。置換更新部１３は、各部分木の候補ｅ’の確率Ｐ（ｅ’，｛ｔ｝｜θ）に応じて、１つの候補を選択し、構文木コーパス｛ｔ｝に含まれる全ての部分木ｅを、選択した部分木ｅ’へ置換する。 The replacement update unit 13 receives the subtree {e _t } ^(u) and the parameter θ ^(u) updated by the Gibbs sampling unit 12 as inputs. The replacement update unit 13 does not circulate each node of the syntax tree, but makes each subtree a target subtree to be processed while circulating the subtrees {e _t } ^(u) one by one in random order, The target subtree e is replaced with another subtree e ′ according to the probability of P (e ′, {t} | θ). Here, {t} is a set of syntax trees t including another subtree e ′. The probability of P (e ′, {t} | θ) is calculated as in the following equation (3).

Here, k is an index indicating the syntax tree included in {t}, and K is the total number of syntax trees included in {t}. P (e ′, t _k | θ) is a joint probability of the subtree e ′ and the syntax tree t in the probabilistic grammar model having the parameter θ shown in the lower part of the equation (1).
At this time, combinations of all subtrees configured by dividing the target subtree e are considered as candidates for the subtree e ′. When the subtree e as shown in FIG. 2 is the target subtree, all subtree combinations (e ′ _{(1) to} e ′ _{(4) in} FIG. 2) that can be configured by dividing the target subtree e are represented. Let it be a candidate for the subtree e ′. However, the partial tree e and the partial tree e ′ may be the same partial tree. In the example of FIG. 2, the subtree e ′ ₍₄₎ is the same subtree as the target subtree e. The replacement updating unit 13 selects one candidate according to the probability P (e ′, {t} | θ) of each subtree candidate e ′, and all subtrees included in the syntax tree corpus {t}. Replace e with the selected subtree e ′.

When the replacement of the subtree is completed, the replacement update unit 13 updates the parameter θ in the same manner as the Gibbs sampling unit 12. Since the replacement update unit 13 updates a plurality of subtrees of the same type included in the syntax tree corpus {t} at a time, there is a possibility that the frequency distribution of the subtree can be significantly changed as compared with the update by the Gibbs sampling unit 12. There is. Therefore, it is possible to find an assignment of a subtree with a high likelihood rather than simply repeating Gibbs sampling many times, and the problem that the probabilistic grammar model remains in the local optimal solution can be solved. The replacement updating unit 13 outputs the updated subtree {e _t } ^(u) and the parameter θ ^(u) .

The end determination unit 14 receives the subtree {e _t } ^(u) and the parameter θ ^(u) updated by the replacement update unit 13 as inputs. The end determination unit 14 determines whether the end condition is satisfied, and repeats the processing of the Gibbs sampling unit 12 and the replacement update unit 13 until the end condition is satisfied. In the end determination, for example, it can be determined to end when the current number of repetitions u reaches a predetermined number of times (for example, 3000 times), and it can be determined to be incomplete if it is less than that. When the termination determination unit 14 determines that the termination condition is satisfied, the grammar rule that maximizes the likelihood of the probabilistic grammar model is obtained for each of the current subtree {e _t } ^(u) and the parameter θ ^(u). Output as {^ e _t } and parameter ^ θ.

Further, when the termination determination unit 14 determines that the termination condition is not satisfied, the repetition count u is increased by 1, and the current subtree {e _t } ^(u) and the parameter θ ^(u) are converted to {e _t } ^{(u + 1)} and θ ^{(u + 1)} are output to the Gibbs sampling unit 12.

Note that determination of the end condition by the end determination unit 14 is not limited to determining whether or not the number of repetitions is the specified number. For example, the difference between the likelihood of the probabilistic grammar model calculated this time based on {e _t } ^(u) and the parameter θ ^(u) updated by the replacement update unit 13 and the likelihood calculated last time is predetermined. When the value is less than or equal to the value, it may be determined that the end condition is satisfied.

  Next, the operation of the grammar rule learning apparatus 10 according to this embodiment will be described. When the syntax tree corpus {t} is input to the grammar rule learning device 10, the grammar rule learning device 10 executes the grammar rule learning processing routine shown in FIG. The syntax tree corpus {t} is input to the grammar rule learning device 10 by using the syntax tree corpus {t} stored in the external device or the external storage medium or the like in the grammar rule learning device 10 via a network or the like. This is done by reading into the storage device. Alternatively, the syntax tree corpus {t} stored in advance in the storage device in the grammar rule learning device 10 may be read to start the grammar rule learning processing routine shown in FIG.

In step 100 of the grammar rule learning processing routine shown in FIG. 3, the initial setting unit 11 accepts the syntax tree corpus {t} as input and sets the initial subtree {e _t } ⁽⁰⁾ and the initial parameter θ ⁽⁰⁾ . To do. Next, in step 102, the end determination unit 14 sets a variable u indicating the number of repetitions to 0.

  Next, in step 104, the Gibbs sampling unit 12 makes each node a target node while circulating each node of the syntax tree in a random order as disclosed in Non-Patent Document 1, for example, and the target node is a partial node. Whether the root node or internal node of the tree is determined probabilistically, the current setting for the target node is updated, and the parameter θ is updated based on equation (1).

Next, in Step 106, the replacement update unit 13 divides the target subtree e by setting each subtree as a target subtree while traversing the subtree {e _t } ^(u) one by one in a random order. All the combinations of subtrees to be configured are candidates for the subtree e ′. Then, the replacement update unit 13 selects one candidate according to the probability P (e ′, {t} | θ) of each subtree candidate e ′, and sets all the candidates included in the syntax tree corpus {t}. The subtree e is replaced with the selected subtree e ′. Further, the replacement update unit 13 updates the parameter θ in the same manner as the Gibbs sampling unit 12.

Next, in step 108, the end determination unit 14 determines whether or not the end condition is satisfied by determining whether or not the current number of repetitions u has reached a predetermined number of times (for example, 3000 times). To do. If the end determination unit 14 determines that the end condition is not satisfied, the end determination unit 14 proceeds to step 110 to increase the number of repetitions u by one, and the current subtree {e _t } ^(u) and the parameter θ ^{(u ) )} Are output to the Gibbs sampling unit 12 as {e _t } ^{(u + 1)} and θ ^{(u + 1)} , and the process returns to step 104. In step 104 in the iterative process, the Gibbs sampling unit 12 repeats the update of the setting of the target node and the update of the parameter θ using the updated {e _t } ^{(u + 1)} and θ ^{(u + 1)} . Also, in step 106 in the iterative process, the replacement update unit 13 repeats subtree replacement and parameter θ update using {e _t } ^{(u + 1)} and θ ^{(u + 1)} updated by the Gibbs sampling unit 12. .

On the other hand, if the end determination unit 14 determines that the end condition is satisfied, the end determination unit 14 proceeds to step 112, and converts the current subtree {e _t } ^(u) and the parameter θ ^(u) to the probabilistic grammar model. The grammar rule {^ e _t } that maximizes the likelihood and the parameter ^ θ are output, and the grammar rule learning processing routine is terminated.

  As described above, according to the grammar rule learning device according to the present embodiment, a plurality of the same type of subtrees included in the syntax tree corpus are replaced with different subtrees at a time, and Update parameters. As a result, the frequency distribution of subtrees can be changed significantly compared to the case where the subtree is updated locally as in the conventional method. The possibility of discovery increases, and the problem that the probabilistic grammar model remains in the local optimal solution can be solved. In addition, when replacing a subtree with another subtree, the subtree frequency distribution can be changed in various ways in order to consider all subtree combinations configured by dividing the original subtree as candidates. Can do.

  Here, in order to verify the effect of the grammatical rule learning apparatus according to the present embodiment, when there is a replacement update unit (this embodiment) and when there is no replacement update unit (conventional technique based on a simple Gibbs sampling method) FIG. 4 shows the experimental results comparing the likelihood of. In this experiment, the syntax tree corpus is composed of a syntax tree of about 40,000 sentences, and the number of repetitions u is 1000. As shown in FIG. 4, it was confirmed that the grammar rule learning device according to the present embodiment can find a grammar rule having a higher likelihood than the conventional method due to the effect of the replacement update unit.

  The present invention is not limited to the above-described embodiment, and various modifications and applications are possible without departing from the gist of the present invention.

  The above-described grammatical rule learning apparatus has a computer system therein, but the “computer system” includes a homepage providing environment (or display environment) if a WWW system is used. .

  Further, in the present specification, the embodiment has been described in which the program is installed in advance. However, the program can be provided by being stored in a computer-readable recording medium.

DESCRIPTION OF SYMBOLS 10 Grammar rule learning apparatus 11 Initial setting part 12 Gibbs sampling part 13 Replacement update part 14 Completion determination part

Claims (3)

An initial setting unit for setting an initial value of the plurality of subtrees and the predetermined parameter in a probabilistic grammar model representing a plurality of syntax trees with a plurality of types of subtrees and predetermined parameters;
The plurality of subtrees and the predetermined parameters set by the initial setting unit, or the plurality of subtrees and the predetermined parameters updated last time by the method of probabilistically updating the subtrees constituting the syntax tree An update unit for updating
In each of the probabilistic grammar models represented by the predetermined parameter, from each of the subtree candidates configured by dividing the first type of subtree included in the plurality of subtrees updated by the updating unit . A subtree candidate selected according to the joint probability between each subtree candidate and the syntax tree set including the subtree candidate is set as a second type of subtree, and all the first trees included in the plurality of syntax trees are included. and the type of subtree, replaced before Symbol second type of subtree, substituted updating unit that updates the predetermined parameter,
Until the predetermined end condition is satisfied, the updating unit repeatedly updates the plurality of subtrees and the predetermined parameter, and the replacement updating unit replaces the subtree and the predetermined parameter. An end determination unit that outputs each of the plurality of subtrees when satisfied as a grammar rule together with a predetermined parameter when the end condition is satisfied,
Grammar rule learning device including An initial setting unit setting initial values of the plurality of subtrees and the predetermined parameter in a probabilistic grammar model representing a plurality of syntax trees with a plurality of types of subtrees and predetermined parameters;
The plurality of subtrees set by the initial setting unit and the predetermined parameter, or the plurality of subtrees updated last time by a method in which the update unit probabilistically updates the allocation of subtrees constituting the syntax tree. And updating the predetermined parameters;
The replacement update unit is represented by the predetermined parameter from each of the subtree candidates configured by dividing the first type of subtree included in the plurality of subtrees updated by the update unit. A subtree candidate selected according to the joint probability of each subtree candidate and the syntax tree set including the subtree candidate in the probabilistic grammar model is set as a second type of subtree, and is included in the plurality of syntax trees. all the first type of subtree, replaced before Symbol second type of subtree, and updating the predetermined parameter,
Until the end determination unit satisfies a predetermined end condition, the updating unit repeatedly updates the plurality of subtrees and the predetermined parameter, and the replacement updating unit replaces the subtree and updates the predetermined parameter. Outputting each of the plurality of subtrees when the end condition is satisfied as a grammar rule together with a predetermined parameter when the end condition is satisfied;
Grammar rule learning method including A grammatical rule learning program for causing a computer to function as each part constituting the grammatical rule learning device of claim 1 .

Images