JP2019091172A - Phrase structure learning device, phrase structure analysis device and method, and program - Google Patents

Abstract

To provide a phrase structure learning device, a phrase structure analysis device and method, and a program which generate learning data for such learning that an attention mechanism accurately outputs correspondence.SOLUTION: In order to allow for generation of learning data for such learning that an attention mechanism which is included in a phrase structure analyzer to output weights of each word in an input sentence for phrase structure labels of respective nodes of a phrase structure tree representing the input sentence accurately outputs correspondences, a phrase structure learning device 200 generates learning data which is for training the attention mechanism and comprises correspondences between words and the phrase structure labels, on the basis of the input sentence and a phrase structure label string comprising the phrase structure labels.SELECTED DRAWING: Figure 15

Description

  The present invention relates to a phrase structure learning device, method, and program, and more particularly to a phrase structure learning device, phrase structure analysis device, method, and program for analyzing the phrase structure of a sentence.

  Phrase structure analysis is a technique for analyzing and outputting a phrase structure tree of an input sentence by a computer. Fig. 1 shows an example of a phrase structure tree. A phrase structure tree is composed of phrases and the hierarchical structure of the phrases.

  These phrases consist of a phrase structure label and a set of words that make up the phrase. A word set constituting a phrase is constituted by a set of words included in lower leaf nodes of the phrase structure label in the phrase structure tree.

  Non-Patent Document 1 is an example of a phrase-based phrase structure analysis method using a neural network. Non-Patent Document 1 does not assume an explicit tree structure, but analyzes and outputs a phrase structure tree as a sequence of phrase structure labels. In the prior art including Non-Patent Document 1, the phrase structure tree shown in FIG. 1 is expressed as a series as shown in FIG. In the phrase structure tree expressed as a series, the words and part-of-speech of all leaf nodes (of the phrase structure labels, the label closest to the word; in FIG. 1, WP, VBZ, JJ are shown) are shown in FIG. Are all replaced with the same label. Also, all labels except XX are phrase structure labels. The phrase structure tree expressed as such a sequence is hereinafter referred to as a "normalized phrase structure tree". FIG. 3 shows a block diagram of a phrase structure analysis method based on the prior art. A sequence-based phrase structure analysis method can output a phrase structure tree in linear time without requiring manual rules or transition rules.

Vinyals, O., Kaiser, L., Koo, T., Petrov, S., Sutskever, I., and Hinton, G. (2015). "Grammar as a foreign language.". In Advancesin Neural Information Processing Systems ( pp. 2773-2781).

  In Non-Patent Document 1, the relation between the input word string and the output phrase structure label is calculated by the attention mechanism included in the phrase structure analyzer that outputs the weight of each word of the input sentence to the phrase structure label. Captured by distribution. FIG. 4 shows an example of the distribution of each weight output by the caution mechanism. Each cell represents the correspondence between the input word and the output phrase structure label, and the black cell represents that the attention mechanism assigned a high probability to the correspondence. The caution mechanism determines during analysis which string in the input string is important. The analyzer outputs the analysis result using the result. The output distribution of the attention mechanism is trained unsupervised without being given explicit phrase structure labels. Therefore, the caution mechanism does not necessarily learn the correspondence of the input string and the output string correctly. As a result, an incorrect correspondence relation outputted by the caution mechanism may cause an error in the outputted phrase structure tree.

  The present invention has been made to solve the above problems, and provides a phrase structure learning device, method, and program capable of generating learning data for learning that the attention mechanism accurately outputs the correspondence. Intended to be provided.

  Another object of the present invention is to provide a phrase structure analysis apparatus, method, and program capable of performing phrase structure analysis with high accuracy, which has been made to solve the above-mentioned problems. .

  In order to achieve the above object, a phrase structure learning device according to the present invention is a phrase structure learning device for learning a phrase structure analyzer that outputs a phrase structure label string for an input sentence, the input sentence and the input sentence To output the weight of each word of the input sentence to the phrase structure label included in the phrase structure analyzer based on the phrase structure label string including the phrase structure label of each node of the phrase structure tree representing A learning data generation unit for generating learning data composed of the correspondence between the word and the phrase structure label for learning a mechanism.

  The phrase structure analysis device according to the present invention is a previously learned phrase structure analyzer that outputs a phrase structure label string for an input sentence, and outputs the weight of each word of the input sentence to the phrase structure label. A phrase structure analysis device including: a phrase structure analysis unit that takes the input sentence as an input and outputs the phrase structure label string for the input sentence using a phrase structure analyzer including a caution mechanism, the caution mechanism comprising: And an association of the word and the phrase structure label generated based on the learning input sentence and the phrase structure label string including the phrase structure label of each node of the phrase structure tree representing the learning input sentence. It is characterized in that it is learned in advance based on learning data.

  A program according to the present invention is a program for causing a computer to function as each part of a phrase structure learning device or a phrase structure analysis device.

  According to the phrase structure learning device, method, and program of the present invention, a phrase structure analyzer is provided based on an input sentence and a phrase structure label string consisting of phrase structure labels of each node of a phrase structure tree representing the input sentence. The attention mechanism responds with high precision by generating learning data consisting of the correspondence between words and phrase structure labels to learn the attention mechanism that outputs the weight of each word of the input sentence to the included phrase structure labels. The effect is that it is possible to generate learning data for learning so as to output the attachment.

  Further, according to the phrase structure analysis device, method, and program of the present invention, phrase structure analysis is performed using a phrase structure analyzer including an attention mechanism, and the attention mechanism included in the phrase structure analyzer is a learning input The learning is made in advance based on learning data consisting of the correspondence between words and phrase structure labels, generated based on the sentence and the phrase structure label string consisting of phrase structure labels of each node of the phrase structure tree representing the input sentence for learning According to the above, it is possible to obtain the effect that the phrase structure analysis can be performed with high accuracy.

It is a figure which shows an example of the phrase structure tree output by phrase structure analysis. It is a figure which shows an example of the series of the phrase structure label of phrase structure tree. It is a figure which shows an example of the block diagram of the conventional phrase structure analysis method. It is a figure which shows an example of distribution of each weight which the caution mechanism 3 outputs. It is a figure which shows an example of the neural network which comprises the phrase structure analyzer 240 in a learning phase. It is a figure which shows an example of the neural network which comprises the phrase structure analyzer in the execution phase. It is a figure which shows the content of the input-output used by embodiment of this invention. It is a block diagram showing composition of phrase structure analysis device 100 concerning an embodiment of the invention. FIG. 2 is a diagram showing a configuration of a neural network that constitutes a phrase structure analyzer 40. It is a figure which shows an example of the detail of the encoding part 1 which comprises a neural network. It is a figure which shows an example of the detail of the decoding part 2 which comprises a neural network. It is a figure which shows an example of the detail of the attention mechanism 3 which comprises a neural network. It is a flowchart which shows the phrase structure-analysis processing routine in the phrase structure-analysis apparatus 100 based on embodiment of this invention. It is a figure which shows an example of the detail of the processing routine in the neural network of the phrase structure analyzer. It is a block diagram showing composition of phrase structure learning device 200 concerning an embodiment of the invention. It is a figure showing an outline of learning of an embodiment of the present invention. FIG. 6 is a diagram showing an example of a learning input sentence X and an answer series L. It is a figure which shows an example of matching with the word which comprises the input sentence for learning, and a phrase structure label. It is a figure which shows an example of matching with the word which comprises the input sentence for learning, and a phrase structure label. It is a figure which shows an example of the learning data obtained as a process result of the learning data generation part 230. FIG. It is a flowchart which shows the phrase structure learning process routine in the phrase structure learning apparatus 200 which concerns on embodiment of this invention. It is a flowchart which shows the production | generation process routine of learning data.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

<Overview of Embodiment of the Present Invention>

  First, an outline of the embodiment of the present invention will be described.

  In the embodiment of the present invention, in a phrase structure analyzer based on a sequence using a neural network as in Non-Patent Document 1, the relationship between words constituting an input sentence and phrase structure labels to be output is clearly indicated for the attention mechanism. Give and learn.

  The learning data for learning the attention mechanism is created using predetermined rules based on the learning data for phrase structure analysis. Specifically, for each phrase structure label that constitutes a phrase structure tree of learning data, the phrase structure label is associated with a set of words included in leaf nodes lower than the node of the phrase structure label according to a predetermined rule. Create with It is defined that correspondence between a phrase structure label to be output and a word included in a phrase whose apex is the phrase structure label is defined as correct correspondence, and is used as learning data for learning the attention mechanism.

  By learning the attention mechanism using such learning data, it is possible to prevent the attention mechanism from outputting a distribution including an incorrect correspondence and to improve the accuracy of the phrase structure analysis. In addition, the attention mechanism is internal processing of the phrase structure analyzer, and although it is expensive to separately create learning data for the attention mechanism, the learning data of the phrase structure analyzer according to the present method is for the attention mechanism. Learning data can be created.

  The following description of the embodiment of the present invention will be divided into a learning phase and an execution phase. In the learning phase, parameters of the neural network constituting the phrase structure analyzer 240 shown in FIG. 5 are determined based on the learning data. In the execution phase, the input is processed based on the neural network constituting the phrase structure analyzer 40 shown in FIG. 6 defined in the learning phase, and the output is determined depending on the learned parameters.

  Hereinafter, FIG. 7 will be used as an example of input and output used in the present embodiment. The position of each word in the input sentence X is i, i = {1,..., N}, and n = 6 in this example. Let t be the position of each phrase structure label in the output sequence Y, and t = {1,..., M}, and m = 15 in this example.

<Configuration of Phrase Structure Analysis Device According to Embodiment of the Present Invention>

  Next, the configuration of the phrase structure analysis device according to the embodiment of the present invention will be described. In the phrase structure analysis device, the execution phase is processed.

The phrase structure analyzer 40 according to the embodiment of the present invention receives a sentence x to be processed as an input, and outputs a phrase structure label y _t . Next, the output y _t is input, and the sequential processing of outputting y _{t + 1} is repeated, and the processing ends when y _{t to} be output becomes a sentence end symbol </ s>.

  As shown in FIG. 8, the phrase structure analysis device 100 according to the embodiment of the present invention includes a CPU, a RAM, and a ROM storing a program for executing a phrase structure analysis processing routine to be described later and various data. Can be configured with a computer including This phrase structure analysis device 100 functionally includes an input unit 10 and an operation unit 20 as shown in FIG.

  The input unit 10 receives an input sentence. The input sentence is obtained by dividing the sentence and adding a sentence head and a sentence end symbol.

  The operation unit 20 includes a phrase structure analysis unit 30 and a phrase structure analysis unit 40.

  The phrase structure analyzer 40 is a previously learned phrase structure analyzer that sequentially outputs a phrase structure label string for an input sentence consisting of words from the top, and is constituted by a neural network. The phrase structure analyzer 40 is a phrase structure analyzer including a caution mechanism 3 that outputs the weight of each word of the input sentence to the phrase structure label. The learning of the phrase structure analyzer 40 will be described in the phrase structure learning device described later.

  Here, the configuration of the neural network of the phrase structure analyzer 40 will be described. As shown in FIG. 9, the neural network constituting the phrase structure analyzer 40 is configured to include the encoding unit 1, the decoding unit 2, the attention mechanism 3, and the output unit 4.

  The encoding unit 1 converts an input sentence into a hidden state.

The decoding unit 2 converts the label yt _-1 output last time into the hidden state.

  The caution mechanism 3 combines the hidden state converted by the encoding unit 1 and the decoding unit 2 to perform weighting, and converts it into the weight of each word of the input sentence.

  The output unit 4 weights the hidden state of the encoding unit 1 based on the weight of each word of the input sentence output by the caution mechanism 3, combines with the hidden state of the decoding unit 2, and determines the label to be output. Do.

  For the processing of one sentence, the processing in the encoding unit 1 may be performed only once, but the other parts will repeat the processing m-1 times.

  Hereinafter, each part of the neural network which comprises the phrase structure analyzer 40 is demonstrated in detail.

FIG. 10 shows the details of the encoding unit 1. Encoding section 1 receives one sentence as an input, and generates a series of words x = {x ₁ ,..., X _i ,..., X _n } contained in one sentence, for each word x _i Convert to a vector h _i

  Specifically, the l-th input word is hidden by the forward recursive neural network which first scans the input word sequence forward from i = 1 to n.

Convert to Similarly, the l-th input word is hidden by the backward recursive neural network that scans the input word sequence backward from i = n to 1

Convert to Finally,

When

Let be connected to be h _i . This process, _{h 1 h i} is the result forward and backward word _conversion, ..., _{h i-1 and, h i + 1, ...,} becomes dependent on _{h n.}

Vector each word x _i

Or

When converting to, a codebook consisting of a set of (word, word vector) created in advance is used. The word vector indicates the feature of the word as a set as a coordinate in the space having a predetermined dimension, and is also called a word distributed expression. In this embodiment, with all input words (including <s> and </ s>) as W, a vector satisfying the following conditions is used, but it is prepared in advance using the method described in Non-Patent Document 2, etc. You may use what.

Condition 1: 1 consists of all W dimensions corresponding to one word.
Condition 2: A one-hot vector in which an element of a dimension corresponding to a set of words is 1 and an element of the other dimensions is 0.

[Non-Patent Document 2]: Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado, and Jeffrey Dean. Distributed Representations of Words and Phrases and their Compositionality. In Proceedings of NIPS, 2013.

  In the present embodiment, the value of each element of the word vector is weighted by the parameters of the neural network. The parameters are updated by learning of the phrase structure learning device described later.

The details of the decoding unit 2 are shown in FIG. The decoding unit 2 receives as input the phrase structure labels y _t-1 and enc ^t-1 output by the output unit 4 last time, and inputs the input information by the forward recursive neural network.

And a hidden state vector of real-valued predetermined dimensions

Convert to and output. Since the phrase structure labels are sequentially input, the result of the conversion is a set of hidden state vectors that are the result of conversion of the previous phrase structure labels.

And the hidden state vectors h ₁ ,..., H _n of the encoding unit 1 depend on enc ^t−1 weighted by the attention mechanism 3. Details of enc ^t will be described later.

When converting the phrase structure label y _t-1 into a vector, a codebook composed of a pair of (phrase structure label, phrase structure vector) created in advance is used. When the total phrase structure label number is V, a vector satisfying the following condition is used as a phrase structure vector.

Condition 1: It consists of all V dimensions in which one dimension corresponds to one phrase structure label.
Condition 2: A one-hot vector in which an element of a dimension corresponding to a pair of phrase structure labels is 1 and an element of the other dimensions is 0.

Decoding unit 2 and to start the process from t = 2, the <s> is the phrase structure label y ₁ of t = 1 as an input at this time, the initial value will be described later in enc ^1, the decoding section 2 initial state

Hidden state of the encoding unit 1

Use

  FIG. 12 shows the details of the caution mechanism 3. Attention mechanism 3 is the conversion result of the phrase structure label of decoding unit 2 to the hidden state

And the conversion results h ₁ ,..., H _n of each word of the encoding unit 1 to the hidden state as inputs, and the weights α _i ^t normalized by the neural network are calculated so that the sum of each correspondence becomes 1 Output as a distribution α ^t . The initial value α ¹ (t = 1) of the caution mechanism 3 is set to a vector of α ₁ ¹ = 1, and the other elements are 0.

In the output unit 4, the distribution outputted by the caution mechanism 3, the conversion results h ₁ ,..., H _n of each word of the encoding unit 1 to the hidden state, and the conversion results of the phrase structure label of the decoding unit 2 to the hidden state

And output the output probability of each phrase structure label. First, the output unit 4 calculates the sum obtained by weighting the conversion result of the encoding unit 1 according to the following equation (1) according to the distribution α ^t output by the caution mechanism 3.

... (1)

(1) A vector obtained by connecting the sum of expressions and the conversion result of the phrase structure label of the decoding unit 2

Are input to the soft max layer, and the output probability of each phrase structure label is determined. The output probability P (y _t | x ₁ ,..., X _n , y ₁ ,..., Y _t−1 ) of the phrase structure label y _t when the number of phrase structure labels is V is a weight matrix W _v and a bias term b The following equation (2) is used to calculate.

... (2)

Calculated by The weighting matrix W _v and the bias term b are parameters of the neural network.

Let y _t with the highest output probability P be the t-th phrase structure label output by the phrase structure analyzer 40.

The phrase structure label y _t is sequentially output by the above processing, and the processing ends when the output y _t becomes a sentence end symbol </ s> (at the time of t = 14 in this embodiment). .

As the initial value of the output unit,

Is <s>. α ¹ uses the initial value of the caution mechanism 3.

  This completes the description of each part of the neural network that constitutes the phrase structure analyzer 40.

  The phrase structure analysis unit 30 uses the phrase structure analysis unit 40 to use the input sentence accepted by the input unit 10 as an input, and outputs a phrase structure label string for the input sentence. Here, the attention mechanism 3 included in the phrase structure analyzer is a phrase structure label string including the input sentence for learning and the phrase structure label of each node of the phrase structure tree representing the input sentence for learning, and is normalized. It is assumed that learning is made in advance on the basis of learning data which is generated on the basis of the phrase structure label string and which corresponds to the word and the phrase structure label. Here, the normalized phrase structure label sequence is a sequence of phrase structure labels of a phrase structure tree after replacement of a word and a part of speech of a leaf node, as described with reference to FIG.

<Operation of Phrase Structure Analysis Device According to Embodiment of the Present Invention>

  Next, the operation of the phrase structure analysis device 100 according to the embodiment of the present invention will be described. The phrase structure analysis device 100 executes a phrase structure analysis processing routine shown in FIG.

  First, in step S100, the input unit 10 receives an input sentence.

  In step S102, using the phrase structure analyzer 40, the input sentence accepted by the input unit 10 is input, and a phrase structure label string for the input sentence is output.

  Next, details of the processing routine in the neural network of the phrase structure analyzer 40 will be described with reference to FIG.

  In step S1000, t = 1.

  In step S1002, the encoding unit 1 receives an input sentence, and converts the input sentence into a hidden state.

  In step S1004, an initial value in each part of the neural network is set.

  In step S1006, t = 2.

In step S1008, the decoding unit 2 converts the initial label or the label y _t-1 output last time into the hidden state.

  In step S1010, in the caution mechanism 3, weighting is performed by combining the hidden states converted by the encoding unit 1 and the decoding unit 2, and conversion is performed to the weight of each word of the input sentence.

  In step S1012, the output unit 4 weights the hidden state of the encoding unit 1 based on the weight of each word of the input sentence output by the caution mechanism 3, combines it with the hidden state of the decoding unit 2, and outputs the result. Determine the power label.

  In step S1014, it is determined whether the output of step S1012 is the sentence end symbol </ s>, and if the sentence end symbol </ s>, the processing is ended, and if it is not the sentence end symbol </ s>, t = t in step S1016 Count up to t + 1 and repeat the process. Note that counting up t is denoted as t = t + 1.

  As described above, according to the phrase structure analysis device 100 according to the embodiment of the present invention, the phrase structure analysis is performed using the phrase structure analyzer 40 including the caution mechanism 3 and included in the phrase structure analyzer 40. The attention mechanism 3 is a phrase structure label string consisting of a learning input sentence and a phrase structure label of each node of a phrase structure tree representing the learning input sentence, and is based on the normalized phrase structure label string Phrase structure analysis can be performed with high accuracy by using learning data generated in advance on the basis of learning data formed by associating words with phrase structure labels.

<Structure of Phrase Structure Learning Device According to the Embodiment of the Present Invention>

  Next, the configuration of the phrase structure learning device according to the embodiment of the present invention will be described. In the phrase structure learning device, the learning phase is processed. Further, in the present embodiment, the encoding unit 1, the decoding unit 2, the attention mechanism 3, and the output unit 4 of the neural network perform learning simultaneously, but learning may be performed separately.

  As shown in FIG. 15, the phrase structure learning device 200 according to the embodiment of the present invention includes a CPU, a RAM, and a ROM storing a program for executing a phrase structure learning processing routine described later and various data. Can be configured with a computer including This phrase structure learning device 200 functionally includes an input unit 210 and an operation unit 220 as shown in FIG.

  FIG. 16 shows a schematic diagram of learning according to the present embodiment. FIG. 16 is divided into learning data generation processing of a learning data generation unit and learning processing of a neural network.

  The input unit 210 receives a learning input sentence and a normalized phrase structure tree. The learning input sentence serving as the input in the learning phase is assumed to be a collection of a plurality of sets of a sentence to be subjected to phrase structure analysis and a phrase structure label sequence which is a correct result as the analysis result. In the present embodiment, the description will be made using, as an example, a set of learning input sentences X and a correct answer series L (normalized phrase structure tree) presented in FIG. 17 as one set of learning data. Normalization here means replacing the words and part-of-speech of all leaf nodes described in the beginning of the manuscript with the same label such as XX, and the normalized phrase structure tree is expressed as a series And point to a normalized phrase structure tree.

  The position of each word in the learning input sentence X is i, i = {1,..., N}, and n = 6 in the present embodiment. Let t be the position of each phrase structure label in the correct answer series L, t = {1,..., M}, and m = 15 in this embodiment.

  The arithmetic unit 220 includes a learning data generation unit 230, a learning unit 232, and a phrase structure analyzer 240. The phrase structure analyzer 240 is the same as the phrase structure analyzer 40 of the phrase structure analyzer 100, and the phrase structure analyzer learned by the phrase structure learning device 200 is referred to as the phrase structure analyzer 240. .

  The learning data generation unit 230 is a phrase structure label string including a learning input sentence and phrase structure labels of each node of a phrase structure tree representing the learning input sentence, and is based on the normalized phrase structure label string To generate learning data consisting of the correspondence between words and phrase structure labels for learning the attention mechanism 3 for outputting the weight of each word of the learning input sentence to the phrase structure label, which is included in the phrase structure analyzer. Do.

The method of creating the correct answer α _i ^t in the learning data generation unit 230 will be described in detail below.

The learning data generation unit 230 outputs the correspondence α of the correct answer based on the learning input sentence X and the normalized phrase structure tree L. α _i ^t is a variable that returns 1 when there is a correspondence between the word x _i and the phrase structure label l _t , and returns 0 when there is not.

In order to obtain the correct answer α _i ^t , the learning data generation unit 230 associates the words constituting the learning input sentence with the phrase structure labels constituting the normalized phrase structure tree. An example of the correspondence is shown in FIG. 18 and FIG.

  A non-terminal symbol XX representing the presence of a word and a part of speech corresponding thereto in the correct answer series L is associated with the word in the learning input sentence X. The <s> and </ s> representing the first and the last of the correct answer series L are respectively associated with <s> and </ s> in the input sentence X. A word corresponding to a phrase structure label other than the non-terminal symbol XX, <s>, </ s> of the correct answer series L is included in a phrase structure tree whose apex is the phrase structure label when the tree structure is assumed. It is selected from a set of words in the learning input sentence. That is, it is selected from a set of words of a learning input sentence which is a subordinate leaf node to a node of the phrase structure label. Use one of the following methods for selection.

1: Select a word that is the head of a phrase structure tree having the phrase structure label as a vertex 2: select the leftmost word in the set of words included in the phrase structure tree having the phrase structure label as a vertex 3: Select the rightmost word among the set of words included in the phrase structure tree having the phrase structure label at the top 4: When the phrase structure label includes “(”, the words constituting the phrase having the label at the vertex If the phrase structure label includes ")", select the rightmost word in the set of words included in the phrase structure tree whose apex is the phrase structure label.

  In the method 1 described above, it is necessary to manually add information to the learning data manually from the head rule or the like or separately analyze which word is the head of the phrase structure tree. Method 4 is a combination of methods 2 and 3.

  The learning data generation unit 230 associates the words of the learning input sentence, which is a lower leaf node with respect to the node of the phrase structure label, for each of the phrase structure labels included in the phrase structure label string thus normalized. Generate learning data.

  In the present embodiment, the above method 4 is adopted, and the processing result is shown in FIG.

The learning unit 232 learns about the phrase structure analyzer 240 including the attention mechanism 3 with the correct phrase structure label l _t as the correct data of the output y _t of the output unit based on the learning data generated by the learning data generation unit 230. I do. A general learning method may be used as a learning method of a neural network. In this embodiment, optimization of learnable parameters is performed by the stochastic gradient descent method.

As for the learning of the caution mechanism 3, each of the hidden state vectors corresponding to each word of the learning input sentence and the hidden state vector for the phrase structure label output immediately before are input as The learning is performed so that the correct solution α _i ^t to be created and the estimated α _i ^t output from the attention mechanism 3 become equal. A general learning method may be used as a learning method of a neural network. In this embodiment, optimization of learnable parameters is performed by the stochastic gradient descent method.

<Operation of Phrase Structure Learning Device According to the Embodiment of the Present Invention>

  Next, the operation of the phrase structure learning device 200 according to the embodiment of the present invention will be described. The phrase structure learning device 200 executes a phrase structure learning process routine shown in FIG.

  First, in step S200, a learning input sentence and a normalized phrase structure tree are received.

  In step S202, a phrase structure label string comprising the learning input sentence and the phrase structure label of each node of the phrase structure tree representing the learning input sentence, the phrase being based on the normalized phrase structure label string The learning data is formed of the correspondence between words and phrase structure labels for learning the attention mechanism 3 for outputting the weight of each word of the learning input sentence to the phrase structure label, which is included in the structure analyzer 240.

In step S204, based on the learning data generated by the learning data generation unit 230, for the phrase structure analyzer 240 including the caution mechanism 3, learning is performed using the correct phrase structure label l _t as the correct data of the output y _t of the output unit. Perform and complete the process. The processing routine in the neural network of the phrase structure analyzer 240 is the same as that shown in FIG.

  Next, details of the learning data generation processing routine in step S202 will be described with reference to FIG.

  In step S2000, t = 1.

  In step S2002, i = 1.

In step S2004, it is determined whether the phrase structure label l _{t of} the normalized phrase structure tree is <s> (or </ s>). If the condition is satisfied, the process proceeds to step S2006, and the condition is not satisfied. In the case, the process moves to step S2008.

In step S2006, it is determined whether word x _{i of the} input sentence for learning is <s> (or </ s>. If the condition is satisfied, the process proceeds to step S2018. If the condition is not satisfied, the process proceeds to step S2020 Do.

In step S2008, it is determined whether the normalized phrase structure tree l _t is XX. If the condition is satisfied, the process proceeds to step S2010. If the condition is not satisfied, the process proceeds to step S2012.

In step S2010, the word _{x i} of the learning input sentence, determines whether corresponding to XX of _{l t,} if the condition is satisfied and proceeds to step S2018, if the condition is not satisfied the process proceeds to step S2020.

In step S2012, it is determined whether containing _{l t} is "(", if it contains the process proceeds to step S2014, if not included proceeds to step S2016.

In step S2014, it is determined whether the word x _i of the learning input sentence is included in the phrase structure tree having l _t as a vertex and is the rightmost. If the condition is satisfied, the process proceeds to step S2018 to satisfy the condition If not, the process proceeds to step S2020.

In step S2016, it is determined whether the word x _i of the learning input sentence is included in the phrase structure tree having l _t as a vertex and is the leftmost. If the condition is satisfied, the process proceeds to step S2018 to satisfy the condition If not, the process proceeds to step S2020.

In step S2018, α _i ^t = 1 is set.

In step S2020, α _i ^t = 0.

  In step S2022, i = i + 1. Note that counting up i is written as i = i + 1.

  In step S2024, it is determined whether i> n. If the condition is satisfied, the process proceeds to step S2026. If the condition is not satisfied, the process proceeds to step S2004.

  In step S2026, it is determined whether t <m. If the condition is satisfied, the process proceeds to step S2026. If the condition is not satisfied, the process ends.

  As described above, according to the phrase structure learning device 200 according to the embodiment of the present invention, the phrase structure label string is composed of an input sentence and a phrase structure label of each node of a phrase structure tree representing the input sentence , A word and a phrase for learning the attention mechanism 3 which outputs the weight of each word of the input sentence to the phrase structure label, which is included in the phrase structure analyzer 240, based on the normalized phrase structure label sequence and By generating learning data including correspondences of structure labels, it is possible to learn the phrase structure analyzer 240 for analyzing the phrase structure with high accuracy using the learning data for the attention mechanism 3.

  The present invention is not limited to the above-described embodiment, and various modifications and applications can be made without departing from the scope of the present invention.

  For example, in the embodiment described above, the phrase structure learning device 200 generates learning data by the learning data generation unit 230, and the learning unit 232 learns the phrase structure analyzer 240 including the attention mechanism 3 as an example. However, the present invention is not limited to this, and the generation of the learning data of the learning data generation unit 230 and the learning of the phrase structure analyzer 240 including the attention mechanism 3 by the learning unit 232 may be realized by different devices. May be

Reference Signs List 1 encode unit 2 decode unit 3 caution mechanism 4 output unit 10, 210 input unit 20, 220 operation unit 30 phrase structure analysis unit 40, 240 phrase structure analysis unit 100 phrase structure analysis device 200 phrase structure learning device 230 learning data generation unit 232 Learning department

Claims (7)

A phrase structure learning device for learning a phrase structure analyzer that outputs a phrase structure label sequence for an input sentence, comprising:
The input sentence for the phrase structure label included in the phrase structure analyzer, based on the input sentence and a phrase structure label string consisting of phrase structure labels of each node of the phrase structure tree representing the input sentence A learning data generation unit for generating learning data composed of correspondence between the word and the phrase structure label for learning an attention mechanism that outputs a weight of each word;
Phrase structure learning device including.   The learning data generation unit is configured to associate the words of the input sentence, which is a leaf node lower than the node of the phrase structure label, for each of the phrase structure labels included in the phrase structure label string. The phrase structure learning device according to claim 1, which generates The phrase structure analyzer outputs the phrase structure labels in order from the top, and
The attention mechanism receives, as inputs, each of a hidden state vector corresponding to each word of the input sentence and a hidden state vector for the phrase structure label output immediately before, the input sentence for the phrase structure label Output the weight of each word of
The phrase structure learning device according to claim 1, further comprising a learning unit that learns the attention mechanism based on the generated learning data. A pre-learned phrase structure analyzer that outputs a phrase structure label string for an input sentence, using a phrase structure analyzer including a caution mechanism that outputs a weight of each word of the input sentence to a phrase structure label, A phrase structure analysis unit that takes the input sentence as an input and outputs the phrase structure label string for the input sentence,
A phrase structure analyzer including
The attention mechanism is
Learning consisting of correspondence between the word and the phrase structure label generated based on the learning input sentence and the phrase structure label string consisting of the phrase structure label of each node of the phrase structure tree representing the learning input sentence A phrase structure analysis device that is learned in advance based on data. A phrase structure learning method in a phrase structure learning device for learning a phrase structure analyzer that outputs a phrase structure label sequence for an input sentence, comprising:
The phrase structure label contained in the phrase structure analyzer, based on the input sentence and a phrase structure label string comprising a phrase structure label of each node of a phrase structure tree representing the input sentence, the learning data generation unit Generating learning data consisting of the correspondence between the words and the phrase structure label, for learning an attention mechanism that outputs the weight of each word of the input sentence with respect to
Phrase structure learning method including. A phrase structure analysis unit is a pre-learned phrase structure analyzer that outputs a phrase structure label string for an input sentence, and includes a caution mechanism that outputs a weight of each word of the input sentence to a phrase structure label. Using the analyzer as input to the input sentence and outputting the phrase structure label string for the input sentence;
A phrase structure analysis method including
The attention mechanism is
Learning consisting of correspondence between the word and the phrase structure label generated based on the learning input sentence and the phrase structure label string consisting of the phrase structure label of each node of the phrase structure tree representing the learning input sentence A phrase structure analysis method that is prelearned based on data.   The program for functioning a computer as each part of the phrase structure learning apparatus of any one of Claims 1-3, or the phrase structure analysis apparatus of Claim 4.