JP5087994B2 - Language analysis method and apparatus - Google Patents

Description

The present invention relates to a language analysis method and apparatus for analyzing a language.

  When machine translation is performed by a natural language processing apparatus such as a machine translation system, for example, the dependency structure of a sentence is analyzed with respect to an input sentence input by a user, and the translation processing is performed based on the analysis result. Done.

  In the natural language processing apparatus, it is often necessary to analyze the dependency structure of the input sentence. The dependency structure refers to, for example, a structure representing a modification / modification relationship (dependency relationship) between words or phrases.

IPSJ Journal, Vol. 40, no. 9 (1999) Kiyotaka Uchimoto and two others, “Japanese dependency analysis using a model based on the maximum entropy method”, p. 3397-3407 IPSJ Journal, Vol. 43, no. 6 (2002) Taku Kudo, Yuji Matsumoto, “Japanese dependency analysis by applying the chunking stage”, p. 1834-1841 Computational Linguistics, Vol. 31, no. 1 (2004) Collins and Koo “Discriminative Learning for Natural Language Parsing”

  As described in Non-Patent Document 1, as a method of analyzing the dependency structure, a rule-based method or a statistical method can be considered, and the statistical method is used for learning parameters and the like necessary for the dependency structure analysis. There is an advantage that it can be automatically estimated from data. In a dependency structure analysis based on a statistical method, a dependency structure for an unknown sentence is determined based on information (features) that is a clue contained in data. Therefore, the accuracy of the dependency structure analysis depends on what features are used. In the technique described in Non-Patent Document 1, a sentence dependency structure is statistically obtained by calculating a probability that two phrases have a dependency relationship using a maximum entropy model.

  In the technique described in Non-Patent Document 2, by using a support vector machine, it is applied step by step to determine whether two adjacent clauses are related or not related, whereby the dependency structure of the whole sentence is applied. Is determined. The dependency structure analysis method shown in Non-Patent Document 2 does not assume that each dependency relationship in a sentence is independent, but pays attention when determining whether or not two clauses are in a dependency relationship. Consideration is also given to the clauses related to the current source clause, the clauses related to the current target clause, and the features obtained from the clauses related to the current target clause. By using such a feature, it is possible to determine the sentence dependency structure with higher accuracy.

  In Non-Patent Document 3, the phrase structure of a sentence is determined using a method based on reranking.

  However, the conventional language analysis techniques have the following problems (A) to (D).

(A) Problems of the technology described in Non-Patent Document 1 In the dependency structure analysis method shown in Non-Patent Document 1, it is assumed that each dependency relationship in the sentence is independent, and the available features are limited. Therefore, there is a problem that the analysis may fail if this assumption is not satisfied.

(B) Problems of the technology described in Non-Patent Document 2 In the dependency structure analysis method shown in Non-Patent Document 2, not only the feature of each phrase, but also the respective destinations of a plurality of phrases included in the sentence. Some sentence unit features can be used. That is, since each dependency relationship in the sentence is determined one by one in order, the features obtained from the clauses for which the dependency relationship has already been determined can be used. However, a feature obtained from a clause whose dependency relationship has not yet been determined cannot be used, and an arbitrary feature of a sentence unit cannot be used.

(C) Problem of the technique described in Non-Patent Document 3 In Non-Patent Document 3, a method based on reranking is used in order to perform phrase structure analysis using features of arbitrary sentence units. In this method, first, the top x candidate solutions (eg, x = 30) that are considered to be correct are obtained by using the conventional method. Next, for each of the x candidates, which candidate is considered to be the most correct is determined using a sentence unit feature. In this method, since all the phrase structures in the sentence have already been determined for each of the x candidates, any feature of the sentence unit can be used. However, the analysis method shown in Non-Patent Document 3 has a problem that a correct answer cannot be output in any way unless the correct solution is included in the first x solution candidates obtained. .

(D) Applicant's previous proposal (Japanese Patent Application No. 2007-10871, in a non-known state, hereinafter simply referred to as “previous proposal”)
In order to solve the problems (A) to (C), the applicant of the present application first uses a Gibbs sampling as a method of analyzing the dependency structure, and statistically utilizes an arbitrary feature in the sentence. A method to analyze the dependency structure of sentences was proposed. This previous proposal has a feature that it is possible to analyze a dependency structure without preparing a solution candidate in advance by using an arbitrary feature of a sentence unit.

  That is, the previous proposal is a language processing method for analyzing a dependency structure for an input sentence, and a phrase unit model parameter that is a parameter of a probabilistic model for analyzing a dependency structure using a phrase unit feature is estimated. A unit model parameter estimating step, a sentence unit model parameter estimating step for estimating a sentence unit model parameter which is a parameter of a probability model for analyzing a dependency structure using sentence unit features, the sentence unit model parameter and the clause A sample generation step for generating a sample of a dependency structure for an input sentence from a probability model defined by a unit model parameter, and a dependency for determining an optimum dependency structure from the sample of the dependency structure for the input sentence generated in the sample generation step And a structure determination step.

However, this proposal also has the following disadvantages.
In the previously proposed method, as a method for selecting a solution of the dependency structure (the relationship destination of each word in the sentence), a method of selecting a dependency destination that maximizes the peripheral probability calculated by Gibbs sampling is used. The dependency structure tree has a property that it does not include a cycle (node sequence circulating in the graph), and further, depending on the language, there is a property that the dependency relationship does not intersect. However, the solution determined by the previously proposed method is not always in the correct form as a dependency structure tree. In addition, the previous proposal method only determines the destination of each word in the sentence, but in the application of natural language processing, not only what word each word in the sentence relates to, but also how It is often necessary that the relationship be related (such as whether it is the subject, the object, or a parallel relationship).

  Therefore, it has been difficult to realize a language analysis technique that is sufficiently technically satisfactory.

  Therefore, in the present invention, in order to solve the above-described problem, a maximum spanning tree search method is applied, and only a correct form solution is output as a dependency structure tree. It is an object of the present invention to provide a language analysis method and apparatus for identifying a dependency-related label.

A language analysis method according to a first aspect of the present invention is a language analysis method for performing language analysis on an input sentence divided into words or phrases using a computer, and has been parsed in a storage means. The phrase unit model parameter estimation procedure for determining the phrase unit model parameter in the phrase unit probability distribution based on the phrase unit probability distribution, the phrase unit model parameter, and the corpus, the phrase unit probability distribution from the phrase unit probability distribution in the corpus A parameter estimation sample generation procedure for generating a sample of a dependency structure tree for each sentence, a sentence unit model parameter estimation procedure for obtaining a sentence unit model parameter in a sentence unit probability distribution based on the sample and the corpus, and the input sentence And an input procedure for receiving the phrase unit model parameter in the input sentence. If, based on the probability distribution of the dependency structure of the sentence which is defined by, and the sentence model parameters, and a dynamic sample generating step of generating a sample of dependent parse tree of a sentence units by Gibbs sampling, that were generated the A solution search procedure for obtaining a score for the sample based on a marginal probability and determining an optimum dependency structure using the Eiser method or the Chu-Liu-Emonds method .

The language analysis method according to the second invention is the language analysis method according to the first invention, wherein when the determined dependency structure is an unlabeled dependency structure, the labelless dependency structure is labeled. It is characterized by having a dependency relationship label determination procedure for identifying a dependency relationship label using a probability model defined by model parameters.

A language analysis method according to a third aspect of the present invention is a language analysis method for performing language analysis on an input sentence divided into words or phrases using a computer, based on a parsed corpus stored in a storage means, Based on the phrase unit model parameter estimation procedure with label information for obtaining the phrase unit model parameter with label information in the phrase unit probability distribution, the phrase unit model parameter with label information, and the corpus, from the phrase unit probability distribution A parameter estimation sample generation procedure for generating a labeled dependency structure tree sample for each sentence in the corpus, and a sentence for determining a sentence unit model parameter with label information in a sentence unit probability distribution based on the sample and the corpus Unit model parameter estimation procedure and input to accept the input sentence Based on the establishment distribution of the dependency structure of the sentence unit defined by the procedure, the phrase unit model parameter with the label information, and the sentence unit model parameter with the label information, the dependency structure tree of the sentence unit using Gibbs sampling A labeled dynamic sample generation procedure for generating a sample, and a solution search procedure for determining an optimum dependency structure from the generated sample using a maximum spanning tree search method .

A language analysis method according to a fourth aspect of the present invention is a language analysis method for performing language analysis using a computer for an input sentence divided into words or phrases, based on a parsed corpus stored in a storage means, Clause unit model parameter estimation procedure for obtaining unlabeled phrase unit model parameters in the phrase unit probability distribution, labeling model parameter estimation procedure for obtaining labeling model parameters using the corpus, and the label information-free clause unit model Generating labeled sample for parameter estimation that generates a labeled dependency structure tree sample for each sentence in the corpus from the phrase-based probability distribution based on the parameter, the corpus, and the labeling model parameter Based on the procedure, the sample and the corpus, The sentence unit model parameter estimation procedure for obtaining the sentence unit model parameter with the label information, the input procedure for receiving the input sentence, the phrase unit model parameter without the label information, and the sentence unit model parameter with the label information Based on the establishment distribution of sentence-by-sentence dependency structure, a labeled dynamic sample generation procedure for generating a sentence-by-sentence dependency structure tree sample using Gibbs sampling, and using the maximum spanning tree search method from the generated sample And a solution search procedure for determining an optimum dependency structure .

A language analysis apparatus according to a fifth aspect of the present invention is a language analysis apparatus that performs a language analysis on an input sentence divided into words or phrases, and is based on a syntactically analyzed corpus stored in a storage means, and the probability of each phrase Based on the phrase unit model parameter estimation unit for determining the phrase unit model parameter in the distribution, the phrase unit model parameter, and the corpus, a sample of the dependency structure tree for each sentence in the corpus is obtained from the phrase unit probability distribution. A parameter generation sample generation unit for parameter estimation, and a sentence unit model parameter estimation unit for determining a sentence unit model parameter in a sentence unit probability distribution based on the sample and the corpus;
Based on the probability distribution of the sentence-dependent dependency structure defined by the input unit that receives the input sentence, and the sentence-unit model parameter and the sentence-unit model parameter in the input sentence, dependency of the sentence unit by Gibbs sampling and a dynamic sample generator for generating samples of tree structure, obtains a score by marginal probabilities against that were generated said sample, to determine the optimal dependency structure using Eiser method or Chu-Liu-Edmonds method solutions And a search unit.

  The language analysis device according to a sixth aspect of the present invention is the language analysis device according to the fifth aspect, wherein, when the determined dependency structure is an unlabeled dependency structure, the labelless dependency structure is labeled. It is characterized by having a dependency relationship label determining unit for identifying a dependency relationship label using a probability model defined by model parameters.

A language analysis apparatus according to a seventh aspect of the present invention is a language analysis apparatus that performs a language analysis on an input sentence divided into words or phrases, and is based on a syntactically analyzed corpus stored in a storage means, and the probability of each phrase Based on the phrase unit model parameter estimation unit with label information for determining the phrase unit model parameter with label information in the distribution, the phrase unit model parameter with label information, and the corpus, from the probability distribution of the phrase unit in the corpus A parameter generation sample generation unit for generating a labeled dependency structure tree sample for each sentence, and a sentence unit model parameter estimation for obtaining a sentence unit model parameter with label information in a sentence unit probability distribution based on the sample and the corpus A part, an input part that receives the input sentence, and the input sentence Said label information with clause unit model parameters, the label and sentence model parameters with information, with labels that produce a sample of the dependent parse tree of a sentence units by Gibbs sampling based on the probability distribution of the dependency structure of the sentence, which is defined by and a dynamic sample generating unit, from that were generated said sample, characterized by having a a solution search means for determining the optimum dependency structure using the maximum spanning tree search method.

A language analyzing apparatus according to an eighth aspect of the present invention is a language analyzing apparatus that performs a language analysis on an input sentence divided into words or phrases, and is based on a parsed corpus stored in a storage means, and a probability of each phrase. Clause unit model parameter estimation unit for obtaining unlabeled phrase unit model parameter in distribution, Labeling model parameter estimation unit for obtaining labeling model parameter using the corpus, Clause unit model parameter without label information, A sample generation unit with a parameter estimation for parameter estimation that generates a sample of a labeled dependency structure tree for each sentence in the corpus from a probability distribution in a unit of clauses based on a corpus and the model parameter for labeling; And the label information in the sentence-wise probability distribution based on the corpus A sentence model parameter estimation unit for obtaining a sentence model parameter attached, an input unit for receiving the input sentence in the input sentence, and the label information without clause unit model parameters, and the label information with Buntan'i model parameters, by Based on the probability distribution of the sentence-dependent dependency structure, a labeled dynamic sample generation unit that generates a sentence-dependent dependency tree sample by Gibbs sampling , and a maximum spanning tree search method from the generated sample And a solution search unit for determining an optimum dependency structure.

  According to the language analysis method and apparatus of the present invention, in the dependency structure analysis method using Gibbs sampling that can use all the features in a sentence and its system, the search method of the maximum spanning tree is used when searching for a solution. By applying, a correct solution can be obtained as a dependency structure tree.

  The language analysis apparatus, for an input sentence, from a probability model defined by a phrase unit model parameter and a sentence unit model parameter, a dynamic sample generation unit that generates a sample of a dependency structure, and the generated sample, And a solution search unit that determines an optimum dependency structure using a maximum spanning tree search method.

(Configuration of Example 1)
FIG. 1 is a schematic configuration diagram of a language analysis apparatus showing Embodiment 1 of the present invention.
This language analysis device is, for example, a device configured by executing a language analysis program by a computer having a central processing unit (hereinafter referred to as “CPU”) and an external storage device or an internal storage device. An analysis unit 10 that determines a dependency relationship by analyzing a dependency structure of a sentence, a model storage unit 20 that stores parameters of a probability model used when analyzing the dependency structure, and a model from a parsed corpus The parameter estimation part 30 etc. which perform parameter learning are comprised.

  The analysis unit 10 includes an input unit 11 such as a keyboard for inputting a sentence to be subjected to dependency structure analysis from a user, an unlabeled dynamic sample generation unit 12, a solution search unit 13, and a dependency relationship label determination. Section 14 and an output section 15 such as a display or a printer. The unlabeled dynamic sample generation unit 12, the solution search unit 13, and the dependency relationship label determination unit 14 are realized, for example, by program control of the CPU.

  Among them, the unlabeled dynamic sample generation unit 12 is connected to the input unit 11 and uses the information of the phrase unit model parameter without label information and the label unit model without label information for the input sentence, A large number of unlabeled dependency structural analysis tree samples are generated from the probability distribution defined by these model parameters, and a solution search unit 13 is connected to the output side. The solution search unit 13 determines a dependency structure of an input sentence using the generated sample, and a dependency relationship label determination unit 14 is connected to the output side. The dependency relationship label determination unit 14 assigns a dependency relationship label to a sentence whose dependency structure has been determined using the information of the labeling model parameter. On the output side, the output unit 15 It is connected. The output unit 15 is a device that outputs a sentence for which a dependency structure and its dependency relationship are determined to a user.

  The model storage unit 20 includes a first storage unit (for example, a no-label information phrase unit model parameter storage unit) 21, a second storage unit (for example, a no-label information sentence unit model parameter storage unit) 22, and a third Storage unit (for example, a labeling model parameter storage unit) 23, and these are configured by an external or internal storage device. Among them, the no-label-information clause unit model parameter storage unit 21 stores parameters of the no-label-information clause unit model used in the unlabeled dynamic sample generation unit 12 and the like. The label informationless sentence unit model parameter storage unit 22 stores parameters of the label informationless sentence unit model used in the unlabeled dynamic sample generation unit 12. Further, the labeling model parameter storage unit 23 stores parameters of the labeling model used by the dependency relationship label determination unit 14.

  The parameter estimation unit 30 includes a syntactically analyzed corpus storage unit 31, a label unit without phrase information model parameter estimation unit 32, a parameter estimation unlabeled sample generation unit 33, a label information without sentence unit model parameter estimation unit 34, and a labeling Model parameter estimation unit 35, which are connected to each other. Among them, the parsed corpus storage unit 31 stores parsed corpora used in the other parameter estimation units 32, 34, and 35 and the sample generation unit 33, and is configured by an external or internal storage device. Has been.

  The other parameter estimation units 32, 34, and 35 and the sample generation unit 33 are realized, for example, by program control of the CPU. Among them, the phrase unit model parameter estimation unit 32 without label information obtains the parameters of the phrase unit model without label information using the corpus stored in the parsed corpus storage unit 31, and the result is used as the phrase unit model without label information. This is stored in the parameter storage unit 21. The parameter estimation unlabeled sample generation unit 33 uses the corpus stored in the parsed corpus storage unit 31 and the label information-free phrase unit model parameter stored in the label information-free phrase unit model parameter storage unit 21. The sample of the dependency structure without the label for each sentence in the corpus is generated from the probability distribution defined by the parameter, and the generated sample is given to the sentence unit model parameter estimation unit 34 without the label information.

  The label information-less sentence unit model parameter estimation unit 34 uses the corpus stored in the parsed corpus storage unit 31 and the sample generated by the parameter estimation unlabeled sample generation unit 33, and uses the label information-less sentence unit. The model parameters are obtained, and the results are stored in the sentence unit model parameter storage unit 22 without label information. Further, the labeling model parameter estimation unit 35 obtains parameters of the labeling model using the corpus stored in the parsed corpus storage unit 31 and stores the result in the labeling model parameter storage unit 23. To do.

(Dependent structure analysis processing in language analysis method of embodiment 1)
FIG. 2 is a flowchart showing the existing structure analysis processing until the language analysis apparatus shown in FIG. 1 analyzes and outputs the sentence inputted by the user.

First, the user inputs a sentence to be subjected to dependency structure analysis by the input unit 11 (step S1). However, it is assumed that the input sentence is already divided into words or phrases, and the part of speech or the like is estimated. Here, the t-th clause (word) in the input sentence is represented by w _t . The entire input sentence is represented by a set W of clauses in the input sentence as shown in the following formula (1), and the number of clauses included in the sentence is represented by | W |.

Further, dependency destination of t-th clause what number clause in a sentence to be represented by h _t, the set of h _t as shown in the following formula (2) to be represented by H.

However, since the clause that is the main word of the whole sentence has no dependency destination, the dependency destination of such a clause is represented by, for example, zero (0). t-th clause of dependency relationships labels (whether the subject or the like) to be represented by l _t, the set of l _t as shown in the following formula (3) to be represented by L.

  Therefore, the dependency structure analysis which is the object of the language analysis method of the first embodiment is to obtain the set H of the relation destination of each clause and the set L of the label when the clause string W is input.

  For the phrase string W input by the input unit 11, the unlabeled dynamic sample generation unit 12 includes the label information-free phrase unit model parameter stored in the label information-free phrase unit model parameter storage unit 21 and no label information. Using unlabeled sentence unit model parameters stored in the sentence unit model parameter storage unit 22 and dynamically analyzing the unlabeled dependency structure by Gibbs sampling from the probability distribution of the dependency structure of the sentence unit defined by those parameters A tree sample is generated (step S2).

  Here, the Gibbs sampling (or Gibbs sampler, heat bath method), which is also shown in the previous proposal, will be briefly described.

Gibbs sampling is a technique for dynamically generating samples by rewriting variable values one after another using random numbers. The part for rewriting the value of the variable is performed by sampling from a conditional distribution. The variable x is divided into several components and written as x = {x _i }, i = 1,..., N. Even if x _i is a natural element originally, it is regrouped into several components. Things can be used.

Suppose that each time one component x _i is selected, the current value is forgotten and a new one is taken. Do not refer to previous values of the same component when renewing. At that time, Gibbs sampling is to select a new value of x _i with a conditional probability P (x _i | x ₁ ,..., X _i−1 , x _{i + 1} ,..., X _N ) with other components fixed. . It is a feature that the new value is not referred to the previous value of the same component. If it is a continuous variable, we will consider conditional density.

  Starting from an arbitrary initial state, the operation of re-taking one component according to a conditional probability with other components fixed is repeated indefinitely. The method of selecting the component i may be selected at random each time, or may be selected in the order determined before starting the calculation. Gibbs sampling is an approach that divides a problem into pieces that can be handled efficiently and integrates them. The point of Gibbs sampling is that the state is partially updated by sampling from the conditional distribution as a method of integration. Even in the dimension, the calculation can be prevented from failing due to the gap between the part and the whole.

It is assumed that the target distribution is posterior distribution π ^* and the probability density function is π (θ | x), and θ can be divided into several blocks as θ = (θ1,..., Θp). In addition, θ _−i = (θ1,..., Θ _i−1 , θ _{i + 1} ,..., Θp) and x are given, and the probability density function of the conditional posterior distribution π ^* _i is expressed as π (θ _i | θ _−i , x) and assume that sampling from this conditional distribution is easy. At this time, the Gibbs sampling algorithm is
(A) The initial value θ (0) = (θ ⁽⁰⁾ ₁ , θ ⁽⁰⁾ ₂ ,..., Θ ⁽⁰⁾ _p ) is determined, and t = 1 is set.
(B) For i = 1,.
θ ^(t) ₁ , ~ π (θ _i , | θ ^(t) _−i , x),
θ ^(t) _−i = (θ ^(t) ₁ ,..., θ ^(t) _i−1 , θ ^(t−1) _{i + 1} ,..., θ ^(t−1) _p ),
Is generated.
(C) Set t to t + 1 and return to (b).

(B) and (c) are repeated, and θ ^(t) = (θ ^(t) ₁ , θ ^(t) ₂ ,..., Θ ^(t) _p ) when t ≧ N for a sufficiently large number N. A probability sample with a posterior distribution π ^* .

This is the end of the explanation of Gibbs sampling. The Gibbs sampling executed by the unlabeled dynamic sample generation unit 12 is the same as the Gibbs sampling described in Reference Document 1 below.
Reference 1; Yukito Iba, 5 others "Calculation Statistics II-Markov Chain Monte Carlo Method and its Surroundings" (2005) Iwanami Shoten

  Various probability distributions of dependency structures in units of sentences, that is, probability distributions of the set H when a phrase string W is given, can be considered, for example, the following as shown in the above proposal: The probability distribution P (H | W) defined by the following equations (4) to (8) can be used.

In these equations (4) to (8), Q (H | W) is the probability distribution of the dependency structure of the phrase unit, λ _k is the sentence unit model parameter, and Ω (W) is the phrase set W. A set of all possible H given, f _k (W, H) is an arbitrary sentence unit feature, μ _i is a phrase unit model parameter, and g _i (W, t , H) is a phrase-unit feature relating to the fact that the t-th phrase in the phrase string W relates to the h-th phrase. As the feature _{g i (W, t, h} ) described, for example, each heading w _t and w _h, part of speech, or the distance between the clauses, the presence or absence of commas between clauses such as the non-patent document 1 Such features as those described above can be used.

  Non-Patent Document 1 describes as follows. A feature is, for example, information for calculating the dependency probability between two clauses. Specifically, a feature is a surface character string, part of speech, usage form, presence or absence of parentheses or punctuation marks, distance between clauses. Or combinations thereof, and are listed in Table 2 (see Non-Patent Document 1, P. 3401). The features listed in Table 2 consist of feature names and feature values. When focusing on two clauses in one sentence, the attributes that each clause (previous clause and subsequent clause) can have or appear between two clauses Represents. Thus, the probability distribution of the dependency structure calculated using the phrase unit feature is calculated assuming independence between each dependency relationship in the sentence.

  For the probability distribution P (H | W) defined as described above, a sample can be generated using Gibbs sampling by the method shown in the previous proposal.

Using the S samples {H ⁽¹⁾ ,..., H ^(S) } generated by the unlabeled dynamic sample generation unit 12, the solution search unit 13 searches for a solution that seems to be an optimal dependency structure. It is determined (step S3). Here, peripheral probabilities defined as the following formulas (9) and (10) are used.

Here, P _t (h | W) is a probability that the t-th word is h. When a candidate that simply maximizes this value is selected as a solution, the resulting structure may be an inappropriate structure as a dependency structure tree including cycles. Therefore, an optimum dependency structure tree is searched using the maximum spanning tree search method. The dependency structure analysis can be solved as a maximum spanning tree search problem as described in Reference Document 2 below, for example.
Reference 2; University of Pennylvania Department of Computer and Information Science Technical Report No. MS-CIS-06-11 (2006) McDonald et al. “Spanning Tree Methods for Discriminating Training of Dependency Parrs”

  As described in Reference 2, particularly in the case of a language in which the dependency relationship does not intersect, an optimum maximum spanning tree can be obtained by using the Eisner method. In the case of a language in which the dependency relationship intersects, An optimal maximum spanning tree can be obtained by using the Chu-Liu-Edmonds method. The algorithm for obtaining these maximum spanning trees is that a spanning tree that maximizes the sum of the scores when the score of the edge from node i to node j in the graph is s (i, j) is shown in the graph. Can be obtained from Therefore, the score s (i, j) is defined as the following equation (11) using the peripheral probability, where i is a related word and j is a related word.

By using the score calculated in this way together with the Eisner method and the Chu-Liu-Edmonds method, the solution search unit 13 can search for an optimum dependency structure.

The dependency relationship label determination unit 14 receives the dependency structure determined by the solution search unit 13 and determines a dependency relationship label for each word (step S4). Since the determination of the dependency relationship label can be solved as a multi-value classification problem, a technique for solving the multi-value classification problem can be used. For example, using the maximum entropy method or the support vector machine described in Reference 3 below, use the word to which the label is attached, the surrounding words and the word form or part of speech of the related word as features. The label can be determined.
Reference 3; Kimming, et al. "Statistical Science Frontier 10 Languages and Psychological Statistics" (2003) Iwanami Shoten

  For example, by using a maximum entropy model, a labeling model represented by a probability distribution such as the following equations (12) to (14) is defined, and a label L that maximizes the probability is obtained. it can.

In these equations (12) to (14), ν _i is a parameter of the labeling model, and e _j (W, H, t, l) is the t-th clause of the clause string W having H as the destination. Is a feature relating to having a dependency-related label of l. The output unit 15 outputs to the user the dependency structure analysis result that is output from the dependency relationship label determination unit 14 and that identifies the dependency structure and the dependency relationship with respect to the input sentence (step S5). The process ends.

(Parameter estimation process in language analysis method of embodiment 1)
FIG. 3 is a flowchart illustrating a parameter estimation process until a parameter necessary for performing the dependency structure analysis in the language analysis method of FIG. 1 according to the first embodiment is learned from the learning data and stored.

First, the phrase unit model parameter estimation unit 32 without label information uses the parsed corpus stored in the parsed corpus storage unit 31, and uses the parsed corpus unit parameter, that is, {μ in the expression (7). _i } is obtained and stored in the phrase unit model parameter storage unit 21 without label information (step S11).

  Next, the parameter estimation unlabeled sample generation unit 33 includes the parsed corpus stored in the parsed corpus storage unit 31 and the unlabeled phrase unit model stored in the label information-free phrase unit model parameter storage unit 21. Parameters are used to generate an unlabeled sample for parameter estimation for each sentence in the parsed corpus from the probability distribution Q (H | W) of the dependency structure of the phrase unit defined by those parameters. Then, it is passed to the sentence unit model parameter estimation unit 34 without the label information (step S12).

The sentence unit model parameter estimation unit 34 without label information includes a parsed corpus stored in the parsed corpus storage unit 31, a parameter-unlabeled sample generated by the parameter estimation label-free sample generation unit 33, and Is used to obtain the sentence unit model parameter {λ _k } without label information and stores it in the sentence unit model parameter storage unit 22 without label information (step S13).

  Thereafter, the labeling model parameter estimation unit 35 obtains the parameters of the labeling model using the parsed corpus stored in the parsed corpus storage unit 31, and obtains the parameters of the labeling model parameter storage unit 23. (Step S14), and the parameter estimation process is terminated.

(Effect of Example 1)
According to the language analysis method and apparatus of the first embodiment, in the dependency structure analysis method using Gibbs sampling that can use all the features in the sentence and the system thereof, the search of the maximum spanning tree is performed when searching for a solution. By applying the method, a correct solution can be obtained as a dependency structure tree. Furthermore, by determining the dependency labels as post-processing after determining the dependency destination of each word, it is possible to efficiently identify the dependency relationship labels with a small amount of calculation.

  In the language analysis method and apparatus according to the second embodiment, the dependency-related label estimation, which is performed as post-processing in the first embodiment, is performed at the same time when the dependency destination is determined.

(Configuration of Example 2)
FIG. 4 is a schematic configuration diagram of a language analysis apparatus showing a second embodiment of the present invention. Elements common to the elements in FIG. 1 showing the first embodiment are denoted by common reference numerals.

  The language analysis apparatus according to the second embodiment is different from the configuration according to the first embodiment in that it does not have the dependency relationship label determining unit 14 and handles label information in both the phrase unit model and the sentence unit model. In other words, the language analysis apparatus according to the second embodiment has a configuration different from that of the first embodiment, and includes the analysis unit 10A, the model storage unit 20A, and the parameter estimation unit 30A.

  The analysis unit 10A is connected to an input unit 11 similar to that of the first embodiment, a labeled dynamic sample generation unit 12A having a different configuration from that of the first embodiment connected to the input unit 11, and the sample generation unit 12A. The same solution search unit 13 as that of the first embodiment and the output unit 15 similar to that of the first embodiment connected to this solution search unit 13 are included. The model storage unit 20A has a different configuration from that of the first embodiment, and includes a first storage unit (for example, phrase unit model parameter storage unit with label information) 21A and a second storage unit (for example, sentence unit with label information). Model parameter storage unit) 22A. Further, the parameter estimation unit 30A includes a syntactically analyzed corpus storage unit 31 similar to that in the first embodiment, a phrase-unit model parameter estimation unit 32A with label information having a configuration different from that of the first embodiment, and a configuration of the first embodiment. The sample generation unit 33A with different parameter estimation for parameter estimation and the sentence unit model parameter estimation unit 34A with label information having a configuration different from that of the first embodiment, the corpus storage unit 31, the parameter estimation units 32A and 34A, and Sample generators 33A are connected to each other.

  In the analyzing unit 10A, the labeled dynamic sample generating unit 12A stores the sentence input from the input unit 11 into the phrase unit model parameter storage unit 21A with label information and the sentence unit model parameter storage unit 22A with label information. Using the stored information, a large number of labeled dependency structure analysis tree samples are generated from the probability distribution defined by these model parameters. The solution search unit 13 connected to the output side of the labeled dynamic sample generation unit 12A determines the labeled dependency structure of the input sentence using the generated sample, and outputs it to the output unit 15. . Other configurations are substantially the same as those of the first embodiment.

(Language analysis method of embodiment 2)
FIG. 5 is a flowchart showing the dependency structure analysis process until the language analysis apparatus shown in FIG. 4 analyzes and outputs the sentence input by the user, and the elements in FIG. Common elements are given common reference numerals.

  In the dependency structure analysis process according to the second embodiment, a sentence input process for dependency structure analysis (step S1) is performed in the same manner as in the first embodiment, and a dynamically labeled sample generation process for an input sentence (unlike the first embodiment) (step S1). Step S2A), an optimum labeled dependency structure search process (step S3A), and a process of outputting a result (S5) as in the first embodiment are executed.

  FIG. 6 is a flowchart showing parameter estimation processing until learning and storing parameters necessary for performing dependency structure analysis in the language analysis method of FIG. This corresponds to FIG.

  In the parameter estimation process of the second embodiment, unlike the first embodiment, the phrase unit model parameter estimation process with label information (step S11A), the parameter estimation labeled sample generation process (step S12A), and the label information The attached sentence unit model parameter estimation process (step S13A) is executed.

  The processing of FIGS. 5 and 6 of the second embodiment is different from the processing of the first embodiment in that label information is handled in the phrase unit model and the sentence unit model. This point will be described below. .

  In the first embodiment, only a relation destination is determined using a phrase unit and a sentence-unit probability model that do not handle label information in consideration of only the relation destination, such as Expressions (4) to (8). On the other hand, in the second embodiment, by using a probability model as shown in the following formulas (15) to (19) considering not only the dependency destination but also the dependency relationship label, both the dependency destination and the label are determined. Decide at the same time.

In these formulas (15) to (19), Q (H, L | W) is the probability distribution of the labeled dependent structure in phrase units, λ _k is the sentence unit model parameter with label information, and Ω (W ) Is a set of all possible combinations of H and L when W is given, f _k (W, H, L) is an arbitrary sentence unit feature, and μ _i is with label information It is a phrase unit model parameter, and g _i (W, t, h, l) is a phrase unit feature relating to the relationship of the t-th phrase in the phrase string W to the h-th phrase with a label of l. When this model is used, sample generation, parameter estimation, and the like can be performed in the same manner as in the first embodiment and the previous proposal.

  In FIG. 5, for the phrase string W input by the input unit 11 (step S1), the labeled dynamic sample generation unit 12A performs the phrase with label information stored in the phrase unit model parameter storage unit 21A with label information. By using the unit model parameter and the sentence unit model parameter with label information stored in the sentence unit model parameter storage unit 22A with label information, the probability distribution of the labeled dependent structure of the sentence unit defined by these parameters is given. A sample of the labeled dependency structure tree is dynamically generated by sampling (step S12).

Using the S labeled samples {H ⁽¹⁾ , L ⁽¹⁾ ,..., H ^(S) , L ^(S) } generated by the labeled dynamic sample generation unit 12A, the solution search unit 13 searches for and determines a solution that seems to be an optimal labeled dependency structure (step S13). Here, the peripheral probabilities defined by the following equations (20) to (22) are used.

Here, P _t (h, l | W) is a probability that the t-th word is h and the label is l. Using this peripheral probability, the score s (h, d, l) when the dependency destination word is h, the dependency source word is d, and the dependency relationship label is l is defined as follows.

By using such a score s (h, d, l), as in the case of the first embodiment, an optimal solution can be obtained by searching the maximum spanning tree.

(Effect of Example 2)
According to the second embodiment, the amount of calculation required is larger than that of the first embodiment, but by using the feature that integrates these pieces of information by simultaneously determining the dependency destination and identifying the label of the dependency relationship. Dependence structure analysis can be performed with high accuracy. That is, it becomes possible to handle the information on the clauses and labels of the relations in an integrated manner, and the dependency structure analysis can be performed with high accuracy.

  In the language analysis method and apparatus of the third embodiment, the first and second embodiments are combined, and label information estimation performed in the phrase unit model in the second embodiment is separately performed using a separately prepared labeling model. This is done as a process.

(Configuration of Example 3)
FIG. 7 is a schematic configuration diagram of a language analyzing apparatus showing a third embodiment of the present invention. Elements common to the elements in FIGS. 1 and 4 showing the first and second embodiments are denoted by common reference numerals. It is attached.

  The language analysis apparatus according to the third embodiment is different from the configuration according to the second embodiment in that the phrase unit model does not handle label information but uses a separately prepared labeling model to handle label information. That is, the language analysis apparatus according to the third embodiment includes an analysis unit 10A that is the same as that of the second embodiment, a model storage unit 20B that is different from the configuration of the second embodiment, and a parameter estimation unit 30B.

  The model storage unit 20B includes a first storage unit (for example, no phrase information phrase unit model parameter storage unit) 21 similar to the first embodiment and a third storage unit (for example, with label information) similar to the second embodiment. A sentence unit model parameter storage unit) 22A, and a second storage unit (for example, a labeling model parameter storage unit) 23 similar to that of the first embodiment. The parameter estimation unit 30B includes a parsed corpus storage unit 31 similar to that in the first embodiment, a phrase unit model parameter estimation unit 32 without label information, and a parameter generation sample generation unit 33A for parameter estimation similar to that in the second embodiment. And a sentence unit model parameter estimation unit 34A with label information and a labeling model parameter estimation unit 35 similar to that of the first embodiment, and these corpus storage unit 31, parameter estimation units 32, 34A, 35, and Sample generators 33A are connected to each other.

  In the analysis unit 10A, the labeled dynamic sample generation unit 12A performs the phrase unit model parameter storage unit 21 without label information, the sentence unit model parameter storage unit 22A with label information, for the sentence input from the input unit 11. Using the model parameter information stored in the labeling model parameter storage unit 23, a large number of labeled dependency structure analysis tree samples are generated from the probability distribution defined by these model parameters.

  Further, in the parameter estimation unit 30B, the parameter estimation labeled sample generation unit 33A includes a corpus stored in the parsed corpus storage unit 31 and no label information stored in the phrase unit model parameter storage unit 21 without label information. Using the phrase unit model parameter and the labeling model parameter stored in the labeling model parameter storage unit 23, a sample of the labeled dependency structure for each sentence in the corpus from the probability distribution defined by the parameter And the generated sample is given to the sentence unit model parameter estimation unit 34A with label information. Further, the labeling model parameter estimation unit 35 obtains parameters of the labeling model using the corpus stored in the parsed corpus storage unit 31, and stores the result in the labeling model parameter storage unit 23). To do. Other configurations are substantially the same as those of the first and second embodiments.

(Language analysis method of Example 3)
In the language analysis method of the third embodiment, the contents of the flowchart showing the dependency structure analysis processing until the sentence input by the user is analyzed and output are the same as those in FIG. 5 of the second embodiment.

  FIG. 8 is a flowchart showing parameter estimation processing until learning and storing parameters necessary for performing the dependency structure analysis in the language analysis method of FIG. 3 and FIG. 6 of the second embodiment.

  In the parameter estimation process of the third embodiment, the label information-free phrase unit model parameter estimation process (step S11) similar to the first embodiment and the labeling model parameter estimation process different from the first embodiment (step S14A). Then, the parameter estimation labeled sample generation process (step S12A) and the label information-added sentence unit model parameter estimation process (step S13A) similar to those in the second embodiment are executed.

  The processing of FIG. 8 of the third embodiment is different from the processing of the second embodiment in that the label unit is not handled by the phrase unit model but the label information is handled by using a separately prepared labeling model. This point will be described below.

  In the second embodiment, a phrase unit model that simultaneously considers the dependency destination and the dependency relationship label, such as equations (17) and (19), is used. Such a model can handle the information of the relationship destination and its label in an integrated manner, so it can be expected that the analysis can be performed with high accuracy, but the combination of the relationship destination and its label must be considered. The calculation amount increases.

  Therefore, in the third embodiment, as in the second embodiment, the relationship model and the label are simultaneously considered for the sentence unit model, and the model that considers only the dependency relationship is used for the phrase unit model as in the first embodiment. And the probability model is defined as in the following equations (23) to (25).

In these formulas (23) to (25), Q (H | W) is a probability distribution of the unlabeled dependency structure defined by the formula (6), and P (L | W, H) is a formula. (12) is a model for labeling, λ _k is a sentence unit model parameter with label information, and Ω (W) is a set of all possible combinations of H and L when W is given. F _k (W, H, L) is an arbitrary sentence unit feature. Sample generation, parameter estimation, and the like when this model is used can be performed in the same manner as in the first embodiment and the previous proposal.

  In the analysis unit 10A, for the phrase string W input by the input unit 11, the labeled dynamic sample generation unit 12A performs the label information-free phrase unit model parameter stored in the label information-free phrase unit model parameter storage unit 21. And the sentence unit model parameter with label information stored in the sentence unit model parameter storage unit 22A with label information, and the model parameter for labeling stored in the model parameter storage unit 23 for labeling, these parameters A labeled dependency structure tree sample is dynamically generated by Gibbs sampling from the probability distribution of the dependency structure of the sentence unit defined by. Based on this generation result, the solution search unit 13 and the output unit 15 perform the same processing as in the second embodiment.

(Effect of Example 3)
According to the third embodiment, the phrase unit model does not need to consider the label information, so the amount of calculation is reduced. However, the sentence unit model considers the label information and can perform dependency structure analysis with high accuracy. It becomes like this. In other words, only the determination of the relationship destination is performed in the phrase unit model, the identification of the label is handled by another model, and the determination of the relationship destination and the label identification are performed simultaneously in the sentence unit model, thereby reducing the calculation amount. However, dependency structure analysis can be performed with high accuracy.

(Modification)
The present invention is not limited to the illustrated embodiment, and the language analysis device and the language analysis method can be variously used and modified in configurations and processing procedures other than those illustrated.

BRIEF DESCRIPTION OF THE DRAWINGS It is a schematic block diagram of the language analyzer which shows Example 1 of this invention. It is a flowchart which shows the existing structure analysis process of FIG. It is a flowchart which shows the parameter estimation process of FIG. It is a schematic block diagram of the language analyzer which shows Example 2 of this invention. It is a flowchart which shows the dependence structure analysis process of FIG. It is a flowchart which shows the parameter estimation process of FIG. It is a schematic block diagram of the language analyzer which shows Example 3 of this invention. It is a flowchart which shows the parameter estimation process of FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10,10A Analysis part 11 Input part 12 Unlabeled dynamic sample generation part 12A Labeled dynamic sample generation part 13 Solution search part 14 Dependency relation label determination part 15 Output part 20, 20A, 20B Model storage part 21 No label information Sentence unit model parameter storage unit 21A Sentence unit model parameter storage unit with label information 22 Sentence unit model parameter storage unit without label information 22A Sentence model parameter storage unit with label information 23 Model parameter storage unit for labeling 30, 30A, 30B Parameters Estimator 31 Parsed corpus storage unit 32 Clause unit model parameter estimation unit without label information 32A Clause unit model parameter estimation unit with label information 33 Parameter unlabeled sample generation unit 33A Parameter estimation label Sample generation unit with label 34 sentence unit model parameter estimation unit without label information 34A sentence unit model parameter estimation unit with label information 35 Model parameter estimation unit for labeling

Claims (12)

A language analysis method for performing language analysis on a computer using an input sentence divided into words or phrases,
A phrase unit model parameter estimation procedure for obtaining a phrase unit model parameter in a phrase unit probability distribution based on the parsed corpus stored in the storage means;
A parameter generation sample generation procedure for generating a sample of a dependency structure tree for each sentence in the corpus from the phrase unit probability distribution based on the phrase unit model parameter and the corpus;
A sentence unit model parameter estimation procedure for obtaining a sentence unit model parameter in a sentence unit probability distribution based on the sample and the corpus;
An input procedure for receiving the input sentence;
Dynamically generating a sentence-dependent dependency tree sample by Gibbs sampling based on a probability distribution of sentence-dependent dependency structures defined by the phrase-unit model parameters and the sentence-unit model parameters in the input sentence Sample generation procedure,
A solution search procedure seeking scores by marginal probabilities, to determine the optimal dependency structure using Eiser method or Chu-Liu-Edmonds method against that were generated said sample,
How language analysis characterized in that it comprises a. The language analysis method according to claim 1, further comprising:
In the case where the determined dependency structure is an unlabeled dependency structure, a dependency model for identifying a dependency relationship label is used for the unlabeled dependency structure using a probability model defined by a model parameter for labeling. The relationship label determination procedure
A language analysis method comprising: A language analysis method for performing language analysis on a computer using an input sentence divided into words or phrases,
Based on the parsed corpus stored in the storage means, the phrase unit model parameter estimation procedure with label information for obtaining the phrase unit model parameter with label information in the phrase unit probability distribution,
A parameter estimation sample generation procedure for generating a labeled dependent structure tree sample for each sentence in the corpus from the phrase unit probability parameter distribution based on the phrase information-added phrase unit model parameter and the corpus;
A sentence unit model parameter estimation procedure for obtaining a sentence unit model parameter with label information in a probability distribution of a sentence unit based on the sample and the corpus;
An input procedure for receiving the input sentence;
Generates a dependency structure tree sample for each sentence using Gibbs sampling based on the establishment distribution of the dependency structure for each sentence unit defined by the phrase unit model parameter with label information and the sentence unit model parameter with label information A labeled dynamic sample generation procedure,
From the generated sample, a solution search procedure for determining an optimal dependency structure using a maximum spanning tree search method ;
How language analysis characterized in that it comprises a. A language analysis method for performing language analysis on a computer using an input sentence divided into words or phrases,
A phrase unit model parameter estimation procedure for obtaining an unlabeled phrase unit model parameter in a phrase unit probability distribution based on the parsed corpus stored in the storage means;
A labeling model parameter estimation procedure for obtaining a labeling model parameter using the corpus;
Based on the phrase information-free phrase unit model parameter, the corpus, and the labeling model parameter, a labeled dependency structure tree sample for each sentence in the corpus is generated from the phrase unit probability distribution. Sample generation procedure with labeled parameters for parameter estimation,
A sentence unit model parameter estimation procedure for obtaining a sentence unit model parameter with label information based on the sample and the corpus;
An input procedure for receiving the input sentence;
Generates a sample dependency structure tree for each sentence using Gibbs sampling based on the establishment distribution of the dependency structure for each sentence unit defined by the phrase unit model parameter without label information and the sentence unit model parameter with label information A labeled dynamic sample generation procedure,
From the generated sample, a solution search procedure for determining an optimal dependency structure using a maximum spanning tree search method;
How language analysis characterized in that it comprises a. A language analysis device that performs language analysis on an input sentence divided into words or phrases,
A phrase unit model parameter estimator for obtaining a phrase unit model parameter in a phrase unit probability distribution based on a parsed corpus stored in the storage means;
A parameter estimation sample generation unit that generates a sample of a dependency structure tree for each sentence in the corpus from the phrase unit probability distribution based on the phrase unit model parameter and the corpus;
A sentence unit model parameter estimation unit for obtaining a sentence unit model parameter in a sentence unit probability distribution based on the sample and the corpus;
An input unit for receiving the input sentence;
Dynamically generating a sentence-dependent dependency tree sample by Gibbs sampling based on a probability distribution of sentence-dependent dependency structures defined by the phrase-unit model parameters and the sentence-unit model parameters in the input sentence A sample generator;
Determine the score by marginal probabilities against that were generated the sample, the solution search unit to determine an optimal dependency structure using Eiser method or Chu-Liu-Edmonds method,
A language analyzer characterized by comprising: The language analysis device according to claim 5 further includes:
A first storage unit for storing the phrase unit model parameters;
A second storage for storing the sentence unit model parameters;
A language analyzer characterized by comprising: The language analysis apparatus according to claim 5 or 6, further comprising:
In the case where the determined dependency structure is an unlabeled dependency structure, a dependency model for identifying a dependency relationship label is used for the unlabeled dependency structure using a probability model defined by a model parameter for labeling. A language analysis apparatus comprising a relation label determination unit. The language analysis device according to claim 7, further comprising:
A language analyzing apparatus comprising a third storage unit for storing the labeling model parameter. A language analysis device that performs language analysis on an input sentence divided into words or phrases,
Based on the parsed corpus stored in the storage means, the phrase unit model parameter estimation unit with label information for obtaining the phrase unit model parameter with label information in the phrase unit probability distribution,
A parameter estimation sample generation unit that generates a labeled dependency structure tree sample for each sentence in the corpus from the phrase-unit probability distribution based on the label information-added phrase unit model parameter and the corpus;
Based on the sample and the corpus, a sentence unit model parameter estimation unit for obtaining a sentence unit model parameter with label information in a sentence unit probability distribution;
An input unit for receiving the input sentence;
In the input sentence, a sample dependency structure tree of sentence units by Gibbs sampling based on the establishment distribution of the dependency structure of sentence units defined by the phrase unit model parameters with label information and the sentence unit model parameters with label information and the labeled dynamic sample generation unit that generates,
From that were generated the sample, the solution search means for determining the optimum dependency structure using the maximum spanning tree search technique,
A language analyzer characterized by comprising: The language analysis apparatus according to claim 9 further includes:
A first storage unit for storing the phrase unit model parameter with label information;
A second storage for storing the sentence unit model parameter with label information;
A language analyzer characterized by comprising: A language analysis device that performs language analysis on an input sentence divided into words or phrases,
A phrase unit model parameter estimation unit for obtaining an unlabeled phrase unit model parameter in a phrase unit probability distribution based on a parsed corpus stored in a storage unit;
A labeling model parameter estimating unit for obtaining a labeling model parameter using the corpus;
A parameter for generating a sample of a dependency structure tree with a label for each sentence in the corpus from a phrase unit probability distribution based on the phrase information-free phrase unit model parameter, the corpus, and the labeling model parameter A sample generator with a label for estimation;
Based on the sample and the corpus, a sentence unit model parameter estimation unit for obtaining a sentence unit model parameter with label information in a sentence unit probability distribution;
An input unit for receiving the input sentence;
In the input sentence, based on the probability distribution of the dependency structure of the sentence unit defined by the phrase unit model parameter without the label information and the sentence unit model parameter with the label information , the dependency structure tree of the sentence unit is obtained by Gibbs sampling. A labeled dynamic sample generator for generating samples;
A solution search unit for determining an optimal dependency structure from the generated sample using a maximum spanning tree search method;
A language analyzer characterized by comprising: In the language analysis apparatus according to claim 11, further,
A first storage unit for storing the phrase unit model parameter without label information;
A second storage unit for storing the labeling model parameters;
A third storage unit for storing the sentence unit model parameter with label information;
A language analyzer characterized by comprising:

Images