JP2009265889A - Language processor and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a language processor capable of precisely extracting only a pair of nouns having a mutual correlation with high possibility, and capable of extracting also a relation between the paired nouns. <P>SOLUTION: This language processor includes a processing-objective word pair feature extracting part for selecting a pair of words included in one sentence as a processing-objective word pair, and for extracting an appearance frequency feature of the processing-objective word pair, based on an input text data, a co-occurrence word feature extracting part for selecting a co-occurrence word, and for extracting an appearance frequency feature of the co-occurrence word, a syntax structure feature extracting part for extracting syntax structure in the sentence of the processing-objective word pair and the co-occurrence word, and for extracting an appearance frequency feature of the syntax structure, and a machine learning processing part for calculating the conditional probability of the processing-objective word pair, the conditional probability of the co-occurrence word, and the conditional probability of the syntax structure, using the obtained appearance frequency feature data, by machine learning processing, so as to be written as a leaning result data into a leaning result data storage part. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to natural language processing. In particular, the present invention relates to a language processing apparatus for extracting information from text and a computer program thereof.

Conventionally, there is a method of using mutual information as a method of extracting related nouns appearing in the same sentence. The mutual information represents a dependence scale for two random variables, and can be used for a word to measure how much the word depends on another word. Non-Patent Document 1 describes mutual information.
Kenji Kita, “Language and Computation 4 Stochastic Language Model”, University of Tokyo Press, p. 11, 1999

  However, when trying to extract a pair of related nouns by using mutual information, the syntax structure when these two words appear is not considered. Therefore, there is a problem that words having a low appearance frequency may have a high mutual information amount because they appear accidentally in the same sentence even when the words have no relation to each other. In addition, when a pair of nouns is extracted using the mutual information amount, there is a problem that it is impossible to grasp the relationship between the two words.

  The present invention has been made based on the above problem recognition, and accurately extracts only pairs of nouns that are highly likely to be related to each other, and also extracts the relationship between the two nouns forming the pair. An object of the present invention is to provide a language processing apparatus and a computer program for the same.

  [1] In order to solve the above problem, a language processing apparatus according to an aspect of the present invention selects a pair of words included in one sentence as a processing target word pair from input text data including a plurality of sentences. And a processing target word pair feature extraction unit (11) for extracting a predetermined feature of the appearance frequency of the processing target word pair in the input text data, and the processing target word pair in the input text data. A co-occurrence word feature extraction unit (co-occurrence noun feature extraction unit) that selects another word appearing in a sentence as a co-occurrence word and extracts a predetermined feature of the appearance frequency of the co-occurrence word in the input text data 12), a syntax structure of a sentence including the processing target word pair and the co-occurrence word in the input text data is extracted, and a predetermined characteristic of the appearance frequency of the syntax structure in the input text data is extracted. A class in which the co-occurrence word represents the relationship of the processing target word pair by referring to processing target concept related word data in which a word related to the processing target concept is stored in advance. On the basis of information on the sentence in the input text data that can be determined to belong to, an appearance frequency characteristic of the processing target word pair, an appearance frequency characteristic of the co-occurrence word, and an appearance frequency characteristic of the syntax structure Assuming that the word belongs to a class that represents the relationship of the processing target word pair, and that the conditional probability that the processing target word pair appears, and that the sentence belongs to the class that represents the relationship of the processing target word pair When the conditional probability that the co-occurrence word appears, and the sentence is assumed to belong to a class representing the relationship of the processing target word pair, the syntax structure is The conditional probability that the current to a characterized by comprising a machine learning processing unit that performs a process of writing the learning result data storage unit (3) as the learning result data (14).

According to this configuration, the processing target word pair feature extraction unit extracts appearance frequency features for processing target word pairs included in the sentence. The co-occurrence word feature extraction unit extracts appearance frequency features for the co-occurrence word. The syntax structure feature extraction unit extracts the syntax structure in the sentence of the processing target word pair and the co-occurrence word and extracts the appearance frequency feature of the syntax structure. When the input text data includes a large amount of sentences, the extracted appearance frequency feature values represent features that are statistically valid as a language. The machine learning processing unit refers to the processing target concept related word data in which the word related to the processing target concept is stored in advance, and specifically, the word corresponding to the processing target concept related word data is included in the sentence. By determining whether or not there is a sentence, a sentence that can be determined that the co-occurrence word belongs to a class representing the relationship of the processing target word pairs is extracted from a plurality of given sentences. These extracted sentences may be sentences in which it is clear that the co-occurrence words belong to a class representing the relationship between the processing target word pairs. Such a sentence acts as a correct sample in the machine learning process. Based on this correct sample, for example, by machine learning processing using an EM algorithm or the like, statistically, for the entire sentence included in the input text data including other than the correct sample, the processing target word pair of the class A conditional probability, a conditional probability of a co-occurrence word for the class, and a conditional probability of a syntax structure for the class are obtained (learning result data). The class (C ₁ ) in this case is based on the proposition that the co-occurrence word represents the relationship of the word pair to be processed, but another class (C ₀ , the co-occurrence word is the object to be processed) of the complement of this class. The respective probabilities for classes that do not represent word pair relationships are also obtained by subtracting the respective conditional probabilities from the overall probability (1). The obtained learning result data can be used to calculate the probability that the processing target word pair belongs to the class, the probability that the co-occurrence word belongs to the class, and the probability that the syntax structure belongs to the class.
That is, it can be determined whether another word (co-occurrence word) appearing in the same sentence indicates a relation name of two nouns to be processed. As a result, the relationship between words with low appearance frequency can be estimated with high accuracy.
A typical example of a word to be processed by this language processing apparatus is a noun. At this time, the processing target word pair is a processing target noun pair. A typical example of a co-occurrence word is a co-occurrence noun.

  In this language processing apparatus, processing target word pairs may be selected by limiting to predetermined processing target attributes. By performing such a limitation, irrelevant processing target word pairs are not included in the candidates, and the reliability of the calculated appearance frequency feature is improved.

  [2] Further, according to one aspect of the present invention, in the language processing device, the syntax structure feature extraction unit includes the first word included in the processing target word pair and the first word based on the syntax analysis result of the sentence. A common destination clause between the second word and the co-occurrence word included in the processing target word pair is extracted, a syntax structure from the first word to the common destination clause, and the common from the second word The syntax structure of the sentence is identified by a combination of the syntax structure up to the dependency destination clause and the syntax structure that modifies the common dependency clause.

  With this configuration, a syntax structure particularly suitable for the statistical processing of the present invention is obtained. As a result, the accuracy of word extraction is improved.

  [3] Further, according to one aspect of the present invention, in the language processing apparatus, the syntax structure feature extraction unit is a word that appears in a list of words representing the syntax structure, and the first word is the above word A plurality of syntax structures in which the ratio of words that are neither the second word nor the co-occurrence word is equal to or greater than a predetermined threshold is defined as a syntax structure group having a similar syntax structure, and appearance frequency characteristics of the syntax structure group Is extracted as an appearance frequency feature of the syntax structure.

  With this configuration, the appearance frequency feature can be extracted for a syntax structure group in which sentences having similar syntax structures are collected. As a result, it is possible to absorb slight fluctuations in terms of words and notations used in sentences and to extract appearance frequency features of statistically stable syntax structures. As a result, even when the number of sentences included in the input text data is relatively small, it is possible to extract words with high accuracy.

  [4] Moreover, one aspect of the present invention is based on the premise that in the language processing apparatus, the processing target word pair appears in a sentence using the learning result data read from the learning result data storage unit. The conditional probability that the sentence belongs to the class, the conditional probability that the sentence belongs to the class assuming that the co-occurrence word appears in the sentence, and the syntax structure appearing in the sentence A probability value calculation processing unit for calculating a conditional probability that a sentence belongs to the class is further provided.

  With this configuration, it is possible to calculate the probability that the processing target word pair belongs to the class, the probability that the co-occurrence word belongs to the class, and the probability that the syntax structure belongs to the class. That is, for example, by using a threshold for probability as appropriate, it is possible to determine whether a sentence and a processing target word pair, a co-occurrence word, or a syntax structure included in the sentence belong to the class.

  [5] In the language processing device according to an aspect of the present invention, the processing target word pair appears in a sentence using the learning result data written in the learning result data storage unit by the language processing device. Assuming that the sentence belongs to the class, the conditional probability that the sentence belongs to the class on the assumption that the co-occurrence word appears in the sentence, and the syntax structure appears in the sentence And a probability value calculation processing unit for calculating a conditional probability that the sentence belongs to the class.

With this configuration, a sentence, a processing target word pair, a co-occurrence word, and a syntax structure included in the class belong to the class by using learning result data that is a result of machine learning processing performed in advance using input text data. It can be determined whether or not.
Also, another sentence that is not included in the original input text data used in the machine learning process can be determined.

  [6] According to another aspect of the present invention, a pair of words included in one sentence is selected as a processing target word pair from input text data including a plurality of sentences, and the processing in the input text data is performed. A processing target word pair feature extraction process for extracting a predetermined feature of the appearance frequency of the target word pair, and another word appearing in a sentence including the processing target word pair in the input text data Selecting a predetermined feature of the appearance frequency of the co-occurrence word in the input text data, the processing target word pair and the co-occurrence word in the input text data, A syntactic structure feature extracting process for extracting a predetermined feature of the appearance frequency of the syntax structure in the input text data, and a word related to the processing target concept. The sentence information in the input text data that can be determined that the co-occurrence word belongs to the class representing the relationship of the processing target word pair by referring to the processing target concept related word data, and the appearance of the processing target word pair Based on the frequency feature, the appearance frequency feature of the co-occurrence word, and the appearance frequency feature of the syntax structure, the processing target word when it is assumed that a sentence belongs to a class representing the relationship of the processing target word pair A conditional probability that a pair appears, and a conditional probability that the co-occurrence word appears when a sentence belongs to a class that represents the relationship between the processing target word pairs, and a sentence that relates the processing target word pair The conditional probability that the syntax structure appears when it is assumed that it belongs to the class that represents, is written into the learning result data storage unit as learning result data It is a program for executing processing performs the processing of the machine learning process on the computer.

  According to the present invention, it is possible to extract a word representing a related word pair and a related name included in the text. In addition, the relationship between words with low appearance frequency can be estimated with high accuracy. It can also extract the features of the syntactic structure when constructing the relationship. Such a technique is also useful in the field of text understanding by machines, and can be expected to be applied in fields such as information analysis in which a large amount of text is analyzed by machines to extract important information.

[First Embodiment]
Next, an embodiment of the present invention will be described with reference to the drawings.
FIG. 1 is a block diagram illustrating a functional configuration of the language processing apparatus according to the present embodiment. In this figure, reference numeral 1 denotes a language processing device. As illustrated, the language processing apparatus 1 includes an input text storage unit 2, a learning result data storage unit 3, output data 4, a processing target word pair feature extraction unit 11, and a co-occurrence noun feature extraction unit 12 (co-occurrence). Word feature extraction unit), syntactic structure feature extraction unit 13, machine learning processing unit 14, and probability value calculation processing unit 15.

  The input text storage unit 2 stores input text data to be processed. This input text data contains a large amount of sentences.

  The processing target word pair feature extraction unit 11 determines a processing target attribute in advance and sets a noun pair belonging to the attribute as the processing target word pair. Features such as the number of appearances of this processing target word pair are extracted from the input text. In other words, the processing target word pair feature extraction unit 11 selects a pair of words included in one sentence as processing target word pairs from the input text data including a plurality of sentences, and processes the processing target in the input text data. Predetermined features of the appearance frequency of word pairs are extracted.

  The co-occurrence noun feature extraction unit 12 generates two nouns (noun pairs) belonging to the processing target attribute appearing in one sentence, with respect to other nouns appearing in the same sentence that are candidates for the relationship. Extract features such as frequency from the input text. In other words, the co-occurrence noun feature extraction unit 12 selects another word appearing in the sentence including the processing target word pair in the input text data as the co-occurrence word, and the co-occurrence word in the input text data. A predetermined feature of the appearance frequency of is extracted.

  The syntax structure feature extraction unit 13 extracts a syntax structure feature between a processing target word pair and a co-occurrence noun. Specifically, the syntax structure feature extraction unit 13 includes features such as the number of appearances of the syntax structure between three phrases of two nouns belonging to the processing target attribute appearing in one sentence and other nouns appearing in the same sentence. Is extracted from the input text. In other words, the syntax structure feature extraction unit 13 extracts the syntax structure of the sentence including the processing target word pair and the co-occurrence word in the input text data, and determines the appearance frequency of the syntax structure in the input text data. Extract features.

  The machine learning processing unit 14 receives the results of the processing target word pair feature extraction unit 11, the co-occurrence noun feature extraction unit 12, and the syntax structure feature extraction unit 13, and performs machine learning processing using an EM algorithm. Specifically, the machine learning processing unit 14 can determine that a co-occurrence word belongs to a class representing a relationship between processing target word pairs by referring to processing target concept related word data in which words related to the processing target concept are stored in advance. Based on the sentence information in the input text data, the appearance frequency feature of the processing target word pair, the appearance frequency feature of the co-occurrence word, and the appearance frequency feature of the syntax structure, the sentence indicates the relationship between the processing target word pair. Conditional probability that a processing word pair will appear when it is assumed that it belongs to the class to be represented, and conditional probability that a co-occurrence word will appear if it is assumed that the sentence belongs to a class that represents the relationship between the processing word pairs , And conditional probabilities that the syntax structure appears when the sentence is assumed to belong to a class that represents the relationship of the word pairs to be processed as learning result data It performs a process of writing the over data storage unit.

  The learning result data storage unit 3 is for storing data (probability value data) obtained as a result of the machine learning process.

The probability value calculation processing unit 15 reads out the learning result data that is the result of the machine learning processing unit 14 from the learning result data storage unit 3, and the probability that the processing target noun pairs are related to each other, and the processing target noun pair co-occurs. The probability that the noun is a word representing a relationship and the probability that the syntactic structure between the noun co-occurring with the processing target noun pair is a relationship is calculated and output. In other words, the probability value calculation processing unit 15 uses the learning result data, and on the assumption that a processing target word pair appears in the sentence, the probability that the sentence belongs to the class, and a co-occurrence word appears in the sentence. Assuming that the sentence belongs to a class, a conditional probability that the sentence belongs to the class is calculated on the assumption that a syntax structure appears in the sentence.
The output data 4 is data output by the probability value calculation processing unit 15.

  FIG. 2 is a flowchart showing an overall processing procedure of the language processing apparatus 1. Hereinafter, the processing flow of the entire language processing apparatus 1 will be described with reference to this flowchart.

  In this apparatus, first, in step S01, the processing target word pair feature extraction unit 11 determines a processing target attribute, and extracts a noun pair belonging to the attribute from the text read from the input text storage unit 2. Examples of attributes to be processed include attributes such as “animal”, “person”, “country”, “organization”, and the like. The processing target word pair feature extraction unit 11 determines a noun pair belonging to the given processing target attribute, and calculates an appearance frequency feature of the noun pair for the input text.

  Next, in step S02, the co-occurrence noun feature extraction unit 12 uses another noun that appears in the same sentence as the noun pair belonging to the processing target attribute determined above (this is a noun that is a relation candidate, but the processing target word pair Therefore, for convenience, the term “co-occurrence noun” is selected as a candidate for the relationship, and the appearance frequency feature for the input text of the co-occurrence noun is calculated.

  In step S03, the syntax structure feature extraction unit 13 determines between the noun pair belonging to the processing target attribute determined by the processing target word pair feature extraction unit 11 and the co-occurrence noun determined by the co-occurrence noun feature extraction unit 12. Extract the syntax structure and calculate the number of occurrences. Here, the syntactic feature is a dependency structure between clauses in a sentence. The syntax structure feature extraction unit 13 performs syntax analysis processing of the corresponding sentence using existing technology, and generates three lists representing the dependency structure of the sentence based on the obtained syntax tree data.

  First, a common connection clause (common connection clause) between the target noun pair and the co-occurrence noun determined by the co-occurrence noun feature extraction unit 12 is extracted. The above three lists are first lists representing the dependency structure from the first noun in the target noun pair to the above-mentioned common dependency clause. Second, it is a list representing the dependency structure from the second noun in the target noun pair to the common dependency destination clause. Thirdly, the list represents a structure that modifies the above-mentioned common relation clause in parts other than these. At this time, processing is performed including the original clause itself as the related clause.

  At this time, the syntax structure feature extraction unit 13 divides each clause into independent word parts such as nouns and verbs and auxiliary word parts such as particles. For example, in the case of processing a sentence “a golden eagle is a terrible natural enemy for a prairie dog”, “prairie dog” and “gold eagle” are noun pairs determined by the processing target word pair feature extraction unit 11, and “natural enemy” Are co-occurrence nouns determined by the co-occurrence noun feature extraction unit 12, the following three lists are extracted as a syntax structure.

1st list = Syntax structure from “Pralee Dog” to common “clause” sentence: “Noun 1”, 2nd list = syntax from “Eagles” to common “clause” sentence Structure: “Noun 2”, is the third list = “Neighbors” is a syntactic structure: Scary, NULL, “Noun 3”

  In this example, “noun 1” is “prairie dog”, “noun 2” is “dog eagle”, and “noun 3” is “natural enemy”. Note that the dependency structure from which the above three lists are extracted is “The eagle is a natural enemy”, “The prairie dog is a natural enemy”, “Horrible natural enemy”, etc. The structure can be obtained by a parsing process.

  Then, the syntax structure feature extraction unit 13 counts the appearance frequencies of those sets whose three lists are exactly the same.

However, instead of counting the appearance frequency of the three lists that are exactly the same, it is also possible to count the appearance frequencies of the sets whose mutual similarity is a predetermined value or more. The similarity used here can be determined, for example, by the ratio of common words other than nouns 1 to 3 appearing in the above three lists. For example, the ratio of common words with the structure taken from the sentence of the first sentence shown above, “The golden eagle is a terrible natural enemy for the prairie dog,” and the second sentence, “The golden eagle is a natural enemy for the prairie dog.” And can be calculated as follows: That is, the three lists obtained from the second sentence are as follows.
The first list = syntactic structure from "Pralee Dog" to "is a natural enemy": "Noun 1", and the second list = syntactic structure from "Inu Eagle" to "is a natural enemy": "Noun 2" is the third List structure = "Noun 3"
The common words between the three lists of the first sentence and the three lists of the second sentence are “for” and “ha”, and they appear in the first sentence and the second sentence. Therefore, the number of common words is 4. Further, the words that are not common are “Awesome” and “NULL”, and the number of words that are not common is two. Therefore, the similarity of these sentences is 4 / (4 + 2), that is, it can be calculated as 4/6.

  As described above, the syntax structure feature extraction unit 13 includes the noun 1 (the first word included in the processing target word pair), the noun 2 (the second word included in the processing target word pair), the co-occurrence word, The common relationship clause is taken out, the syntax structure from the first word to the common relationship clause, the syntax structure from the second word to the common relationship clause, and the other portions are the common relationships. The syntactic structure of the sentence is identified by a combination with the syntactic structure that modifies the previous clause.

  Further, when the syntax structure feature extraction unit 13 counts the appearance frequency of a pair whose similarity is equal to or higher than a predetermined value instead of counting the appearance frequency of the completely same syntax structure, the syntax structure feature The extraction unit 13 includes a plurality of syntax structures that are words that appear in the list of words representing the syntax structure and that have a common ratio of words that are not the noun 1, the noun 2, or the co-occurrence noun more than a predetermined threshold. Is a syntactic structure group having a similar syntactic structure, and the appearance frequency feature of this syntactic structure group is extracted as the appearance frequency feature of the syntactic structure.

Here, the appearance frequency features calculated by the processing target word pair feature extraction unit 11, the co-occurrence noun feature extraction unit 12, and the syntax structure feature extraction unit 13 will be described. Information on these appearance frequency features is used in the machine learning process by the machine learning processing unit 14 later.
And noun pairs belonging to the processing target attribute given appear in one sentence, and other nouns appearing in the statement, 3-tuple syntax structure between these three noun is expressed as t _i. In addition, a noun pair included in this triplet is CPt _i , another noun appearing in the same sentence and a noun that is a candidate for relation is RPt _i, and a syntactic structure between these three nouns is SPt _i . .

The processing target word pair feature extraction unit 11 counts the total number of appearances of noun pair types based on the extraction result. Further, the processing target word pairs feature extraction unit 11 obtains information on whether the noun pair CPt _i included in a certain 3-tuple t _i is included in the 3-tuple t _k.

The co-occurrence noun feature extraction unit 12 counts the total number of types of co-occurrence nouns based on the extraction result. Moreover, co-occurrence noun feature extraction unit 12 obtains information on whether the co-occurrence noun RPt _i included in a certain 3-tuple t _i is included in the 3-tuple t _k.

The syntax structure feature extraction unit 13 counts the total number of types of syntax structure based on the result of the above analysis. Also, syntactic structure feature extraction unit 13 acquires information on whether the syntactic structure SPt _i included in a certain 3-tuple t _i is included in the 3-tuple t _k.

The machine learning processing unit 14 performs learning processing using the output from the processing target word pair feature extraction unit 11, the co-occurrence noun feature extraction unit 12, and the syntax structure feature extraction unit 13 as input data.
First, in step S04, the machine learning processing unit 14 extracts sentences that can be determined to clearly represent a relationship from the input data. For example, when an animal is a concept to be processed, the co-occurrence noun obtained by the co-occurrence noun feature extraction unit 12 (the co-occurrence noun is a candidate representing the relationship between word pairs) is “weak”, “large” Extracts sentences that are synonyms or synonyms such as “favorite”, “favorite”, “natural enemy”, “enemy”, “friend”, “great enemy”, “special”, “kind”, “prey”, “eat” To do. This is because these are nouns that can be clearly judged to express the relationship with respect to the processing target concept of animals. The relationship between the processing target concept and the noun (word) to be extracted here is stored in a storage unit (not shown) as processing target concept related word data defined in advance. For example, conceptual dictionary data can be used as the target data. The machine learning processing unit 14 reads (refers to) the processing target concept related word data from the storage unit and compares them, so that the co-occurrence noun obtained by the co-occurrence noun feature extraction unit 12 has the processing target concept. Based on the determination, a sentence that can be determined to represent the relationship is extracted from the input data.

In the ternary set t _i , CPt _i , RPt _i , SPt _i constituting the ternary set represent c ₁ as a class (class), and c ₀ as a class not represented (class). . Those probabilities can be defined by equation (1) below.

In Expression (1), P (CPt _i | c _j ) is a probability that two noun pairs CPt _i included in t _i appear in class c _j . P (RPt _i | c _j ) is a probability that a noun (a co-occurring noun that appears in the same sentence) RPt _i included in t _i appears in class c _j . P (SPt _i | c _j ) is a probability that a syntactic structure SPt _i between three nouns included in t _i appears in the case of class c _j .

Next, in step S05, the machine learning processing unit 14 performs machine learning using an EM algorithm (Expectation-maximization algorithm) using this equation. In addition, although the procedure of the learning process using EM algorithm is shown next, it is described also in the following reference.
Reference: Kamel Nigam et al., “Text Classification from Labeled and Unlabeled Document using EM.”, Machine Learning, Vol.39, No.2 / 3, pp.103-134 (2000).

This machine learning process (the process in step S05) will be described with reference to another flowchart.
FIG. 3 is a flowchart showing a procedure of machine learning processing performed by the machine learning processing unit 14 using the EM algorithm.
First, in step S21, the machine learning processing unit 14 reads the text data to be processed from the input text storage unit 2, and the initial probability P (c _j | t _i ) of the class c _j to which t _i obtained from the text data belongs. Is calculated by the following equation (2). The class c _j is either c ₀ or c ₁ , and their definition is as described above.

In the calculation of the initial probability, the machine learning processing unit 14 belongs to the class c ₁ in the case of expressing the relationship for the ternary set t _i extracted from the sentence (described above) that is clearly determined to represent the relationship. Count the number as 1. For the ternary set t _i extracted from other sentences, the number of times belonging to the class c ₁ when expressing the relationship is counted as a predetermined value (for example, 0.5) of 0 or more and less than 1. This predetermined value is not limited to 0.5 and can be changed as appropriate. Further, when c ₁ is determined for a certain t _i of a sentence, c ₀ for the t _i of the sentence is determined by c ₀ = 1−c ₁ . Then, taking the count sum of for all sentences that t _i appears, with the results of the c ₀ and c ₁ count, calculates the numerator of equation (2).

  In addition, the process which calculates the initial probability of step S21 is E step of EM algorithm.

Next, in step S22, the machine learning processing unit 14 obtains the probability P (CPt _i | c _j ) of occurrence of the noun pair CPt _i under the class c _j using the equation (3) under the class c _j . The probability P (SPt _i | c _j ) of occurrence of the co-occurrence noun RPt _i is expressed by the equation (4), and the probability P (SPt _i | c of occurrence of the syntactic structure SPt _i between the three nouns under the class c _{j. j} ) is calculated by equation (5).

That is, equation (3) is an equation for calculating the conditional probability of the word pair to be processed when class c _j , and equation (4) calculates the conditional probability of the co-occurrence word when class c _j. Equation (5) is an equation for calculating the conditional probability of the syntax structure for class c _j .
In addition, the process which calculates each probability of step S22 is M step of EM algorithm.

In the above formulas (3), (4), and (5), | CP | represents the total number of noun pairs, | RP | represents the total number of nouns that are relation candidates, and | SP | This represents the total number of occurrences of the syntax structure, and | T | N (CPt _i | t _k ) is a function indicating whether or not a noun pair included in the ternary set t _i is included in the ternary set t _k . N _(RPt i | _{t k)} is a function indicating whether a noun which is a relationship candidates included in the 3-tuple _{t i} (co-occurrence noun) are included in the 3-tuple _{t k.} N _(SPt i | _{t k)} is a function indicating whether 3 noun syntax structure included in the three-tuple _{t i} is included in the 3-tuple _{t k.} Each of these functions indicating whether or not they are included returns 1 as a value when they are included, and returns 0 as a value when they are not included.

As represented by Equation (3), the first term of the denominator of the probability P (CPt _i | c _j ) is the total number of noun pairs. The second term in the denominator, the conditional probability of c _j which assumes t _k of if they contain noun pair CPt _m to 3-tuple t _k (conveniently referred to as Xc), all 3-tuple and Sum of all noun pairs. The first term of the numerator is the constant term (1). The second term of the numerator is the sum total of all three terms of the noun pair CPt _i of Xc.
Further, as expressed by equation (4), the first term of the denominator of the probability P (RPt _i | c _j ) is the total number of co-occurrence nouns. The second term in the denominator, the conditional probability of c _j which assumes t _k of if they contain co-occurrence noun RPt _m to 3-tuple t _k (conveniently referred to as Xr), all three tuple And it is the sum total about all the co-occurrence nouns. The first term of the numerator is the constant term (1). The second term of the numerator is the Xr, the sum of all the 3-tuple for that co-occurrence noun RPt _i.
Further, as expressed in equation (5), the first term of the denominator of the probability P (SPt _i | c _j ) is the total number of occurrences of the syntax structure. The second term of the denominator is all ternary groups of the conditional probabilities of c _j (referred to as Xs for convenience) assuming t _k when the ternary set t _k contains the syntax structure SPt _m and The sum of all syntax structures. The first term of the numerator is the constant term (1). The second term of the numerator is the sum of all three terms of the syntactic structure SPt _i of Xs.

Next, in step S23, the machine learning processing unit 14 calculates the probabilities P (CPt _i | c _j ) and P (RPt _i | c _j ) calculated by the equations (3), (4), and (5), respectively. And the value of P (SPt _i | c _j ), the expected value of P (c _j | t _i ) is calculated by the following equation (6).

In step S _< b _> 24, the machine learning processing unit 14 calculates the value of P (c _j ) according to the following equation (7) using the result of the equation (6).

In Expression (7), | c | indicates the number of classes to be classified. Here, there are two classes c ₀ and c ₁ , and | c | is 2.

  In step S25, the machine learning processing unit 14 determines the convergence condition, and if not converged, returns to step S22 (step S25: NO), and if converged, the entire learning process shown in this flowchart is performed. The process ends (step S25: YES).

Specifically, the determination of the convergence condition is based on the fact that the amount of change ΔP (c _j ) from the previous calculation of the value of P (c _j ) calculated in step S24 is a predetermined threshold (for example, 1.0 × It is carried out depending on whether it is less than 10 ⁻³ ). That is, if the change amount ΔP (c _j ) is equal to or greater than the threshold (step S25: NO), the process returns to step S22, and new P (c _j ) and P (c _j | t _{i are} again performed according to the procedure of this flowchart. ) Is used to calculate the values of P (CPt _i | c _j ), P (RPt _i | c _j ), and P (SPt _i | c _j ) (step S22), and the change ΔP in step S25. The processes in steps S22 to S25 are repeated until (c _j ) becomes a value smaller than the threshold value. In step S25, if the change amount ΔP of _{P (c j) (c j} ) is smaller than the threshold (step S25: YES), and ends the whole learning process shown in this flowchart.

Returning to the flowchart of FIG. 2, in step S06, the machine learning processing unit 14 determines the probability values P (c _j | t _i ), P (CPt _i | c _j ), P last calculated in the above processing procedure. (RPt _i | c _j ) and P (SPt _i | c _j ) are written in the learning result data storage unit 3.

In step S07, the probability value calculation processing unit 15 outputs P (c _j | t _i ), P (CPt _i | c _j ), P output from the machine learning processing unit 14 and written in the learning result data storage unit 3. (RPt _i | c _j ) and P (SPt _i | c _j ) are read, and the probability value is calculated based on these values. The probability value calculation processing unit 15 calculates the probability P (c _j | CPt _i ) that the processing target noun pair is related to and the probability P (c _j | that the noun co-occurring with the processing target noun pair is related. RPt _i ) and the probability P (c _j | SPt _i ) that is a structure in which the syntactic structure between the noun pair that co-occurs with the noun pair to be processed indicates a relationship, and these are respectively the expressions (8) and (9 ) And (10).

The probability value calculation processing unit 15 outputs these calculated values as output data 4. The probability value P (c _j | CPt _i ) indicates the degree of whether or not the noun pair CPt _i is related. The probability value P (c _j | RPt _i ) indicates the degree of whether or not the noun RPt _i co-occurring with the noun pair represents a relationship. The probability value P (c _j | SPt _i ) indicates a degree of whether or not the syntax structure SPt _i between the processing target noun pair and the co-occurring noun is a structure indicating a relationship. The determination can be made based on these output data.

Note that the probability value calculation processing unit 15 determines whether the probability values P (c _j | CPt _i ), P (c _j | RPt _i ), and P (c _j | SPt _i ) are equal to or greater than a predetermined threshold value. And the determination result may be output.

<Example of processing results>
Next, the results of performing the above-described series of processing for extracting the relationship from the text on the actual data will be described. Here, the processing target attribute (target concept) is “animal”, and the processing target data is closed caption data of a television program related to animals produced and broadcast by the Japan Broadcasting Corporation (NHK).

FIG. 4 is a schematic diagram showing a list of extracted noun pairs CPt _i and probability values P (c ₀ | CPt _i ) related thereto. The data in this figure shows the data output by the probability value calculation processing unit 15 sorted in ascending order of P (c ₀ | CPt _i ). Since P (c ₁ | CPt _i ) = 1−P (c ₀ | CPt _i ), in this figure, there is a higher possibility (degree) of having a relationship in the noun pairs listed above. For example, P (c ₀ | CPt _i ) for a noun pair in which “noun 1” is “dolphin” and “noun 2” is “bora” is 0.031. For example, P (c ₀ | CPt _i ) for a noun pair in which “noun 1” is “salmon” and “noun 2” is “brown bear” is 0.044. Further, for example, P (c ₀ | CPt _i ) for a noun pair in which “noun 1” is “snow owl” and “noun 2” is “lemming” is 0.044. The same applies to the following.

FIG. 5 is a schematic diagram illustrating the extracted co-occurrence noun RPt _i and the value of the probability value P (c ₀ | RPt _i ) related thereto. The data in this figure shows the data output by the probability value calculation processing unit 15 sorted in ascending order of P (c ₀ | RPt _i ). Since P (c ₁ | RPt _i ) = 1−P (c ₀ | RPt _i ), in this figure, the possibility (degree) of a noun representing a relationship is higher as the co-occurrence noun listed above. It is expensive. For example, P (c ₀ | RPt _i ) is 0.011 when “noun 3” is “companion”. For example, when “noun 3” is “eat”, P (c ₀ | RPt _i ) is 0.012. The same applies to the following.

FIG. 6 is a schematic diagram showing the syntax structure SPt _i between the noun pair to be processed and the co-occurring noun and the value of the probability value P (c ₀ | SPt _i ) related thereto. The data in this figure shows the data output by the probability value calculation processing unit 15 sorted in ascending order of P (c ₀ | SPt _i ). Since P (c ₁ | SPt _i ) = 1−P (c ₀ | SPt _i ), in this figure, the more likely the syntax structure listed above is, the possibility (degree) ) Is high.

  The notation of the syntax structure in this figure will be described. As symbols appearing in the notation, “NP1” represents noun 1, “NP2” represents noun 2, and “REL” represents a relationship noun. The syntax structure notation pattern is as follows. In other words, the common clauses of noun 1, noun 2, and co-occurrence words are taken out, and the syntactic structure from noun 1 to the common clause is common to the syntactic structure from noun 2 to the common clause. Three structures of the syntax structure that modifies the clause at the destination are shown separated by a separator character “=”. This first pattern is used when a candidate noun appears after noun 1 and noun 2.

For example, the data on the first line of this figure shows that the syntax structure from noun 1 to relation candidate noun is “NP1, ha”, and the syntax structure from noun 2 to relation candidate noun is “NP2, a”. , Corresponding to a syntax structure in which the relationship candidate noun is modified as “REL”, and P (c ₀ | SPt _i ) at that time is 0.034. The same applies to other rows of data.

  Note that the language processing apparatus 1 according to the present embodiment extracts a common connection destination of the noun 1 and the noun 2 in the sentence, and a syntax structure from the noun 1 to the common connection destination or a syntax structure from the noun 2 to the common connection destination. Only the case where the related candidate noun is included is processed. A case where no relation candidate noun is included in the syntax structure from the noun 1 to the common relation destination or the syntax structure from the noun 2 to the common relation destination or when the relation candidate noun precedes the noun 1 is excluded from the processing target.

  The data of the processing result examples shown in these drawings can be determined to be an appropriate result. In other words, the proper ones are listed at the top of the processing results, such as word pairs, co-occurrence nouns representing relationship names, and syntactic structures representing relationships. That is, it was confirmed that the language processing apparatus 1 according to the present embodiment is effective.

  In addition, you may make it implement | achieve all or one part function of the language processing apparatus in embodiment mentioned above with a computer. In that case, a program for realizing these functions may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read into a computer system and executed. Here, the “computer system” includes an OS and hardware such as peripheral devices. The “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. Further, the “computer-readable recording medium” dynamically holds a program for a short time, like a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line. It is also possible to include those that hold a program for a certain time, such as a volatile memory inside a computer system serving as a server or client in that case. The program may be a program for realizing a part of the functions described above, and may be a program capable of realizing the functions described above in combination with a program already recorded in a computer system.

Although a plurality of embodiments have been described above, the present invention can also be implemented in the following modifications.
In the above embodiment, the input text storage unit 2, the learning result data storage unit 3, the output data 4, the processing target word pair feature extraction unit 11, the co-occurrence noun feature extraction unit 12, and the syntax structure feature extraction unit 13, the machine learning processing unit 14, and the probability value calculation processing unit 15 are all configured as one unit. For example, the processing target word pair feature extraction unit 11 and the co-occurrence noun feature extraction unit 12, a syntactic structure feature extraction unit 13, a machine learning processing unit 14, a machine for performing machine learning processing, and a probability value calculation processing unit 15. You may divide and comprise into the apparatus which performs the part of a process (judgment process). At this time, the learning result data is configured to be transferred via a storage unit shared by both apparatuses or via a communication line. When the apparatus is divided in this way, the part up to the machine learning process and the part of the probability value calculation process can be performed separately. It is also possible to perform machine learning processing in advance and perform repeated probability value calculation processing using the learning result data obtained as a result. In addition, if the sentence is in a field similar to the input text (a sentence in which the learning result data is valid), the probability value calculation process can be performed on a sentence that is not included in the original input text.

  The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes designs and the like that do not depart from the gist of the present invention.

  The present invention can be used for automatic extraction of information from a large amount of text, knowledge acquisition, and the like.

It is the block diagram which showed the function structure of the language processing apparatus by embodiment of this invention. It is the flowchart which showed the whole process sequence of the language processing apparatus by the embodiment. It is the flowchart which showed the procedure of the machine learning process using EM algorithm among the processes of the language processing apparatus by the embodiment. FIG. 6 is a schematic diagram showing the result of processing according to the embodiment and showing the higher probability that the processing target noun pair is related in ascending order of P (c ₀ | CPt _i ). FIG. 6 is a schematic diagram showing the result of processing according to the embodiment, in which the highest probability that a noun co-occurring with a noun represents a relationship is in ascending order of P (c ₀ | RPt _i ). It is the data of the processing result according to the embodiment, and shows the upper order of the probability that the syntax structure between the noun pair to be processed and the co-occurring noun indicates the relationship in ascending order of P (c ₀ | SPt _i ). FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Language processing apparatus 2 Input text memory | storage part 3 Learning result data memory | storage part 4 Output data 11 Processing object word pair feature extraction part 12 Co-occurrence noun feature extraction part (co-occurrence word feature extraction part)
13 Syntax structure feature extraction unit 14 Machine learning processing unit 15 Probability value calculation processing unit

Claims (6)

A pair of words included in one sentence is selected as a processing target word pair from input text data including a plurality of sentences, and a predetermined feature of the appearance frequency of the processing target word pair in the input text data is extracted. A processing target word pair feature extraction unit,
Selecting another word appearing in a sentence including the processing target word pair in the input text data as a co-occurrence word, and a predetermined characteristic of the appearance frequency of the co-occurrence word in the input text data A co-occurrence word feature extraction unit to extract;
A syntax for extracting a syntax structure of a sentence including the processing target word pair and the co-occurrence word in the input text data, and extracting a predetermined feature of an appearance frequency of the syntax structure in the input text data A structural feature extraction unit;
The sentence in the input text data that can be determined that the co-occurrence word belongs to a class representing the relation of the processing target word pair by referring to processing target concept related word data in which a word related to the processing target concept is stored in advance. Based on the information, the appearance frequency feature of the processing target word pair, the appearance frequency feature of the co-occurrence word, and the appearance frequency feature of the syntax structure, a machine learning process is performed, and a sentence of the processing target word pair The conditional probability that the processing target word pair appears when assuming that it belongs to a class representing a relationship, and the co-occurrence word when assuming that a sentence belongs to a class representing the relationship of the processing target word pair A conditional probability of appearance, and a conditional probability of occurrence of the syntax structure on the assumption that the sentence belongs to a class representing the relationship of the processing target word pairs. And machine learning processing unit that performs a process of writing the learning result data storage unit as the learning result data,
A language processing apparatus comprising: The language processing apparatus according to claim 1,
The syntactic structure feature extraction unit is configured to share a first word included in the processing target word pair, a second word included in the processing target word pair, and the co-occurrence word based on a syntax analysis result of the sentence. A syntactic structure from the first word to the common dependency clause, a syntactic structure from the second word to the common dependency clause, and a syntactic structure that modifies the common dependency clause Identify the syntactic structure of the sentence by combining with
A language processing apparatus. The language processing device according to claim 2,
The syntactic structure feature extraction unit has a predetermined ratio of words that appear in a list of words representing the syntactic structure and are not common to the first word, the second word, and the co-occurrence word. A plurality of syntactic structures that are equal to or greater than the threshold value of a syntactic structure group having a similar syntactic structure, and an appearance frequency feature of the syntax structure group is extracted as an appearance frequency feature of the syntax structure.
A language processing apparatus. In the language processing device according to any one of claims 1 to 3,
Using the learning result data read from the learning result data storage unit, a conditional probability that the sentence belongs to the class on the assumption that the processing target word pair appears in the sentence, and the co-occurrence word in the sentence A probability value calculation processing unit that calculates a conditional probability that the sentence belongs to the class on the assumption that it appears and a conditional probability that the sentence belongs to the class on the assumption that the syntax structure appears in the sentence; ,
A language processing apparatus, further comprising:   Assuming that the processing target word pair appears in a sentence using the learning result data written in the learning result data storage unit by the language processing device according to claim 1. The conditional probability that the sentence belongs to the class, the conditional probability that the sentence belongs to the class assuming that the co-occurrence word appears in the sentence, and the syntax structure appearing in the sentence A language processing apparatus comprising a probability value calculation processing unit for calculating a conditional probability that a sentence belongs to the class. A pair of words included in one sentence is selected as a processing target word pair from input text data including a plurality of sentences, and a predetermined feature of the appearance frequency of the processing target word pair in the input text data is extracted. Processing target word pair feature extraction process,
Selecting another word appearing in a sentence including the processing target word pair in the input text data as a co-occurrence word, and a predetermined characteristic of the appearance frequency of the co-occurrence word in the input text data A co-occurrence word feature extraction process to be extracted;
A syntax for extracting a syntax structure of a sentence including the processing target word pair and the co-occurrence word in the input text data, and extracting a predetermined feature of an appearance frequency of the syntax structure in the input text data Structural feature extraction process;
The sentence in the input text data that can be determined that the co-occurrence word belongs to a class representing the relation of the processing target word pair by referring to processing target concept related word data in which a word related to the processing target concept is stored in advance. Based on the information, the appearance frequency feature of the processing target word pair, the appearance frequency feature of the co-occurrence word, and the appearance frequency feature of the syntax structure, the sentence belongs to the class representing the relationship of the processing target word pair The conditional probability that the processing target word pair appears when it is assumed, and the conditional probability that the co-occurrence word appears when a sentence belongs to a class that represents the relationship of the processing target word pair, And a conditional probability that the syntax structure appears when it is assumed that the sentence belongs to a class representing the relationship between the processing target word pairs, and learning result data A machine learning process to perform the process for writing the learning result data storage unit Te,
A program that causes a computer to execute this process.

Images