JP2010250814A - Part-of-speech tagging system, training device and method of part-of-speech tagging model - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a part-of-speech tagging system, which enables part-of-speech tagging for a large tagging set and enhances part-of-speech tagging accuracy. <P>SOLUTION: The part-of-speech tagging system 1 comprises a part-of-speech tagging model training device 12 which trains a part-of-speech tagging model for each node and each hierarchy based on a part-of-speech hierarchy tree 15 by using a text with a first tag in the part-of-speech tag training set 10, and a part-of-speech tagging device 22 which implements part-of-speech tagging for the text to be tagged by using a trained part-of-speech tagging model 13. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to the field of natural language processing, and in particular, to a part-of-speech tagging system and a part-of-speech tagging model training apparatus and method.

  The number of natural language texts that can be processed by computers has increased significantly with the spread of the Internet and the development of an information society. Correspondingly, the demand for applications that handle large amounts of information such as text extraction, information extraction, information processing between languages, and man-machine dialogue is also increasing rapidly. Natural language processing technology is one of the main technologies that address the above demands. “Part-of-speech tagging” is to tag the correct part-of-speech for each word in the text, which is the basis for natural language processing. The result of part-of-speech tagging typically directly affects higher level processing (word frequency statistical analysis, syntax, chunks, semantic analysis, etc.) of natural language processing. Therefore, it is extremely important to realize a method and system for tagging parts of speech with high efficiency and high accuracy.

  In natural language processing, part-of-speech tagging is one of the sequence tagging problems. Until now, conditional random fields (CRF) have been widely used as a way to address the sequence tagging problem in natural language processing. CRF is essentially a type of indirect graph model for calculating conditional possibilities. Conditional possibilities are used to specify the value of the output node when given the value of the input node. CRF can express characteristics of elements such as long distance dependency and overlap, and can be used in information extraction having strong global relevance. Using CRF can effectively avoid strong correlation assumptions in directional graph models such as Maximum Entropy (ME) and Hidden Markov Models (HMM), so tagging offsets that occur in the directional graph model The problem can be solved. This makes CRF one of the best statistical learning models for sequence tagging problems. In order to obtain an effective part-of-speech tagging model, it is necessary to introduce a large number of features and train using a large set of tags. However, CRF training processing requires enormous time and computational resources, and the time and computational resources required for training increase geometrically as the number of tags increases. Therefore, the CRF model is rarely applied to large-scale system applications (for example, part-of-speech tagging systems) that handle large tag sets. The CRF model is primarily applied to applications that use a small number of features and a small training corpus. Given the relatively high accuracy required for part-of-speech tagging, finding a method for applying the CRF model to part-of-speech tagging applications that use large tag sets and feature corpora is an urgent task.

  Several solutions have already been proposed to address the above problems. For example, Non-Patent Document 1 (Cohn T, Smith A, Osborne M. Scaling conditional random fields using error-correcting codes (conditional random field scaling using error correction codes). In Proc. In Computational Linguistics (ACL'05), Ann Arbor, Michigan: Association for Computational Linguistics, June 2005, pp. 10-17.), A method of applying CRF to a large tag set is proposed. In this non-patent document 1, an ECOC (error correction output code for dealing with the problem of CRF training in a large tag set. This is an encoding process that defines a redundancy determination function and a final determination based on the determination function. This is an ensemble method composed of a decoding process for constructing a classification function). Details of this method are shown below.

Model training phase (encoding phase)
1) Assume that a tag set has m labels (for example, a noun is NN, a verb is VB, an adjective is JJ, and an adverb is RB), and an ECOC of length n is manually selected. Here, the correction code is used to map each label to an n-bit vector, as shown in the following example.

  Then, using the above encoding, the original tagging problem (also called “multi-classification problem”) is converted into n independent binary classification problems. In this case, one column encoding corresponds to one binary classifier. Taking the third classifier selected by the black box as an example, this classifier distinguishes words tagged as “NN” and “JJ” from words tagged as “VB” and “RB”. Used to do.

  2) Build a training corpus for these binary classifiers (this corpus can be built by modifying the original corpus, which simply means tagging labels in the training corpus For example, if you are building a corpus for the third classifier, all you need is to replace all “NN” and “JJ” in the original corpus with “1” and all Simply replace “VB” and “RB” with “0”). Once the corrected corpus is obtained, the binary classifier is trained using conventional CRF training methods.

上記の表に示されるように、１つの語に対応して１つのｎビットベクトルがある。一部の従来方式では、各ベクトルを表３内の符号化ベクトルと１つずつ比較し、一致するラベルを検出してタグ付けに使用することが可能である。例えば、「ｄｅｖｅｌｏｐｓ（開発する）」という語の場合、それに対応するｎビットベクトルは「ＶＢ」の符号化に最も近いので、「ＶＢ−ｖｅｒｂ」としてタグ付けする。 Model usage phase (decryption phase)
1) Give one sentence (eg, “NEC develops world-leading technology to present IP phone spam” (NEC develops world-leading IP phone spam prevention technology)).
2) Tag each sentence using each binary classifier trained above and record the results. The result should be:

As shown in the table above, there is one n-bit vector corresponding to one word. In some conventional schemes, each vector can be compared with the encoded vector in Table 3 one by one, and the matching label can be detected and used for tagging. For example, in the case of the word “develops”, the corresponding n-bit vector is closest to the encoding of “VB” and is therefore tagged as “VB-verb”.

At present, the above method is still far from true application in the following points because the known technology cannot efficiently cope with the problem of applying CRF to part-of-speech tagging with a large tag set. .
1) Although the performance of the method of Non-Patent Document 1 largely depends on the selection of ECOC encoding, it is difficult to select an ideal ECOC.
2) This method essentially does not solve the problem that training takes too much time and depends heavily on expensive computational resources. In the training phase, training of n binary classifiers is required, but the value of n depends on the choice of ECOC. In part-of-speech tagging, the value of n is generally quite large, so it still takes a long training time and depends on expensive computational resources. Further, in the decoding phase, it is necessary to use all the binary classifiers one by one, and the coding matching process is extremely complicated. Therefore, the use of a trained model takes a long time, and expensive computational resources are indispensable.

Cohn T, Smith A, Osborne M .; Scaling conditional random fields using error-correcting codes (conditional random field scaling using error correction codes). In Proc. the 43rd Annual Meeting of the Association for Computational Linguistics (ACL'05), Ann Arbor, Michigan: Association for Computational Linguistics, June. 10-17.

  The present invention solves the problem of applying conventional CRF to part-of-speech tagging with large tag sets by introducing techniques for classifying and combining CRFs cascaded with part-of-speech hierarchies. The present invention automatically analyzes internal relationships between different parts of speech from the training set, builds a part of speech hierarchy tree based on the internal relationships, and organizes all parts of speech. Based on this part-of-speech hierarchy tree, the present invention introduces a cascaded CRF model for reducing the number of tags in each hierarchy, and specifies the introduction relationship between individual models. Finally, the present invention can automatically train and obtain a cascaded CRF part-of-speech tagging model, even for large tag sets. In view of the problem that the training set is potentially scarce, in addition to the above, the present invention trains a part-of-speech inference model based on word construction rules for unknown words to further improve the accuracy of part-of-speech tagging.

  According to a first aspect of the present invention, there is a part-of-speech tagging system based on a part-of-speech hierarchy tree and using the first tagged text in the part-of-speech tag training set, for each part-of-speech tag and for each node. Part-of-speech tagging comprising: a part-of-speech tagging model training device for training a tagging model; and a part-of-speech tagging device for tagging the part of speech of the text to be tagged using the trained part-of-speech tagging model. A system is provided.

  According to a second aspect of the present invention, there is a part-of-speech tagging method based on a part-of-speech hierarchy tree and using the first tagged text in the part-of-speech tag training set for each part-of-speech tag and for each node. Part-of-speech tagging, comprising: a part-of-speech tagging model training step for training a tagging model; and a part-of-speech tagging step for tagging the part of speech of the text to be tagged using the trained part-of-speech tagging model. A method is provided.

  According to a third aspect of the present invention, there is provided a training apparatus for a part-of-speech tagging model, wherein the first tagged text in the part-of-speech tag training set is converted into a second text for each hierarchy based on the part-of-speech tag tree. A CRF model training corpus construction unit that builds a CRF model training corpus by tagging by node and a second text tagged by the CRF model training corpus construction unit to obtain a part-of-speech tagging model Is provided with a CRF model training unit that trains individual CRF models layer by layer and node by node.

  According to a fourth aspect of the present invention, there is provided a method for training a part-of-speech tagging model, wherein a first tagged text in a part-of-speech tag training set is classified into a second text for each hierarchy based on a part-of-speech tag tree. Build CRF model training corpus by tagging by node, CRF model training corpus construction step, and second text tagged by CRF model training corpus construction unit to obtain part-of-speech tagging model And a CRF model training step for training individual CRF models layer by layer and node by node.

The present invention essentially solves the problem of applying CRF to part-of-speech tagging with large tag sets as follows.
1) The present invention solves the problem that the CRF model can be applied to part-of-speech tagging with a large tag set and takes a long time for training and is highly dependent on expensive computational resources. The method and system of the present invention makes it possible to train a part-of-speech tagging model on any ordinary PC computer regardless of model.
2) Part-of-speech tagging accuracy improves for the following reasons. That is, (i) sequential part-of-speech tagging is a task with high global relevance, but global optimization can be efficiently performed by introducing a CRF model, so that part-of-speech tagging accuracy is improved. In addition, (ii) by introducing a part-of-speech estimation mechanism for unknown words based on word construction rules, it is possible to deal with the problem that training set is scarce, and the accuracy of overall part-of-speech tagging is improved.
3) Since the method of the present invention is fully automated, labor costs for training and optimizing part-of-speech tagging models can be significantly reduced.

1 is a schematic diagram of a part of speech tagging system according to a first embodiment of the present invention; 3 is a flowchart of a part-of-speech tagging method according to the first embodiment of the present invention; 1 is a schematic diagram of a part-of-speech hierarchy tree construction device according to the present invention. It is a flowchart of the method of part hierarchy tree construction method by this invention. It is a structural example of a part of speech hierarchy tree. It is an example of the data structure of a part of speech hierarchy tree. It is an example of the data structure of a part of speech hierarchy tree. 1 is a schematic block diagram of a part-of-speech tagging model training device according to the present invention. 3 is a flowchart of a part-of-speech tagging model training method according to the present invention. 1 is a schematic diagram of a part-of-speech tagging device according to the present invention. 3 is a flowchart of a part-of-speech tagging method according to the present invention. It is the schematic of the part-of-speech tagging system by the 2nd Example of this invention. It is a flowchart of the part-of-speech tagging method by the 2nd Example of this invention. It is the schematic of the part-of-speech tagging system by the 3rd Example of this invention. It is a flowchart of the part of speech tagging method by 3rd Example of this invention.

  Next, preferred embodiments of the present invention will be described with reference to the drawings. In addition, when the same reference symbol or number is used in different drawings, it indicates the same or similar component. In the following, detailed descriptions of known functions and configurations are omitted to avoid obscuring the subject matter of the present invention.

  FIG. 1 is a schematic diagram of a part of speech tagging system according to a first embodiment of the present invention. In the part-of-speech tagging system 1, the part-of-speech tag training set 10 is composed of a large number of tagged texts (ie, tagged text sets). The part-of-speech hierarchy tree construction device 14 analyzes the relation between different parts of speech based on the tagged text in the part-of-speech tag training set 10, and constructs the part-of-speech hierarchy tree 15 based on the analyzed relation. Tagged part-of-speech that exists in the tag training set 10 is organized in a hierarchy. Here, the relationship may be a similarity between parts of speech, for example. The part-of-speech tagging model training device 12 performs training to generate a part-of-speech tagging model 13. The part-of-speech tagging model training device 12 reads the tagged text from the part-of-speech tag training set 10, and trains the CRF part-of-speech tagging model 13 for part-of-speech tagging based on information about the part-of-speech hierarchy in the part-of-speech hierarchy tree 15. Build a training process. The generated part of speech tagging model is a cascaded part of speech tagging model. The part-of-speech tagging device 22 performs part-of-speech tagging for words in any untagged text based on the generated part-of-speech tagging model.

  Although the part-of-speech tagging system including the part-of-speech hierarchy tree construction device 14 is shown in FIG. In this case, the part of speech is tagged to the untagged text using a pre-built part of speech hierarchy tree. The part of speech hierarchy tree may be, for example, a manually constructed hierarchy tree. Furthermore, the part-of-speech tagging system may include only a part-of-speech tagging model training device 12 that generates a part-of-speech tagging model 13 for part-of-speech tagging.

  The part of speech hierarchy tree 15 organizes parts of speech as a tree structure in a hierarchical manner. FIG. 4a is an example of a part of speech hierarchy tree. This part-of-speech hierarchy tree has four hierarchies of hierarchies 0, 1, 2, and 3, and hierarchies 2 and 3 each have six nodes. The leaf nodes of the part-of-speech hierarchy tree correspond to true part-of-speech, and the other nodes are randomly defined dummy class names. 4b and 4c are examples of the data structure of the part of speech hierarchy tree shown in FIG. 4a.

  FIG. 1b is a flowchart of the part-of-speech tagging method. In S101, the part-of-speech hierarchy tree construction device 14 constructs the part-of-speech hierarchy tree 15 and organizes the tagged parts of speech existing in the part-of-speech tag training set into a hierarchy. In S102, the part-of-speech tagging model training device 12 reads the tagged text from the part-of-speech tag training set 10, and generates a part-of-speech tagging model 13 for part-of-speech tagging based on information about the part-of-speech hierarchy in the part-of-speech hierarchy tree 15. . This part of speech tagging model is a cascaded part of speech tagging model. In S103, the part-of-speech tagging device 22 performs part-of-speech tagging on the input text using the generated part-of-speech tagging model 13.

  Next, the construction of the part-of-speech hierarchy tree 15 will be described with reference to FIGS.

  FIG. 2 is a schematic diagram of the part-of-speech hierarchy tree construction device 14 of the present invention. In this figure, the part of speech feature template selection unit 140 selects a part of speech feature template that expresses the grammatical behavior of the part of speech. The grammatical behavior of parts of speech can be expressed in various ways. Examples of features that can be selected as part-of-speech feature templates include: the word immediately before the current word in the tagged text, the part of speech of the previous word, the word immediately after the current word, and the part of speech of the immediately following word. The thing containing is mentioned. The feature vector construction unit 141 constructs a feature vector for each part of speech that exists in the part of speech tag training set 10 based on the selected part of speech feature template. The similarity calculation unit 142 calculates a similarity between any two parts of speech included in the part of speech tag training set 10 using the constructed feature vector. Clustering unit 143 clusters all of the parts of speech in part-of-speech tag training set 10 based on the calculated similarity, using a conventional hierarchical clustering algorithm, and creates a part-of-speech hierarchy tree 15 according to preset rules. Generate.

FIG. 3 is a flowchart of a process in which the part-of-speech hierarchy tree construction device generates a part-of-speech hierarchy tree. In S301, the part of speech feature template selection unit 140 selects a part of speech feature group as a part of speech feature template. At this time, for example, the word immediately before the current word in the tagged text, the part of speech of the previous word, the next word of the current word, and the part of speech of the next word are selected. For example, in the tagged text “Hong Kong / ns review / v 10 / m large / a outstanding / a youth / n”, if the word “review” is selected as the current word, the part of speech of the current word is “V”. The part-of-speech feature group in this case is expressed as follows.

In S302, the feature vector construction unit 141 constructs a feature vector for each part of speech in the part of speech tag training set 10 based on the part of speech feature template. For example, there are a total of dz words and lz parts of speech in the part of speech tag training set. Unit 141, given the part of speech feature group selected above, can construct the following vector for any part of speech x.
1) x <previous word> immediately preceding word vector—this vector has a dimension of dz, and the element corresponding to that vector represents the frequency with which the specified word appears immediately before the word of part of speech x.
2) x <previous word's POS> The part of speech vector of the immediately preceding word—this vector has a dimension of lz, and the element corresponding to the vector is the frequency at which the specified part of speech appears immediately before the word of part of speech x Represents.
3) x <next word> Next word vector—This vector has a dimension of dz, and the element corresponding to the vector represents the frequency with which the specified word appears immediately after the word of part of speech x.
4) x <next word's POS> Part-of-speech vector of next word—This vector has a dimension of lz, and the element corresponding to the vector is the frequency at which the specified part-of-speech appears immediately after the part-of-speech x word. Represents.

In S <b> 303, the similarity calculation unit 142 calculates the similarity between any two parts of speech included in the part of speech tag training set 10 by performing the following steps, for example, for x <b> 1 and x <b> 2.
1) First, for the feature vectors of two parts of speech (x1, x2), the following similarity between each pair is calculated.
Simc (x1 <previous word>, x2 <previous word>)
Simc (x1 <previous word's POS>, x2 <previous word's POS>)
Simc (x1 <next word>, x2 <next word>)
Simc (x1 <next word's POS>, x2 <next word's POS>)
2) Calculate the overall similarity using the following formula:
Sim (x1, x2) = w1 * Simc (x1 <previous word>, x2 <previous word>) +
w2 * Simc (x1 <previous word's POS>, x2 <previous word's POS>) +
w3 * Simc (x1 <next word>, x2 <next word>) +
w4 * Simc (x1 <next word's POS>, x2 <next word's POS>),
Here, w1 + w2 + w3 + w4 = 1.

  In S304, the clustering unit 143 uses a hierarchical clustering algorithm (eg, K-means algorithm) to cluster all parts of speech based on the calculated similarity and based on pre-set rules. Generate a hierarchical tree. In the present invention, the rule set in advance may include a definition such as “the number of nodes in each layer is less than n (n is a positive integer)”. In this case, for example, n can be set to 8.

  In the following, the generation of a part-of-speech tagging model will be described with reference to FIGS. 5a and 5b. FIG. 5a is a block diagram of the part-of-speech tagging model training device 12 according to the present invention. The part-of-speech tagging model training apparatus 12 includes a CRF model training corpus construction unit 121, a CRF model training unit 122, and a logic circuit 120. The CRF model training corpus construction unit 121 performs part-of-speech tagging on the training text read from the part-of-speech tag training set 10 for each hierarchy and each node based on the part-of-speech hierarchy tree 15. The CRF model training unit 122 trains the CRF model for each corresponding hierarchy and node based on the training text tagged by the CRF model training corpus construction unit 121. The logic circuit 120 controls the CRF model training corpus construction unit 121 and the CRF model training unit in the part-of-speech tagging model training process. The logic circuit 120 holds the number of layers of the part-of-speech hierarchy tree, and increments the number of layers every time the CRF model training corpus construction unit 121 and the CRF model training unit process one hierarchy, This continues until all nodes in the hierarchy have been processed.

FIG. 5b is a flowchart of a process in which the part-of-speech tagging model training device generates a part-of-speech tagging model. This is a nested training method including a two-layer loop, and a training mode from top to bottom is employed. Therefore, the training result of one layer affects the next layer, and the training of the same layer can be executed independently. Part of speech hierarchical tree is constituted by n hierarchy, the hierarchy i have m _i-number of nodes, and that the current node is referred to as "j". First, the logic circuit 120 assigns a value of “0” to the hierarchy i in S601, and assigns a value of “1” to the node j in S602. Subsequently, in S603, the CRF model training corpus construction unit 121 constructs a training corpus for the <i, j> CRF model, and the part-of-speech tagging labels included in the tagged text in the original part-of-speech tag training set 10 are: Replace each subnode name of the current node corresponding to that label in the part of speech hierarchy tree. In S604, the CRF model training unit 122 trains the <i, j> CRF model using the <i, j> CRF model training corpus and the selected feature template. Here, when i = 0, the feature template selected by the CRF model training unit 122 includes two words before and after, the characters before and after the current word, and the co-occurrence frequency between the two words before and after. When i> 0, in addition to the feature template used at level 0, the part-of-speech of the two words before and after the tagging result of the previous level, the co-occurrence between parts of speech, and between the word and part-of-speech Feature templates including co-occurrence are also used at the same time. S605 the value of j is incremented at, in S606, j whether greater than _{m i} is determined. If j is _{m i} is smaller than, the process proceeds to S603. If j is greater than _{m i,} the value of i is incremented in S607, the process proceeds to S602, the processing to the node of all layers of the part of speech hierarchical tree has finished S603 and S604 is continued. In this way, cascaded part-of-speech tagging models can be trained even in large tag sets.

Here is an example of the following sentence that has been fully tagged:
Hong Kong / ns Evaluation / v Ten / m Large / a Outstanding / a Youth / n
At level 0, a <0, 1> CRF model training corpus is constructed. First, the sentence is re-tagged. Referring to the part-of-speech hierarchy tree shown in FIG. 4a, the subnodes of node 1 in hierarchy 0 are “label1”, “label2”, “label3”, and “label4”. The true part-of-speech “v” in FIG. 4a corresponds to the sub-node “label1” in the first hierarchy of the part-of-speech hierarchy tree. Thus, all words tagged with “v” in the original training set are retagged as “label1”.

The sentence re-tagged in hierarchy 0 is as follows:
Hong Kong / label3 review / label1 ten / label2 large / label1 standout / label1 youth / label3
In Tier 0, the CRF model is trained. The selected feature template includes two words before and after “Hong Kong” and “evaluation”, characters before and after the current word, and co-occurrence between the two words before and after (where “co-occurrence” "Means a situation in which two words appear simultaneously in a context).

  Thereafter, the above sentence is re-tagged again in hierarchy 1. For the first node <1,1> in layer 1, a <1,1> CRF model training corpus is constructed. Referring to the part-of-speech hierarchy tree of FIG. 4a, the node <1,1> has sub-nodes “label11” and “label12”. Therefore, the word tagged with “label1” in hierarchy 0 is further tagged with “label11, label12” (ie, a set of sub-node names of the current node).

  “Hong Kong / label3 reputation / label1 10 / label2 large / label1 prominent / label1 youth / label3”, which was the result of tagging at level 0, is “Hong Kong / label3 rating” after re-tagging node <1,1>. Out / label12 + 10 / label2 large / label11 Outstanding / label11 youth / label3 ”.

  Thereafter, the CRF model training is executed for the node <1, 1>. The selected feature template includes, in addition to the above-described feature template at level 0, two parts of speech before and after included in the tagging result of the immediately preceding layer, co-occurrence between parts of speech, and co-occurrence between words and parts of speech. including. For example, in the case of the word “evaluation”, the feature template includes “label3” and “label2” which are parts of speech of two words “Hong Kong” and “ten” before and after, the co-occurrence between these parts of speech, Including co-occurrence between parts of speech.

  Similarly, the above-described CRF model training corpus construction process and CRF model training process are executed for each of the nodes <1, 2>, <1, 3>, and <1, 4>, and the nodes of all layers This is continued until the CRF model training corpus construction process and the CRF model training process are completed.

  FIG. 6a is a block diagram of a part-of-speech tagging device. The part of speech tagging device 22 includes a logic circuit 222, a CRF model feature construction unit 220, and a CRF part of speech tagging unit 221. The logic circuit 222 controls the CRF model feature construction unit 220 and the CRF part-of-speech tagging unit 221 according to the cascaded part-of-speech tagging model during execution of the part-of-speech tagging process. Under the control of the logic circuit 222, the CRF model feature construction unit 220 constructs a feature group for using the <i, j> CRF model for each layer and each node for text to be tagged. The CRF part-of-speech tagging unit 221 executes part-of-speech tagging for each corresponding hierarchy and node based on the feature data constructed by the feature construction unit 220 under the control of the logic circuit 222.

FIG. 6b is a flowchart of the cascaded CRF part-of-speech tagging method performed by the part-of-speech tagging apparatus. There are a total of n hierarchy part-of-speech tagging model, the hierarchy i have m _i-number of nodes, and that the current node is referred to as "j". First, in S901, the logic circuit 222 assigns a value “0” to the hierarchy i, and assigns a value “1” to the node j in S902. In step S903, the CRF model feature construction unit 220 constructs feature data for using the <i, j> CRF model. The CRF model feature construction unit 220 constructs input feature data for the CRF model based on the feature template set in the part-of-speech model training process. The following two methods can be used for different levels i.
1) When i is “0”, a process of capturing information in the feature template for the CRF model is executed. That is, the related feature information is extracted directly from the text input as the tagging target, and the information is taken into the template, thereby generating the input feature data of the CRF model.
2) When i is other than “0”, in addition to extracting feature information at layer 0, feature information is obtained from the tagging result of the text to be tagged using the CRF model at layer i-1. Is extracted to generate input feature data of the CRF model.

  In S904, the text is tagged using the <i, j> CRF model of the part-of-speech tag model 10 based on the generated feature data.

S905 the value of j is incremented at, in S906, j whether greater than _{m i} is determined. If j is _{m i} is smaller than, the process proceeds to S903. If j is greater than _{m i,} the value of i is incremented in S907, the process proceeds to S908 and S902, the processing until the nodes of all hierarchies of word class hierarchy tree has finished S903 and S904 is continued. In this way, by performing part-of-speech tagging on text for each hierarchy, part-of-speech tagging with a large tag set is realized.

  In the following, in order to deepen the understanding of the entire tagging process, a simple example will be taken up and described.

Suppose that the text to be tagged is “Beijing Engage 10 Dai Yicheng City”.
Tier 0 (using <0,1> CRF model)
The tagging result is “Beijing / label3 envelopment / label1 ten / label2 large / label1 Yui / label1 castle city / label3”.
Layer 1 (uses all CRF models for this layer)
1. Using the <1,1> CRF model, a result of “Beijing / label3 envelopment / label12 ten / label2 large / label11 Yii / label11 castle city / label3” is obtained.
2. The same applies to the following using the <1,2> CRF model.
......
The tagging result after processing for level 1 is “Beijing / label32 envelopment / label12 + 10 / label21 large / label11 Yui / label11 castle city / label31”.
Tier 2
1. By using the <2, 1> CRF model, a result of “Beijing / label32 envelopment / label12 ten / label21 large / a Yui / a Castle City / label31” is obtained.
2. The same applies hereinafter using the <2, 2> CRF model.
The complete tagging result finally obtained is “Beijing / ns envelopment / v 10 / m large / a Yui / a castle city / n”.

  FIG. 7a is a schematic block diagram of a part of speech tagging system according to a second embodiment of the present invention. This part-of-speech tagging system further includes an evaluation device 16, an adjustment device 17, and a test set construction device 18 in addition to the components of the part-of-speech tagging system of FIG. The test set construction device 18 randomly selects a part-of-speech tagging text set from the part-of-speech tag training set 10 as a test set for the text set to be tagged. The evaluation device 16 evaluates the tagging result after the test set is subjected to the part of speech tagging process using the part of speech tagging model. In this case, the evaluation device 16 evaluates the tagging accuracy based on the result of the trial. The adjustment device 17 adjusts the part-of-speech hierarchy tree construction device 14 based on the evaluation result of the evaluation device in order to construct a higher-performance part-of-speech hierarchy tree.

  FIG. 7b is a flowchart of the part of speech tagging process performed by the part of speech tagging system. As shown in FIG. 7b, in S701, the test set construction device 18 randomly extracts a subset of the part-of-speech tag training set 10 as a test set. In S702, the part-of-speech tagging system uses the trained part-of-speech tagging model 13 to perform part-of-speech tagging processing on the test set. In step S <b> 703, the evaluation device 16 evaluates the accuracy of the test set tagged with the part of speech and passes the evaluation result to the adjustment device 17. Thereafter, in S704, the adjustment device 17 determines the performance of the part-of-speech tagging model based on the evaluation result. If the performance of the part-of-speech tagging model does not satisfy the predetermined condition, the clustering result is obtained in S705. To adjust the thresholds of w1, w2, w3, and w4 used by the part-of-speech hierarchy tree construction device. In S706, the adjustment device adjusts the clustering result using a heuristic rule (heuristic rule). In this case, for example, a rule defining that “n and ns are classified into different clusters” is used.

  FIG. 8a is a block diagram of a part of speech tagging system according to a third embodiment of the present invention. In the case of an unknown word, since there is usually no training data for the word in the training corpus, the tagging accuracy is relatively low, and the overall tagging accuracy is reduced due to the influence. Since the part-of-speech tagging system of the present invention can correct the part-of-speech of an unknown word, the overall part-of-speech tagging accuracy of the system is improved. This part-of-speech tagging system further includes an unknown word part-of-speech inference model construction device 19 and an unknown word part-of-speech correction device 21 in addition to the components of the part-of-speech tagging system of FIG. The unknown word part-of-speech estimation model construction device 19 learns word construction rules from the existing part-of-speech tag training set 10, and constructs an unknown word part-of-speech estimation model 20 according to the learned word construction rules. The unknown word part-of-speech correction device 21 corrects the part of speech of the unknown word in the text tagged with the part-of-speech tagging model 13 using the unknown word part-of-speech estimation model 20.

  FIG. 8b shows a part-of-speech tagging method according to a third embodiment of the present invention. As shown in FIG. 8b, in S801, the unknown word part-of-speech inference model construction device 19 first executes direct constituent segmentation processing on the words in the part-of-speech tag training set, and analyzes the attributes of the direct constituents after the division. (Ie, identify the direct constituents of each word in the part-of-speech tag training set and tag the attributes of the direct constituents) to obtain a sequence of word constituents.

  Here, the definition of the direct constituent will be briefly described. Small units that make up large units are referred to as large unit components. Therefore, a small unit that directly constitutes a large unit is called a “direct constituent”. Each word in the part-of-speech tag training set is a word itself, not a component smaller than a word. Therefore, the analysis of “direct constituents” and direct constituent attributes here is different from general word division and part-of-speech tagging. The analysis of “direct constituents” and direct constituent attributes here means that a word consisting of two or more characters in the part-of-speech tagging training set is divided into units immediately below. For example, in the case of a single word composed of two characters, the unit immediately below means individual characters (morphemes) constituting the word. In the case of a word composed of three or more characters, the word is divided into one word (maximum match) existing in the dictionary and one morpheme. In the case of the term “science and technology department”, it is assumed that two words “science” and “technology” exist in the dictionary, and “science and technology” and “technical department” do not exist. / Parts ". If “Science”, “Technology Department”, and “Technology” exist in the dictionary, the division result is “Science / Technology Department”. Thus, a direct constituent can be a word or a morpheme. The direct constituent attribute mainly means a syntactic attribute expressed in a part-of-speech tag format. Direct constituent attributes can include all possible part-of-speech tags.

  Table 3 shows the results of direct constituent segmentation and attribute analysis for the two words “cold violence, scavenging”.

The sequence that can be obtained from the above is as follows.
Cold violence → Cold 2 a N_B Violence 4 n N_E
Sweep → Sweep 2 v V_B Shoot 2 v V_E.
If “cold radiation” is an unknown word, the sequence of word components obtained is “cold 2 a radiation 2 v”.

  In S802, the unknown word part-of-speech estimation model construction device 19 selects a part-of-speech feature template.

In S803, the unknown word part-of-speech inference model construction device 19 converts the sequence of word components obtained using the selected part-of-speech feature template, and uses any known machine learning algorithm to know the unknown word inference model 20 Is generated. For example, when the unknown word estimation model 20 is used, the following can be acquired as the part of speech of “cold shooting”.
POS (cold 2 a V_B, shot 2 v V_E) = V.

  In S804, the part-of-speech tagging system uses the generated unknown word guessing model 20 to re-tag the unknown words in the text tagged with the part-of-speech tagging model 13.

It is assumed that the sequence of word components is “sweep 2 v V_B shot 2 v V_E” and the following feature template is selected.
// Part of speech U01 of constituent word:% x [−1, 2] // Second feature of previous one constituent element (/) (“/” represents a null feature)
U02:% x [0,2] // second feature of current constituent (a)
// Constituent word length U03:% x [1,1] // First feature of the next one constituent element (2, 2)
// constituent word U04:% x [0, 0] // zero feature of one current constituent,
The sequence of word components is converted into the following input data for any machine learning method such as CRF.
if (T (-1,2) = '/') tag = 'V_B'
if (T (0,2) = 'v') tag = 'V_B'
if (T (1,1) = '2') tag = 'V_B'
if (T (0,0) = 'sweep') tag = 'V_B'

if (T (-1,2) = 'v') tag = 'V_E'
if (T (0,2) = 'v') tag = 'V_E'
if (T (1,1) = '2') tag = 'V_E'
if (T (0,0) = 'shooting') tag = 'V_E'

  Unknown words in the final text tagged with the part-of-speech tagging model 13 are re-tagged using the generated unknown word guessing model 20, but using the generated unknown word guessing model 20, It is also possible to retag unknown words in the text tagged with the part of speech tagging model 13 in the current hierarchy. In other words, the part-of-speech tagging result of the current hierarchy can be corrected and used as feature data for the next hierarchy.

  In the above, Chinese text was used as an example to describe an embodiment of the present invention, but it is clear that the present invention can be applied to part-of-speech tagging in any language such as English or Japanese.

  The foregoing description is only illustrative of preferred embodiments of the invention and is not intended to limit the invention. Those skilled in the art will understand that these embodiments may be arbitrarily modified and replaced without departing from the scope and spirit of the invention as defined by the appended claims. Let's go.

  Further, a part or all of the above-described embodiment can be described as in the following supplementary notes, but is not limited thereto.

(Appendix 1)
A part-of-speech tagging system,
A part-of-speech tagging model training device that trains the part-of-speech tagging model hierarchically and nodely using the first tagged text in the part-of-speech tag training set based on the part-of-speech hierarchy tree;
A part-of-speech tagging system comprising: a part-of-speech tagging device that tags a part of speech of a text to be tagged using a trained part-of-speech tagging model.

(Appendix 2)
The part-of-speech tagging model training device comprises:
A CRF that builds a CRF model training corpus by tagging the first tagged text in the part-of-speech tag training set against the second tagged text hierarchically and nodely based on the part-of-speech hierarchical tree. A model training corpus building unit,
A CRF model training unit that trains the CRF model in a corresponding hierarchical and nodal manner to obtain a part-of-speech tagging model by using a second tagged text tagged by the CRF model training corpus building unit; The part-of-speech tagging system according to supplementary note 1, comprising:

(Appendix 3)
The CRF model training corpus building unit replaces the tagged part of speech of the first tagged text with the name of the subnode of the node corresponding to the position of the tagged part of speech in the part of speech hierarchical tree. The part-of-speech tagging system according to appendix 2, wherein tagging is executed node by node.

(Appendix 4)
The CRF model training unit is
(A) The current level is “0”, and the feature template includes two words before and after the second text, characters before and after the current word, and the co-occurrence frequency between the two words before and after. (B) The current level is not “0”, and the feature template is a feature template selected at level 0, two words before and after the second text in the immediately preceding hierarchy, and the co-occurrence frequency between parts of speech The part-of-speech tagging system according to appendix 3, wherein the CRF model is trained hierarchically and nodely by selecting a feature template according to a case where co-occurrence frequencies between words and parts of speech are included.

(Appendix 5)
The part of speech tagging device comprises:
A CRF model feature construction unit that constructs feature data hierarchically and nodely to apply the CRF model to the text to be tagged;
The CRF part-of-speech tagging unit that tags part-of-speech of text to be tagged hierarchically and nodely according to the feature data constructed by the CRF model feature construction unit. Part-of-speech tagging system.

(Appendix 6)
The CRF model feature construction unit is:
(A) The current level is 0, and feature data is used to input to the feature template selected at level 0 during CRF model training and extracted from the text to be tagged (b) For the CRF model, if the current level is not 0, the feature data extracted at level 0 is used, and the feature data is extracted from the second text tagged with the previous level CRF model The part-of-speech tagging system according to appendix 5, wherein feature data is constructed.

(Appendix 7)
The part-of-speech tagging system according to appendix 1, further comprising a part-of-speech hierarchy tree construction device that analyzes a relationship between part-of-speech of tagged text in the part-of-speech tag training set.

(Appendix 8)
The part of speech hierarchy tree construction device
A part-of-speech feature template selection unit that selects a feature template that represents a part-of-speech feature;
A feature vector construction unit that constructs a feature vector for the part of speech in the part of speech tag training set according to the selected feature template;
A similarity calculation unit for calculating a similarity between parts of speech using the feature vector;
The part-of-speech tagging system according to appendix 7, further comprising: a clustering unit that clusters the parts of speech based on the similarity and generates a part-of-speech hierarchical tree.

(Appendix 9)
A test set construction device for randomly selecting a text set as a test set from the part of speech tag training set;
An evaluation device that evaluates the results of part-of-speech tagging of text tagged from a test set using the part-of-speech tagging model;
The part-of-speech tagging system according to appendix 8, further comprising an adjustment device that adjusts the part-of-speech hierarchy tree according to the evaluation result.

(Appendix 10)
The part-of-speech tagging system according to appendix 9, wherein the adjustment unit adjusts a threshold value used for calculating a similarity between parts of speech by the part-of-speech hierarchy tree construction device.

(Appendix 11)
An unknown word part-of-speech estimation model construction device that learns word construction rules from a part-of-speech tag training set and constructs a part-of-speech guess model of an unknown word;
An unknown word part-of-speech correction device that tags an unknown word part-of-speech using an unknown word part-of-speech inference model and corrects the part-of-speech of the unknown word tagged using the part-of-speech tagging model; The part-of-speech tagging system according to appendix 1 or appendix 2.

(Appendix 12)
Part-of-speech tagging method,
A part-of-speech tagging model training step based on the part-of-speech hierarchy tree to train the part-of-speech tagging model for each hierarchy and for each node using the first tagged text in the part-of-speech tag training set;
A part-of-speech tagging method comprising: tagging part-of-speech tagging of a text to be tagged using a trained part-of-speech tagging model.

(Appendix 13)
The part-of-speech tagging model training step comprises:
A CRF that builds a CRF model training corpus by tagging the first tagged text in the part-of-speech tag training set against the second tagged text hierarchically and nodely based on the part-of-speech hierarchical tree. Model training corpus construction step,
A CRF model training step of training the CRF model in a corresponding hierarchical and nodal manner to obtain a part of speech tagging model by using the second tagged text tagged by the CRF model training corpus construction step. The part-of-speech tagging method according to supplementary note 12, characterized by comprising:

(Appendix 14)
The CRF model training corpus building step replaces the tagged part of speech of the first tagged text with the name of the subnode of the node corresponding to the position of the tagged part of speech in the part of speech hierarchical tree, and 14. The part-of-speech tagging method according to appendix 13, including a step of performing tagging in a node manner.

(Appendix 15)
The CRF model training step comprises:
(A) The current level is “0”, and the feature template includes two words before and after the second text, characters before and after the current word, and the co-occurrence frequency between the two words before and after. (B) The current level is not “0”, and the feature template is a feature template selected at level 0, two words before and after the second text in the immediately preceding hierarchy, and the co-occurrence frequency between parts of speech 15. Part-of-speech tagging according to appendix 14, including the step of training the CRF model hierarchically and nodally by selecting feature templates according to the case of co-occurrence between words and parts of speech Method.

(Appendix 16)
The part of speech tagging step comprises:
A CRF model feature construction step of constructing feature data hierarchically and nodely to apply the CRF model to the text to be tagged;
The CRF part-of-speech tagging step of tagging part-of-speech of text to be tagged hierarchically and nodely according to the feature data constructed by the CRF model feature construction step. Part-of-speech tagging method.

(Appendix 17)
The CRF model feature construction step comprises:
(A) The current level is 0, and feature data is used to input to the feature template selected at level 0 during CRF model training and extracted from the text to be tagged (b) For the CRF model, if the current level is not 0, the feature data extracted at level 0 is used, and the feature data is extracted from the second text tagged with the previous level CRF model The part-of-speech tagging method according to supplementary note 16, comprising the step of constructing feature data.

(Appendix 18)
The part-of-speech tagging method according to appendix 12, further comprising a part-of-speech hierarchy tree construction step of analyzing a relationship between part-of-speech of tagged text in the part-of-speech tag training set.

(Appendix 19)
The part of speech hierarchy tree construction step includes:
A part-of-speech feature template selection step for selecting a feature template representing a part-of-speech feature;
A feature vector construction step of constructing a feature vector for the part of speech in the part of speech tag training set according to the selected feature template;
A similarity calculation step of calculating a similarity between parts of speech using the feature vector;
The part-of-speech tagging method according to appendix 18, further comprising: clustering parts-of-speech based on similarity and generating a part-of-speech hierarchy tree.

(Appendix 20)
A test set construction step of randomly selecting a text set as a test set from the part of speech tag training set;
Using the part of speech tagging model to evaluate a part of speech tagging result of text tagged from a test set;
The part-of-speech tagging method according to appendix 19, further comprising an adjustment step of adjusting the part-of-speech hierarchy tree according to the evaluation result.

(Appendix 21)
21. The part-of-speech tagging method according to appendix 20, wherein the adjustment step includes a step of adjusting a threshold value used for calculating a similarity between parts of speech by the part-of-speech hierarchy tree construction step.

(Appendix 22)
Learn the word construction rules from the part-of-speech tag training set, and build an unknown word part-of-speech inference model construction step to construct a part-of-speech inference model for an unknown word,
An unknown word part-of-speech correction step of tagging an unknown word part-of-speech using an unknown word part-of-speech inference model and correcting an unknown word part-of-speech tagged using the part-of-speech tagging model The part-of-speech tagging method according to appendix 12 or appendix 13.

(Appendix 23)
A training device for a part-of-speech tagging model,
Build a CRF model training corpus by tagging the first tagged text in the part-of-speech tag training set to the second text layer by layer and node by node based on the part of speech hierarchy tree Unit,
A CRF model training unit that trains individual CRF models hierarchically and node by node using second text tagged by the CRF model training corpus building unit to obtain a part-of-speech tagging model Part-of-speech tagging model training device characterized by that.

(Appendix 24)
A part-of-speech tagging model training method,
CRF model training corpus construction step for constructing a CRF model training corpus by tagging the first tagged text in the part-of-speech tag training set to the second text for each hierarchy and each node based on the part-of-speech hierarchy tree When,
CRF model training steps for training individual CRF models layer by layer and node by node using second text tagged by the CRF model training corpus building unit to obtain a part of speech tagging model A training method for a part-of-speech tagging model characterized by that.

10: Part of speech tag training set 12: Part of speech tagging model training device 13: Part of speech tagging model 14: Part of speech hierarchy tree construction device 15: Part of speech hierarchy tree 22: Part of speech tagging device 140: Part of speech feature template selection unit 141: Feature vector construction Unit 142: Similarity calculation unit 143: Clustering unit 120: Logic circuit 121: CRF model training corpus construction unit 122: CRF model training unit 220: CRF model feature construction unit 221: CRF part-of-speech tagging unit 222: Logic circuit 16: Evaluation device 17: Adjustment device 18: Test set construction device 19: Unknown word part of speech estimation model construction device 20: Unknown word part of speech estimation model 21: Unknown word part of speech correction device

Claims (10)

A part-of-speech tagging system,
A part-of-speech tagging model training device that trains the part-of-speech tagging model hierarchically and nodely using the first tagged text in the part-of-speech tag training set based on the part-of-speech hierarchy tree;
A part-of-speech tagging system comprising: a part-of-speech tagging device that tags a part of speech of a text to be tagged using a trained part-of-speech tagging model. The part-of-speech tagging model training device comprises:
A CRF that builds a CRF model training corpus by tagging the first tagged text in the part-of-speech tag training set against the second tagged text hierarchically and nodely based on the part-of-speech hierarchical tree. A model training corpus building unit,
A CRF model training unit that trains the CRF model in a corresponding hierarchical and nodal manner to obtain a part-of-speech tagging model by using a second tagged text tagged by the CRF model training corpus building unit; The part-of-speech tagging system according to claim 1. The part of speech tagging device comprises:
A CRF model feature construction unit that constructs feature data hierarchically and nodely to apply the CRF model to the text to be tagged;
The CRF part-of-speech tagging unit for tagging part-of-speech of text to be tagged hierarchically and nodely according to feature data constructed by the CRF model feature construction unit. Part-of-speech tagging system. An unknown word part-of-speech estimation model construction device that learns word construction rules from a part-of-speech tag training set and constructs a part-of-speech guess model of an unknown word;
An unknown word part-of-speech correction device that tags an unknown word part-of-speech using an unknown word part-of-speech inference model and corrects the part-of-speech of the unknown word tagged using the part-of-speech tagging model; The part-of-speech tagging system according to claim 1 or 2. Part-of-speech tagging method,
A part-of-speech tagging model training step based on the part-of-speech hierarchy tree to train the part-of-speech tagging model for each hierarchy and for each node using the first tagged text in the part-of-speech tag training set;
A part-of-speech tagging method comprising: tagging part-of-speech tagging of a text to be tagged using a trained part-of-speech tagging model. The part-of-speech tagging model training step comprises:
A CRF that builds a CRF model training corpus by tagging the first tagged text in the part-of-speech tag training set against the second tagged text hierarchically and nodely based on the part-of-speech hierarchical tree. Model training corpus construction step,
A CRF model training step of training the CRF model in a corresponding hierarchical and nodal manner to obtain a part of speech tagging model by using the second tagged text tagged by the CRF model training corpus construction step. The part-of-speech tagging method according to claim 5, further comprising: The part of speech tagging step comprises:
A CRF model feature construction step of constructing feature data hierarchically and nodely to apply the CRF model to the text to be tagged;
7. The CRF part-of-speech tagging step of tagging part-of-speech of text to be tagged hierarchically and nodely according to the feature data constructed by the CRF model feature construction step. Part-of-speech tagging method. Learn the word construction rules from the part-of-speech tag training set, and build an unknown word part-of-speech inference model construction step to construct a part-of-speech inference model for an unknown word,
An unknown word part-of-speech correction step of tagging an unknown word part-of-speech using an unknown word part-of-speech inference model and correcting an unknown word part-of-speech tagged using the part-of-speech tagging model The part-of-speech tagging method according to claim 5 or 6. A training device for a part-of-speech tagging model,
Build a CRF model training corpus by tagging the first tagged text in the part-of-speech tag training set to the second text layer by layer and node by node based on the part of speech hierarchy tree Unit,
A CRF model training unit that trains individual CRF models hierarchically and node by node using second text tagged by the CRF model training corpus building unit to obtain a part-of-speech tagging model Part-of-speech tagging model training device characterized by that. A part-of-speech tagging model training method,
CRF model training corpus construction step for constructing a CRF model training corpus by tagging the first tagged text in the part-of-speech tag training set to the second text for each hierarchy and each node based on the part-of-speech hierarchy tree When,
CRF model training steps for training individual CRF models layer by layer and node by node using second text tagged by the CRF model training corpus building unit to obtain a part of speech tagging model A training method for a part-of-speech tagging model characterized by that.

Images