KR101997783B1 - Syllable-based Korean POS Tagging using POS Distribution and Bidirectional LSTM CRFs and Method of the same - Google Patents

Abstract

Bidirectional LSTM (Long Short Term Memory) CRFs (bi-directional) bi-directional loudspeakers with deep sequential labeling have been developed by combining the morpheme- LSTM-CRFs), comprising: a syllable separating unit for separating input sentences into syllable units; and a syllable-based morpheme analyzer for analyzing syllables based on syllables separated from the syllable separating unit a classification part for performing phonetic unit part tagging using bi-LSTM-CRFs to assign phonetic tags of morpheme including syllable; and a learning corpus An error removing unit for removing an error by processing a conversion having no ambiguity in a syllable; The results of four tag is attached through a circular restore it is composed by including a circular restored to convert morpheme unit.

Description

{Syllable-based Korean POS Tagging using POS Distribution and Bidirectional LSTM CRFs and Method of the Same}

The present invention relates to a morpheme analyzer, and more particularly, to a bidirectional LSTM (Long Short) which has a good performance in a region where a sequential label has been recently obtained by a deep learning by combining a morpheme unit separation process and a part- Term Memory) CRFs.

Accurate analysis is important because inaccurate results of Korean morphological analysis can have a serious impact on syntax analysis, semantic attachment, and machine translation. There are two general types of morpheme analysis: morpheme analysis and part-of-speech. Morphological analysis is the generation of morphemes and part-of-speech pairs with the smallest meaning. And part-tagging is to determine the most appropriate morpheme and part-of-speech pair considering the meaning and context of each word in candidates from morphological analysis.

In order to analyze Korean words in existing morpheme units, morpheme restoration and segmentation of morpheme units as well as part-of-speech determination are required. Since the morphological ambiguity and material ambiguity occur in each process, the process to deal with it is relatively complicated. Recently, there has been an increase in the number of syllabic tagging to solve this problem. When tagging a part of a syllable unit, it is possible to combine with functions such as spacing, and perform better than other language transplantation and previous studies.

Syllable unit morpheme analysis divides the input sentence into syllable units and uses a machine learning - based classifier such as CRF to determine the part - of - speech label including the B and I tags indicating the beginning and the succeeding morpheme in syllable unit. In addition, Korean uses diacritic dictionaries for effective syllable unit morphological analysis because diverse phonological phenomena occur as a group. The preliminary analysis dictionary is to make the morpheme-analyzed phrases in advance by using specific criteria and use it when tagging parts of speech. In addition, a circular restoration dictionary is additionally used to solve irregular verbs. Circular restoration dictionaries are used to attach complex tags to complex morphemes through simple rules.

For syllable unit morpheme analysis, we need a machine learning based classifier that can process sequential labeling as mentioned above, and there is a syllable unit morpheme analysis study using Structural SVM and CRF.

Structural SVM and CRF, however, must use various qualities to determine the label for a syllable. In particular, it is important to use information about syllables or phrases existing before and after the current syllable. Accurate analysis is important because the inaccurate results of this morpheme analyzer can have a catastrophic effect on classification analysis, semantic attachment, and machine translation.

Recently, syllable unit morpheme analysis method using Bidirectional LSTM (Long Short Term Memory) CRFs (bi-LSTM-CRFs), which has good performance in areas with many sequential labels, can be applied. Since bi-LSTM-CRFs affect the state of the state layer for the current input at the forward step and the state at the backward step influences the previous state, the machine learning for other sequential labeling Unlike a small number of qualities can get good results.

Korean Patent Application No. 10-2010-0077308 (filed on August 11, 2010) Korean Patent Application No. 10-2015-0089121 (filed on June 23, 2015)

Accordingly, the present invention has been made to solve the above-mentioned problems, and it is an object of the present invention to provide a method and apparatus for decoding a morpheme, The present invention provides a syllable unit morpheme analyzer and analysis method using Bidirectional LSTM (Long Short Term Memory) CRFs (bi-LSTM-CRFs).

Other objects of the present invention are not limited to the above-mentioned objects, and other objects not mentioned can be clearly understood by those skilled in the art from the following description.

According to another aspect of the present invention, there is provided a morpheme morpheme analyzer using a part-of-speech distribution and bidirectional LSTM CRFs, the morpheme morpheme analyzer comprising: a syllable separator for separating input sentences into syllable units; A classification unit for performing tagging of syllable unit parts using bi-LSTM-CRFs through a vector for syllables based on syllables and assigning partly tagged morpheme tags containing syllables; And an error correction unit for correcting errors by eliminating errors from the error removal unit by performing a circular restoration on the syllable tag by syllable unit, And a circular reconstruction unit for reconstructing the reconstructed image.

Preferably, the classifier is configured to learn a 64-dimensional syllable-unit embedding vector by using a word embedding algorithm word2vec to generate a vector for a syllable and use it as an input vector.

Preferably, the pre-analysis dictionary of the error elimination uses a dictionary dictionary and a noun dictionary.

Preferably, the circular reconstruction unit corrects the irregular transformation when there is an irregular transformation, and converts the corrected irregular transformation dictionary.

According to another aspect of the present invention, there is provided a method of analyzing a syllable unit morpheme using a part-of-speech distribution and bidirectional LSTM CRFs, the method comprising the steps of: (A) separating a sentence input through a syllable- LSTM-CRFs through a vector for a syllable based on the separated syllable through a classification unit to assign a part-of-speech tag of a morpheme including a syllable; and (C) (D) removing the error through a circular decompression unit and processing the syllable tag in units of syllables by a circular restoration unit; And converting the result of attaching the part-of-speech tag into a morpheme unit by circular restoration.

Preferably, step (A) uses syllable unit qualities and word unit qualities for CRF learning.

Preferably, the word unit qualities include extracting unique words in the entire corpora corpus, assigning IDs to the respective extracted phrases, and using the assigned IDs in terms of their qualities. do.

Preferably, the step (B) is characterized by using an embedding vector of a 64-dimensional syllable unit as an input vector by using a word embedding algorithm word2vec to generate a vector for the syllable.

Preferably, the step (C) is characterized in that the pre-analysis dictionary uses a dictionary dictionary and a noun dictionary.

Preferably, the noun dictionary is constructed of nouns that are not subject to ambiguous analysis.

Preferably, the dictionary dictionary includes a dictionary dictionary 1 that does not take context information into consideration and a dictionary dictionary 2 that resolves ambiguity in consideration of context information.

Preferably, the step (D) is performed by using an irregular conversion dictionary when irregular conversion is required during circular restoration.

Preferably, the step (D) includes the steps of: selecting the most frequent result when there is the same conversion in the irregular conversion dictionary in the irregular conversion; and And a step of completing the step.

As described above, the morphological analyzer and analysis method using the morphological distribution of the present invention and the bi-directional LSTM CRFs have the following effects.

First, bi-LSTM-CRFs are used to eliminate the complexity of the existing morpheme analyzer by inputting the part-of-speech tag distribution that expresses the part-of-speech well.

Second, morphological analysis is performed by syllable unit, and morphological segmentation and part - of - speech decision steps are performed separately without dividing them.

Third, using bi-LSTM-CRFs suitable for sequential labels and adding the part-of-speech qualities to the input vector, human's time and effort are less invested than the existing morpheme analyzer by adding information on parts of each syllable. High performance.

Fourth, morpheme segmentation and part - of - speech decision are solved in one syllable unit. By pre - constructing pre - analysis dictionary and circular restoration dictionary, various phonological phenomena can be solved.

1 is a block diagram showing the configuration of a part-of-speech morpheme analyzer using a part-of-speech distribution and bidirectional LSTM CRFs according to an embodiment of the present invention;
FIG. 2 is a flowchart for explaining a part-of-speech distribution according to an embodiment of the present invention and a method of analyzing syllable unit morpheme using bidirectional LSTM CRFs
FIG. 3 shows an embodiment of bi-LSTM-CRFs in which syllables are input in the present invention
FIG. 4 is a diagram comparing the tagging results of syllable unit parts with the existing CRF and bi-LSTM-CRFs of the present invention

Other objects, features and advantages of the present invention will become apparent from the detailed description of the embodiments with reference to the accompanying drawings.

Preferred embodiments of the part-of-speech morphology analyzing method and the syllable unit morpheme analyzing method using bi-directional LSTM CRFs according to the present invention will be described with reference to the accompanying drawings. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art. It is provided to let you know. Therefore, the embodiments described in the present specification and the configurations shown in the drawings are merely the most preferred embodiments of the present invention and are not intended to represent all of the technical ideas of the present invention. Therefore, various equivalents It should be understood that water and variations may be present.

FIG. 1 is a block diagram showing the configuration of a morphological distribution analyzer according to an embodiment of the present invention and a syllable morpheme analyzer using bidirectional LSTM CRFs.

As shown in FIG. 1, the syllable separating unit 100 separates input sentences into syllable units, and bi-LSTM-1 through syllable vectors based on the syllables separated by the syllable separating unit 100. [ A classifier 200 for assigning a phoneme tag of a morpheme containing a syllable by performing tagging of a syllable unit part using CRFs, and a classifier 200 for classifying the result of determining the part of speech tag in units of syllables in the classifier 200 (300) for eliminating an error by processing a transformation having no ambiguity in a learning corpus; and an error removal unit (300) for removing an error, And a circular reconstruction unit 400 for transforming the unit into a unit.

In order to generate a vector for a syllable in the classifier 200, an embedding vector of a 64-dimensional syllable unit is learned and used as an input vector using word2vec, which is a typical word embedding algorithm.

The pre-analysis dictionary of the error removal section 300 is a dictionary construction in which there is no ambiguity in part marking tagging, and a dictionary dictionary and a noun dictionary are used.

The circular restoring unit 400 corrects and converts the irregular transformation through an irregular transformation dictionary when irregular transformation exists.

The operation of the morphological distribution according to the present invention and the syllable morpheme analyzer using bidirectional LSTM CRFs will now be described in detail with reference to the accompanying drawings. Like reference numerals in FIG. 1 or FIG. 2 denote the same members performing the same function.

2 is a flowchart for explaining a part-of-speech distribution according to an embodiment of the present invention and a syllable unit morpheme analysis method using bidirectional LSTM CRFs.

Referring to FIG. 2, the sentence input through the syllable separation unit 100 is divided into syllable units (S10).

For syllable-based morphological part-tagging, it is important to determine the part-of-speech tag for the separated syllable. For example, the phrase "global" in a sentence is divided into syllable units, and then a part-of-speech tag is assigned to each syllable as shown in Table 1 below.

B-NNG represents the start syllable of the morpheme where the part of speech is NNG, and I-NNG represents the syllable of the morpheme where the part of speech is NNG. To attach phonemic tags in syllable units, a machine learning-based classifier such as CRF should be studied. To do this, use the qualities shown in Table 2 below for each syllable.

If the morpheme is an irregular morpheme, it indicates the morpheme to which the irregular conversion dictionary should be applied by attaching the "DIC" tag to the part of speech tag attached in syllable unit. A syllable with a "DIC" tag solves the problem of irregularities by using a random conversion dictionary in the round reconstruction step. For example, in the case of "global ", a part of speech tagging is performed as" global / NNG + / VCP + c / ETM costume / NNG ", where " Used for learning.

In the present invention, syllable unit qualities and word unit qualities mentioned in Table 2 are used for CRF learning. For the effective use of the unit features, the idioms are extracted from the whole corpus and then the IDs are assigned to each word and the assigned IDs are used as the qualities. In addition, the beginning of the sentence is represented by: S, and in trigram phrases: S is also used for the previous word position. Also, the end of the sentence is represented by: O. For example, at the beginning and end of a sentence, qualities are generated as shown in Table 3 below.

The CRF must use several qualities to attach part-of-speech tags for one syllable. On the other hand, bi-LSTM-CRFs can obtain good results by using only vectors for syllables.

Therefore, tagging of syllable part-of-speech parts using bi-LSTM-CRFs is performed through vectors for syllables based on the syllables separated by the syllable separating part 100 through the classifying part 200, (S20). At this time, to generate a vector for the syllable in the classifier 200, a word-based embedding algorithm word2vec is used to learn an embedding vector of a 64-dimensional syllable unit as an input vector.

3 is an embodiment of bi-LSTM-CRFs in which syllables are input in the present invention.

As shown in FIG. 3, bi-LSTM-CRFs is a syllable of "L" when a phrase "to fashion show" comes in to learn in syllable unit, "Becomes the input. When the syllable "se" is entered, the state of the previous syllable affects the state of the current syllable, so that the state of the current syllable has the same meaning as the state actually representing "fashion". Forward bi-LSTM-CRFs until all syllables enter the input. In the backward step, in contrast to forward, the syllable is preceded by a "syllable" followed by a syllable "show". After the forward and backward steps for the word, the two steps are calculated through the back-propagation algorithm after calculating the cost of the result and the correct answer. Recently, research has been carried out to improve the performance by reflecting the transition probability between tags. For this purpose, forward algorithm like CRF and Viterbi search algorithm have been used to find optimal tag sequence.

bi-LSTM-CRFs is effective because it only uses syllable input, unlike CRF, because it learns by reflecting the information of the preceding syllable and the next syllable.

Meanwhile, the input vector of the syllable is expanded to improve the performance of the syllable-based morphological part-of-speech tagging using the bi-LSTM-CRFs. To do this, we extend the input vector by expressing the part-of-speech distribution of the morpheme in which the syllable is included in the learning corpus. Syllables can have different parts of speech depending on the morpheme they contain.

For example, the syllable "lower" may be some syllable of "sky" having a noun tag in the learning corpus, or some syllable of "whitish" having an adjective tag. The morphological distribution of the morpheme in which these syllables appear in the learning corpus is expressed as a vector and used as the input of bi-LSTM-CRFs. The vector representing the partly syllable distribution is expressed as 134 dimensions with the addition of three tags representing the beginning, end, and blank of the sentence in 131 dimensions reflecting B, I and DIC tags in 46 parts of speech. The value of each dimension computes the frequency of the speech tag for the syllable from the corpus. For every syllable, we calculate all frequencies that occur in the corpus and then make a probability value through softmax to determine the value of the vector.

Table 4 shows an example of a part-of-speech distribution vector for a syllable "."

As shown in Table 4, the syllable "Rang" appears 435 times in the learning corpus, and has a part-of-speech tag such as B-NNP, I-NNP, I-NNG and B-JKB. The generated vector is combined with the syllable embedding vector and finally generates a vector of 198 (64 + 134) dimension, which is used as an input of bi-LSTM-CRFs.

Next, an error is removed by processing a non-verbal conversion in the learning corpus through a pre-analysis dictionary with respect to a result of determining the parts-of-speech tag in units of syllables in the classifying unit 200 through the error removing unit 300 (S30) .

In this case, the preliminary analysis dictionary is a dictionary construction in which there is no ambiguity in speech tagging, and it is necessary to convert it into speech tag existing in the learning corpus regardless of the result of CRF syllable tag addition. The pre-analysis dictionary uses a dictionary dictionary and a noun dictionary. For example, in the case of a noun "entertainer", there is no case in which a part-of-speech tag other than "NNG" is attached to a corpus. Table 5 shows an example of applying the noun dictionary.

The noun dictionary was built with nouns that can not be analyzed as ambiguous. However, if it is a noun, if it is a short noun, it is tagged as the same letter or another part of speech as "E / NNG" and "E / JX". To solve this problem, we constructed 169,004 nouns and compound nouns over a certain length.

In addition, the dictionary of the dictionary has constructed the dictionary 1 which does not take context information into account and the dictionary 2 which resolves the ambiguity considering the context information. In order to solve this problem, when the dictionary dictionary 2 has only the word phrases, the problem of the word with the ambiguity of the context becomes a problem. Therefore, the last phrases tag included in the word phrases of the corresponding phrases to be included in the dictionary dictionary are stored together to solve the ambiguity. The dictionary 1 has 1,552,635 words and the dictionary 2 has 37,233 words. Table 6 shows an example of applying the dictionary dictionary.

Then, the error removing unit 300 removes the error through the circular restoring unit 400, and converts the result of attaching the part-of-speech tag in units of syllables into a morpheme unit by circular restoration (S40). At this time, at the time of the circular restoration, when the irregular transformation described above is required, the irregular transformation dictionary is used. Table 7 shows an example in which an irregular conversion dictionary is applied.

When the irregular conversion has the same conversion in the irregular conversion dictionary, the highest frequency result is selected. After applying the random transformation, finally, the morphemes having the same part-of-speech tag are combined to complete the morpheme part-tagging. For example, in the case of "Production / NNG + Worker / NNG + S / XSN", it is changed to "Production Worker / NNG + S / XSN" because part NNG is the same.

Example

In the present invention, Sejong Corpus was used to evaluate syllable-based morphological part-tagging. Finally, in order to compare the CRF-based method with the proposed bi-LSTM-CRFs method, we selected 500,000 randomly selected words and 400,000 words were used for the learning and 100,000 words were used for the test. In order to judge the model that shows the best performance among the proposed methods, it is necessary to select 50,000 words at random and learn 4-language words, The 28th Korean and Korean Information Processing Conference Proceedings (2016) were tested and evaluated. For the evaluation, the following equations (1) and (2) were used.

FIG. 4 is a graph comparing the syllable unit parts tagging result using the existing CRF and bi-LSTM-CRFs of the present invention.

As shown in FIG. 4, when the bi-LSTM-CRFs based syllabic tagging method using the syllable distribution vector of the syllable according to the present invention is applied, the syllable part tagging method using the syllable distribution vector according to the present invention is applied to a 92.93% Tagging performance and showed 97.09% performance improvement of CRF by 3.01% when circular restoration was performed after applying pre-analysis dictionary and irregular conversion dictionary.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. It will be apparent to those skilled in the art that various modifications may be made without departing from the scope of the present invention. Accordingly, the true scope of the present invention should be determined by the technical idea of the appended claims.

Claims (13)

A syllable separating unit for separating the input sentence into syllable units,
LSTM-CRFs based on syllables separated by the syllable separating unit, and assigning a part-of-speech tag of a morpheme including a syllable;
An error elimination unit for processing an irrelevant conversion in the learning corpus through a preliminary analysis dictionary to the result of determining the part of speech tag in the syllable unit in the classifying unit,
And a circular decompression unit for removing the error from the error elimination and converting the result of attaching the part-of-speech tag to the morpheme unit by circular restoration in units of syllable. The morphological distribution and the syllable unit morpheme using bidirectional LSTM CRFs Analyzer. The method according to claim 1,
Wherein the classification unit learns a 64-dimensional syllable-unit embedding vector by using a word embedding algorithm word2vec to generate a vector for a syllable and uses the embedding vector as an input vector to generate a bilingual LSTM CRFs Syllable unit morpheme analyzer. The method according to claim 1,
Wherein the analysis dictionary of the error elimination uses a phrase dictionary and a noun dictionary, and a syllable morpheme analyzer using bidirectional LSTM CRFs. The method according to claim 1,
Wherein the circular decompression unit corrects the irregular transformation dictionary when there is an irregular transformation dictionary, and converts the corrected irregular transformation dictionary to a syllable unit morpheme analyzer using bidirectional LSTM CRFs. (A) separating an input sentence into syllable units in a syllable separating unit,
(B) a step of performing syllable part-of-speech tagging using bi-LSTM-CRFs through a vector for a syllable in the classifying part based on syllables separated by the syllable separating part, ,
(C) removing errors from the result of determining the part-of-speech tag in units of the syllable by processing a non-causal transformation in the learning corpus through the pre-analysis dictionary in the error elimination;
(D) removing the error and converting the result of attaching the part-of-speech tag to the syllable tag in a syllable unit by performing a circular restoration in the circular restoration unit. Morphological analysis method. 6. The method according to claim 5, wherein step (A) performed in the syllable separating unit
A method for analyzing syllable unit morphemes using speech segment features and bidirectional LSTM CRFs using syllable unit qualities and word unit qualities for CRF learning. The method according to claim 6,
Wherein the word unit qualities include extracting unique phrases from the entire corpus,
Assigning an ID to each extracted word;
And expressing the assigned ID as a quality and using the allocated ID. The method of analyzing syllable unit morpheme using bidirectional LSTM CRFs. 6. The method of claim 5, wherein step (B) performed in the classifier
A word-embedding algorithm word2vec is used to generate a vector for the syllable, and an embedding vector of a 64-dimensional syllable unit is learned and used as an input vector. Unit morphological analysis method. 6. The method as claimed in claim 5, wherein the step (C)
A Method of Analysis of Syllable Unit Morphology Using Part - of - Speech Distribution and Bi - directional LSTM CRFs. 10. The method of claim 9,
Wherein said noun dictionary is constructed of nouns that are not subject to ambiguity analysis, and a method for analyzing syllable unit morphology using bi-directional LSTM CRFs. 10. The method of claim 9,
The method of claim 1, wherein the dictionary includes a dictionary of dictionary 1 that does not consider context information and a dictionary of dictionary 2 that resolves the ambiguity by taking context information into account. The method of analyzing syllable unit morphology using two-way LSTM CRFs. 6. The method of claim 5,
Wherein the step (D) performed by the circular decompression unit is performed using an irregular transformation dictionary when irregular transformation is required at the time of round restoration, and a method for analyzing a syllable unit morpheme using bidirectional LSTM CRFs. 6. The method of claim 5, wherein step (D) performed in the circular reconstruction unit
Selecting a result having the highest frequency when there is the same transformation in the irregular transformation dictionary in the irregular transformation,
And finally combining the morphemes having the same part-of-speech tag after completing the irregular transformation, and completing the morphological part-of-speech tagging, and analyzing the syllable unit morpheme using bi-directional LSTM CRFs.

Images