KR102575034B1 - Method, Computing Device and Computer-readable Medium for Dependency Parsing Based on Sequence Labeling - Google Patents

Abstract

The present invention relates to a dependent syntax analysis method based on sequence labeling, a computing device, and a computer-readable medium, and more particularly, for a plurality of words included in an input sentence to be analyzed, a plurality of embeddings for each word. A word embedding value including a value is derived, and a plurality of word embedding values derived for each word are input to a framework composed of an encoder-decoder to determine the relative position of a governing station having a dependent relationship for each word included in the input sentence. It relates to a sequence labeling-based dependency syntax analysis method for predicting and labeling dependency names, a computing device, and a computer-readable medium.

Description

Dependency parsing method based on sequence labeling, computing device and computer-readable medium {Method, Computing Device and Computer-readable Medium for Dependency Parsing Based on Sequence Labeling}

The present invention relates to a dependent syntax analysis method based on sequence labeling, a computing device, and a computer-readable medium, and more particularly, for a plurality of words included in an input sentence to be analyzed, a plurality of embeddings for each word. A word embedding value including a value is derived, and a plurality of word embedding values derived for each word are input to a framework composed of an encoder-decoder to determine the relative position of a governing station having a dependent relationship for each word included in the input sentence. It relates to a sequence labeling-based dependency syntax analysis method for predicting and labeling dependency names, a computing device, and a computer-readable medium.

Syntactic analysis is for understanding the grammatical structure of a given sentence, and corresponds to an essential element for understanding a natural language. Syntactic analysis may resolve ambiguity in a corresponding language by constructing a dependency tree expressing a dependency relationship between words or phrases included in the corresponding sentence with respect to the input sentence. As such, the field of syntax analysis is used for machine translation or information retrieval, and its importance is also increasing.

Specifically, the syntax analysis method is divided into a phrase structure parsing based on a phrase structure grammar and a dependency parsing based on a dependent grammar according to the grammatical structure.

Among them, dependency syntax analysis expresses two words as a dependency relationship consisting of a dominant element and a dependent element, and analyzes the syntax based on this. The Head is the central element of meaning, and the Dependent corresponds to the element that complements the meaning of the dominant. On the other hand, the dominant and dependent elements always have a single dependent relationship, and the semantic and structural ambiguity of the sentence can be resolved through the dependent relationship between the dominant and dependent elements. That is, the dependent syntax analysis method has the advantage of being relatively robust in analyzing sentences in which word order is freely arranged or omitted, such as Korean, by expressing the dependent relationship between two words.

Meanwhile, dependent syntax analysis methods can be largely divided into graph-based syntax analysis, transition-based syntax analysis, and sequence labeling-based syntax analysis. Graph-based syntactic analysis is a method of finding a dependency tree with a maximum score by using a greedy algorithm after obtaining scores for dependencies between all words (words) included in a sentence and names of the dependencies. Transition-based syntactic analysis is a method in which the dependency relationship between two words included in a sentence is determined based on the stored stack, buffer, and transition history, and a dependency tree is gradually created using this.

On the other hand, sequence labeling-based syntactic analysis considers the dominant and dependent relationship names as one label, and labels each word included in the input sentence with a specific label, and the dependent syntactic analysis results are classified according to the order of each word. predict sequentially. As such, the sequence labeling-based syntax analysis has the advantage of not including the problems of the above-described graph-based syntax analysis and transition-based syntax analysis.

Specifically, unlike the graph-based dependent syntax analysis method that determines the optimal dependent syntax analysis result for all words through an algorithm with high time complexity, the sequence labeling-based syntax analysis uses a framework of encoder-decoder structure for each word. Reliance on Syntax analysis results are predicted sequentially, so the time complexity is not relatively high, and transition-based parsing requires a separate data structure such as a stack or buffer, but sequence labeling-based parsing does not require a separate data structure. Syntax analysis is possible.

Therefore, there is a need to develop a new method for analyzing natural language more effectively based on a sequence labeling-based syntax analysis method.

The present invention relates to a dependent syntax analysis method based on sequence labeling, a computing device, and a computer-readable medium, and more particularly, for a plurality of words included in an input sentence to be analyzed, a plurality of embeddings for each word. A word embedding value including a value is derived, and a plurality of word embedding values derived for each word are input to a framework composed of an encoder-decoder to determine the relative position of a governing station having a dependent relationship for each word included in the input sentence. An object of the present invention is to provide a sequence labeling-based dependency syntax analysis method, a computing device, and a computer-readable medium for predicting and labeling dependency names.

In order to solve the above problems, in one embodiment of the present invention, a sequence labeling-based dependent syntax analysis method for an input sentence performed in a computing device including one or more processors and one or more memories, A word embedding value derivation step of receiving an input sentence including a word and deriving a word embedding value for each of the plurality of words by merging a plurality of embedding values derived according to a plurality of preset expression rules for each of the plurality of words; Through an encoder including a plurality of encoder cells, an output that inputs each of a plurality of word embedding values according to the word order in the input sentence to each of the encoder cells and outputs an output representation for each of the plurality of word embedding values. representation output step; and through a decoder including a plurality of decoder cells, input information obtained by merging each of a plurality of output representations according to word order in the input sentence and one or more output representations adjacent to the corresponding output representation is input to each of the decoder cells, A labeling step of labeling the control points of the word corresponding to the corresponding output representation and the names of the dependencies between the control points, wherein the encoder has a bidirectional LSTM (BLSTM) structure and the decoder has a LSTM structure; Each of the plurality of encoder cells receives a hidden representation from each of the encoder cells adjacent to the first direction of the corresponding encoder cell and the encoder cells adjacent to the second direction of the corresponding encoder cell, and based on the word embedding value and the plurality of hidden representations, the corresponding word Provides a dependency syntax analysis method based on sequence labeling that outputs an output representation for an embedding value.

In one embodiment of the present invention, the encoder further includes an attention module that receives a plurality of output representations from each of the plurality of encoder cells, and the attention module receives the plurality of output representations based on the plurality of output representations received. A plurality of attention weights input to each decoder cell of can be derived.

In one embodiment of the present invention, the plurality of expression rules include a first expression rule expressing the corresponding word in units of morphemes, and the step of deriving the word embedding value is performed on the corresponding word according to the first expression rule. The first embedding value may be derived by embedding the morpheme, and when the corresponding word includes a plurality of morphemes, the first embedding value may be derived by embedding the first morpheme of the corresponding word.

In one embodiment of the present invention, the plurality of expression rules include a second expression rule expressing the corresponding word in syllable units, and the word embedding value derivation step is included in the corresponding word according to the second expression rule. One or more syllable embedding values may be derived by embedding each of the one or more syllables, and a second embedding value may be derived by merging the one or more syllable embedding values.

In one embodiment of the present invention, the plurality of expression rules express a corresponding word as a part of speech for each of one or more morphemes of the corresponding word based on first part of speech tag information preset for the language of the input sentence. 3 expression rules, and the step of deriving the word embedding value derives a third embedding value by embedding the part-of-speech of the morpheme for the corresponding word according to the third expression rule, and the word includes a plurality of morphemes. In this case, the third embedding value may be derived by embedding the part-of-speech of the first morpheme of the corresponding word.

In one embodiment of the present invention, the plurality of expression rules include a fourth expression rule expressed as a part-of-speech tag for a corresponding word based on second part-of-speech tag information preset to express a plurality of languages in the same form. In the word embedding value derivation step, a fourth embedding value may be derived by embedding a part-of-speech tag for a corresponding word according to the fourth expression rule.

In one embodiment of the present invention, the plurality of expression rules are expressed as word tags generated by merging part-of-speech tags for each of one or more morphemes included in the corresponding word based on preset third part-of-speech tag information. In the step of deriving a word embedding value, a fifth embedding value may be derived by embedding a word tag for a corresponding word according to the fifth expression rule.

In one embodiment of the present invention, the labeling step may include distance information between a position of a word corresponding to a corresponding output representation and a position of a dominant position of a word corresponding to a corresponding output representation in a plurality of words included in the input sentence. ; and a name of a dependency relationship with the governing station.

In one embodiment of the present invention, the labeling step may include determining between a position of a word corresponding to the specific output representation in a plurality of words included in the input sentence and a position of a dominant position for a word corresponding to the specific output representation. distance information considering the POM tag for the ruling office; and a name of a dependency relationship with the governing station.

In order to solve the above problems, in one embodiment of the present invention, a computing device including one or more processors and one or more memories, which performs a sequence labeling-based dependent syntax analysis method for an input sentence, comprising a plurality of words A word embedding value derivation unit that receives an input sentence including a word and derives a word embedding value for each of the plurality of words by merging a plurality of embedding values derived according to a plurality of preset expression rules for each of the plurality of words; Through an encoder including a plurality of encoder cells, an output that inputs each of a plurality of word embedding values according to the word order in the input sentence to each of the encoder cells and outputs an output representation for each of the plurality of word embedding values. representation output unit; and through a decoder including a plurality of decoder cells, input information obtained by merging each of a plurality of output representations according to word order in the input sentence and one or more output representations adjacent to the corresponding output representation is input to each of the decoder cells, A labeling unit for labeling the control points of the word corresponding to the corresponding output representation and the names of the dependencies between the control points, wherein the encoder has a Bidirectional LSTM (BLSTM) structure and the decoder has a LSTM structure; Each of the plurality of encoder cells receives a hidden representation from each of the encoder cells adjacent to the first direction of the corresponding encoder cell and the encoder cells adjacent to the second direction of the corresponding encoder cell, and based on the word embedding value and the plurality of hidden representations, the corresponding word Provided is a computing device that outputs an output representation for an embedding value.

In order to solve the above problems, in one embodiment of the present invention, a computer program is executed in a computing device including one or more processors and one or more memories, based on sequence labeling for input sentences performed by the computing device. A computer-readable medium for implementing a dependent syntax analysis method, wherein the computer-readable medium stores instructions for causing the computing device to perform the following steps, the following steps comprising a plurality of words A word embedding value derivation step of receiving an input sentence including and deriving a word embedding value for each of the plurality of words by merging a plurality of embedding values derived according to a plurality of preset expression rules for each of the plurality of words; Through an encoder including a plurality of encoder cells, an output that inputs each of a plurality of word embedding values according to the word order in the input sentence to each of the encoder cells and outputs an output representation for each of the plurality of word embedding values. representation output step; and through a decoder including a plurality of decoder cells, input information obtained by merging each of a plurality of output representations according to word order in the input sentence and one or more output representations adjacent to the corresponding output representation is input to each of the decoder cells, A labeling step of labeling the control points of the word corresponding to the corresponding output representation and the names of the dependencies between the control points, wherein the encoder has a bidirectional LSTM (BLSTM) structure and the decoder has a LSTM structure; Each of the plurality of encoder cells receives a hidden representation from each of the encoder cells adjacent to the first direction of the corresponding encoder cell and the encoder cells adjacent to the second direction of the corresponding encoder cell, and based on the word embedding value and the plurality of hidden representations, the corresponding word A computer-readable medium is provided that outputs an output representation of an embedding value.

According to an embodiment of the present invention, in the word embedding value derivation step, a plurality of embedding values are derived for each word using a plurality of expression rules, and the word embedding value for each word generated by merging the plurality of embedding values is sequenced. By inputting to the encoder of the framework for performing the labeling-based dependency syntax analysis method, the prediction result of the dependency syntax analysis can be improved compared to the prediction result according to the conventional method.

According to an embodiment of the present invention, each of the plurality of decoder cells included in the decoder additionally receives an attention weight from the attention module included in the encoder and performs labeling on the word, so that the prediction result of dependency syntax analysis is obtained in the conventional case. Compared to the prediction result according to the method, an improvement effect can be exerted.

According to one embodiment of the present invention, in the labeling step, in order to label the dominant location for each word and the dependent relationship name with the corresponding dominant location as one label, the dependent relationship name and the relative distance between the dependent and dominant location are determined. By labeling with paired labels, an effect capable of achieving high performance compared to conventional methods can be exerted.

1 schematically illustrates a result of performing dependency syntax analysis on an input sentence by a computing device that performs a sequence labeling-based dependency syntax analysis method according to an embodiment of the present invention.
2 schematically illustrates internal components of a computing device that performs a dependency syntax analysis method based on sequence labeling according to an embodiment of the present invention.
3 schematically illustrates detailed steps of a sequence labeling-based dependency syntax analysis method according to an embodiment of the present invention.
4 schematically illustrates a process of deriving an embedding value according to an embodiment of the present invention.
5 schematically illustrates a fourth expression rule for converting a word converted by a third expression rule according to an embodiment of the present invention.
6 schematically illustrates a fifth expression rule for converting a morpheme analysis result of a word into a word tag according to an embodiment of the present invention.
7 schematically illustrates a process of performing a labeling step according to an embodiment of the present invention.
8 schematically illustrates a framework to which an encoder and a decoder are connected according to an embodiment of the present invention.
9 schematically illustrates the structure of a unidirectional LSTM configured in a decoder according to an embodiment of the present invention.
10 schematically shows the structure of a bidirectional LSTM configured in an encoder according to an embodiment of the present invention.
11 schematically illustrates the internal configuration of a computing device according to an embodiment of the present invention.

In the following, various embodiments and/or aspects are disclosed with reference now to the drawings. In the following description, for purposes of explanation, numerous specific details are set forth in order to facilitate a general understanding of one or more aspects. However, it will also be appreciated by those skilled in the art that such aspect(s) may be practiced without these specific details. The following description and accompanying drawings describe in detail certain illustrative aspects of one or more aspects. However, these aspects are exemplary and some of the various methods in principle of the various aspects may be used, and the described descriptions are intended to include all such aspects and their equivalents.

Moreover, various aspects and features will be presented by a system that may include a number of devices, components and/or modules, and the like. It should also be noted that various systems may include additional devices, components and/or modules, and/or may not include all of the devices, components, modules, etc. discussed in connection with the figures. It must be understood and recognized.

"Example", "example", "aspect", "exemplary", etc., used herein should not be construed as preferring or advantageous to any aspect or design being described over other aspects or designs. . The terms '~unit', 'component', 'module', 'system', 'interface', etc. used below generally mean a computer-related entity, and for example, hardware, hardware It may mean a combination of and software, software.

Also, the terms "comprises" and/or "comprising" mean that the feature and/or element is present, but excludes the presence or addition of one or more other features, elements and/or groups thereof. It should be understood that it does not.

In addition, terms including ordinal numbers, such as first and second, may be used to describe various components, but the components are not limited by the terms. These terms are only used for the purpose of distinguishing one component from another. For example, a first element may be termed a second element, and similarly, a second element may be termed a first element, without departing from the scope of the present invention. The terms and/or include any combination of a plurality of related recited items or any of a plurality of related recited items.

In addition, in the embodiments of the present invention, unless otherwise defined, all terms used herein, including technical or scientific terms, are generally understood by those of ordinary skill in the art to which the present invention belongs. has the same meaning as Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related art, and unless explicitly defined in the embodiments of the present invention, an ideal or excessively formal meaning not be interpreted as

1 schematically illustrates a result of performing dependency syntax analysis on an input sentence by a computing device 1000 performing a sequence labeling-based dependency syntax analysis method according to an embodiment of the present invention.

As shown in FIG. 1, a computing device 1000 performing the sequence labeling-based dependency syntax analysis method receives an input sentence to be analyzed, and the above-described sequence labeling-based dependency syntax analysis method for a plurality of words included in the input sentence. Syntax analysis can be performed.

In an embodiment of the present invention described later, in order to facilitate the description of the invention, as shown in FIG. 1 as an example of an input sentence, the sentence 'Kyungsim is twenty-five this year' is used, and the corresponding input Sequence labeling-based dependence performed by the computing device 1000 for a total of five plural words included in a sentence ('Gyeongsimi', 'this year', 'age', 'twenty', and 'five') Explain the syntax analysis process.

That is, through the sequence labeling-based dependent syntactic analysis process performed by the computing device 1000, the dominant elements having the corresponding word as dependent elements are predicted in the order of a plurality of words included in the input sentence, and the dependent elements and the corresponding dominant elements are predicted. It predicts the names of dependencies between cows. Specifically, in the present invention, dependency syntax analysis may be performed through labeling in which a dominant element and a dependent relationship name for a dependent element are used as one label.

FIG. 2 schematically illustrates internal components of a computing device 1000 that performs a dependency syntax analysis method based on sequence labeling according to an embodiment of the present invention.

As shown in FIG. 2, the computing device 1000 includes a word embedding value derivation unit 1100, an output representation output unit 1200, and a labeling unit 1300 in order to perform the sequence labeling-based dependent syntax analysis method. and DB 1400.

The word embedding value deriving unit 1100 performs a word embedding value deriving step (S10) to derive a word embedding value for each of a plurality of words included in the input sentence. Specifically, for this purpose, the word embedding value deriving unit 1100 converts each of a plurality of words according to a plurality of expression rules, and embeds each of the converted words according to each of a plurality of expression rules to obtain a plurality of words. Derive the embedding value. In this way, the word embedding value for each word is derived based on the plurality of embedding values derived for each word. The word embedding value corresponds to a value expressed in a vector form in order to effectively perform sequence labeling-based dependency syntax analysis on the word, and the word embedding value is input to the output representation output unit 1200 to be described later. Finally, labeling of the corresponding word may be performed.

The output representation output unit 1200 performs an output representation output step (S20), inputs a plurality of word embedding values derived for each word to the encoder 1430, and inputs a plurality of word embedding values through the encoder 1430. An output representation for each embedding value is derived. The output representation may correspond to a feature value for the corresponding word considering a relationship between one or more words located before the first direction of the corresponding word and one or more words located before the second direction of the corresponding word. In this way, a plurality of output representations derived for each of a plurality of word embedding values may be input to a labeling unit 1300 to be described later.

The labeling unit 1300 performs a labeling step (S30), inputs a plurality of output representations corresponding to a plurality of word embedding values to the decoder 1440, and through the decoder 1440 for each of a plurality of words. Derive labeling results. The labeling result may correspond to information in which a corresponding word is a dependent element, and a dominant element for the dependent element and a name of a dependency relationship with the dominant element are used as one label. In this way, in the labeling step (S30), by performing labeling on each of a plurality of words included in the input sentence, dependency syntax analysis on the input sentence may be performed.

The DB 1400 stores elements required for steps (S10 to S30) performed by the word embedding value derivation unit 1100, the output representation output unit 1200, and the labeling unit 1300, respectively. In the case where each step is performed, the elements required by the step can be provided. Specifically, when the word embedding value derivation step (S10) is performed in the word embedding value derivation unit 1100, the plurality of expression rules 1410 and the plurality of expression rules 1410 for converting the word of the input sentence An embedding module 1420 for embedding the converted word may be stored in the DB 1400. In addition, when the output representation output step (S20) is performed in the output representation output unit 1200, an encoder 1430 that receives a word embedding value and outputs an output representation may be stored in the DB 1400, When the labeling step (S30) is performed in the labeling unit 1300, the decoder 1440 that receives the output representation and outputs the labeling result may also be stored in the DB 1400.

Hereinafter, processes for performing the sequence labeling-based dependency syntax analysis method of the present invention by internal components included in the computing device 1000 will be described in detail.

3 schematically illustrates detailed steps of a sequence labeling-based dependency syntax analysis method according to an embodiment of the present invention.

As shown in FIG. 3 , in one embodiment of the present invention, as a sequence labeling-based dependency syntax analysis method for an input sentence performed in a computing device 1000 including one or more processors and one or more memories, a plurality of A word embedding value derivation step of receiving an input sentence including a word and deriving a word embedding value for each of the plurality of words by merging a plurality of embedding values derived according to a plurality of preset expression rules for each of the plurality of words. (S10); Through the encoder 1430 including a plurality of encoder cells 1431, each of a plurality of word embedding values according to the word order in the input sentence is input to each of the encoder cells 1431, and the plurality of word embedding values an output representation output step (S20) of outputting an output representation for each; and input information obtained by merging each of a plurality of output representations according to the word order in the input sentence and one or more output representations adjacent to the corresponding output representation through a decoder 1440 including a plurality of decoder cells 1441, respectively. It may include a labeling step (S30) of inputting to the decoder cell 1441 of the output representation and labeling the dominant location for the word corresponding to the corresponding output representation and the name of the dependent relationship with the corresponding dominant location.

Specifically, in the word embedding value derivation step (S10), in order to input a plurality of words included in the input sentence to the encoder 1430 used in the output representation output step (S20), a plurality of expressions are expressed for each of the plurality of words. Embedding is performed to express the features of a plurality of words according to rules, and the word embedding value for each word is derived by concatenating the plurality of embedded embedding values into one.

As such, in the present invention, a word is used as an analysis unit for performing sequence labeling-based dependency syntax analysis, and in particular, in order to effectively perform dependency syntax analysis on unregistered words, as described above, the word is included. By expressing the word to be expressed as a word embedding value according to a plurality of expression rules, it is possible to easily perform dependency syntax analysis on unregistered words.

In the output representation output step (S20), an output representation for each word is output using an encoder 1430 composed of a plurality of encoder cells 1431. Specifically, each of the plurality of encoder cells 1431 is a cell unit of an LSTM neural network in which each of the plurality of encoder cells 1431 is connected in a first direction and an LSTM in which each of the plurality of encoder cells 1431 is connected in a second direction It may include a cell unit of a neural network. Preferably, the first direction may mean a sentence progression direction of the input sentence, and the second direction may mean a reverse direction of the sentence progression of the input sentence.

Meanwhile, the encoder 1430 includes a plurality of encoder cells 1431 equal to or greater than the number of words included in the input sentence, and therefore, each of the plurality of encoder cells 1431 sequentially connected has a sentence of the input sentence. A word embedding value for each word corresponding to the proceeding order is input, and an output representation corresponding to the word embedding value can be output.

In the labeling step (S30), labeling of the dominant and dependent relation names of the word corresponding to each output representation is input using the decoder 1440 composed of a plurality of decoder cells 1441, and the corresponding word It is possible to predict the names of the governing places and dependencies for . Specifically, each of the plurality of decoder cells 1441 may include a cell unit of an LSTM neural network connected in a first direction.

Meanwhile, the decoder 1440 includes a plurality of decoder cells 1441 equal to or greater than the number of output representations output from the encoder 1430, and therefore, each of the plurality of decoder cells 1441 sequentially connected has an input An output representation for each word corresponding to the sentence progression sequence of the sentence is input, and a labeling result corresponding to the corresponding output representation can be output.

In addition, in the present invention, not only the output representation corresponding to the corresponding word is input to each decoder cell 1441, but the corresponding output representation and one or more output representations adjacent to the corresponding output representation are merged and input to the decoder cell 1441. It can be.

4 schematically illustrates the process of deriving the word embedding value step (S10) according to an embodiment of the present invention.

As shown in FIG. 4, in the word embedding value derivation step (S10), the corresponding word is converted according to a plurality of expression rules for each of a plurality of words included in the input sentence, and the converted word according to the plurality of expression rules is embedded. A plurality of embedding values are derived, and the word embedding value for the corresponding word is derived by merging them. On the other hand, in one embodiment of the present invention, the plurality of expression rules include the first to fifth expression rules, and hereinafter, a process of converting a word according to each expression rule and embedding it will be described. FIG. As shown in , the derivation of the word embedding value for the word 'Gyeongsimi' will be described as an example.

As shown in FIG. 4, in one embodiment of the present invention, the plurality of expression rules include a first expression rule for expressing the corresponding word in morpheme units, and the word embedding value deriving step (S10), A first embedding value may be derived by embedding the morpheme of the corresponding word according to the first expression rule, and when the corresponding word includes a plurality of morphemes, the first embedding value may be derived by embedding the first morpheme of the corresponding word. there is.

Specifically, the first expression rule corresponds to an expression rule for converting a word into a morpheme unit. When the corresponding morpheme includes a single morpheme, the morpheme may be a representative morpheme for the corresponding word, and when the corresponding word includes a plurality of morphemes, the first morpheme of the corresponding word is designated as the representative morpheme of the corresponding word. Decide. For example, if the morpheme analysis of the word 'Gyeongshimi' shown in FIG. 4 is performed, it can be analyzed as two morphemes of 'Gyeongshimi/NNP' and 'Eun/JX'. In this case, the 'Gyeongsimi/NNP' may correspond to a representative morpheme of the word 'Gyeongsimi' according to the first expression rule, that is, a conversion result according to the first expression rule.

Meanwhile, by embedding a word converted according to the first expression rule, a first embedding value for the corresponding word may be derived. Specifically, an embedding module stored in the DB 1400 may be used to embed a word converted according to the first expression rule, and more specifically, an embedding module that embeds a word converted according to the first expression rule is a large-capacity sentence. It may correspond to a pretrained model using the word2vec method using a corpus as training data. Preferably, in the case of targeting Korean-based input sentences shown in FIG. 1, training data may correspond to a corpus composed of Korean sentences.

As shown in FIG. 4, in one embodiment of the present invention, the plurality of expression rules include a second expression rule for expressing the corresponding word in syllable units, and the word embedding value deriving step (S10) comprises: One or more syllable embedding values may be derived by embedding each of one or more syllables included in the corresponding word according to the second expression rule, and a second embedding value may be derived by merging the one or more syllable embedding values.

Specifically, the second expression rule corresponds to an expression rule for converting words into syllable units. The second expression rule corresponds to an element for solving a problem in which dependent syntax analysis for an input sentence or a corresponding word is not normally performed when a corresponding word or a word included in the corresponding word is out of vocabulary. . For example, the word 'gyeongsimi' shown in FIG. 4 can be converted into four syllables 'gyeong', 'sim', 'i' and 'eun' according to the second expression rule, and each syllable A second embedding value according to the second expression rule can be derived by embedding and merging each of the representations for .

More specifically, an embedding module stored in the DB 1400 may be used to embed each of the plurality of syllables converted in syllable units according to the second expression rule, and each of the syllables converted according to the second expression rule may be embedded. The embedding module may correspond to a convolutional neural network model learned in units of syllables using a large-capacity sentence corpus as training data. Preferably, in the case of targeting Korean-based input sentences shown in FIG. 1, training data may correspond to a corpus composed of Korean sentences.

As shown in FIG. 4 , in one embodiment of the present invention, the plurality of expression rules, each of one or more morphemes of the corresponding word based on the first part of speech tag information preset for the language of the input sentence. A third expression rule expressed as a part-of-speech for is included, and the word embedding value derivation step (S10) derives a third embedding value by embedding the part-of-speech of the morpheme for the corresponding word according to the third expression rule. And, when the corresponding word includes a plurality of morphemes, a third embedding value may be derived by embedding the part-of-speech of the first morpheme of the corresponding word.

Specifically, the third expression rule corresponds to an expression rule for tagging a specific POS tag according to the first POS tag information to a corresponding word according to the first POS tag information preset for the language of the input sentence. Meanwhile, the first part-of-speech tag information corresponds to a Language-Specific part-of-Speech Tag (XPoS), and the first part-of-speech tag information may be configured differently according to the language of the input sentence. For example, when an input sentence is Korean, the first Poomsae tag information may include a 'Sejong Poomsae tag' established in the 21st century Sejong Plan.

The third expression rule is an expression rule for using the morpheme itself of the corresponding word as a feature, and, as in the above-mentioned first expression rule, when the corresponding word includes a single morpheme, it is a part-of-speech for the corresponding morpheme. and, when the corresponding word includes a plurality of morphemes, a first morpheme among the plurality of morphemes may be converted into a part of speech tag corresponding to the representative morpheme based on the first part of speech tag information as a representative morpheme of the corresponding word. For example, 'NNP' corresponding to the Sejong part-of-speech tag of 'Gyeongshimi', which is a representative morpheme of the word 'Gyeongsimi' shown in FIG. 4, may correspond to the conversion result according to the third expression rule.

Meanwhile, a third embedding value for a corresponding word may be derived by embedding a specific part-of-speech tag converted according to the third expression rule. Specifically, an embedding module stored in the DB 1400 may be used to embed a specific part-of-speech tag converted according to the third expression rule, and more specifically, an embedding module for embedding a word converted according to the third expression rule Conventional embedding modules of various methods for converting normal information into vectors may be used, and in another embodiment of the present invention, the embedding module may be the same as the embedding module in the first expression rule.

As shown in FIG. 4 , in one embodiment of the present invention, the plurality of expression rules are converted into a part-of-speech for a corresponding word based on second part-of-speech tag information preset to express a plurality of languages in the same form. In the word embedding value derivation step (S10), a fourth embedding value may be derived by embedding a part-of-speech tag for a corresponding word according to the fourth expression rule.

Specifically, the fourth expression rule corresponds to an expression rule for tagging a specific tag according to second part of speech tag information for a corresponding word according to preset second part of speech tag information in order to express various languages in a general form. On the other hand, the second product tag information corresponds to Universal Part-of-Speech Tag (UPoS), and UPoS is The Universal Dependency Treebank for use in pan-linguistic annotation by tying Stanford's dependency tag set and Google POS system together. Corresponds to Project, and corresponds to tag information created by converting various language resources of each language into one unified format (CoNLL U-Format). UPoS includes information on whether a corresponding word is an adverb or an adjective.

Since the UPoS-based second part-of-speech tag information is not a part-of-speech in morpheme units, but a part-of-speech in phrase (phrase) units, information tagged in units of morphemes by the above-described XPoS-based first part-of-speech tag information is applied to the fourth expression rule. Based on this, a process of converting to UPoS form is required. An embodiment of the fourth expression rule corresponding to this conversion rule is shown in FIG. 5 .

As shown in FIG. 5, the fourth expression rule corresponds to an expression rule for converting a word converted by the third expression rule into UPoS based on the second part of speech tag information. Specifically, the word converted into a specific POS tag included in the first POS tag information by the third expression rule is included in the second POS tag information (UPoS in FIG. 5) according to the fourth expression rule shown in FIG. It can be converted to a specific part of speech. For example, if the word 'Gyeongshimi' shown in FIG. 4 is converted according to the third expression rule, it can be converted into a part-of-speech tag 'NNP'. It can be converted to 'PROPN' according to the expression rules.

Meanwhile, a fourth embedding value for a corresponding word may be derived by embedding a specific tag converted according to the fourth expression rule. Specifically, an embedding module stored in the DB 1400 may be used to embed a specific part-of-speech tag converted according to the fourth expression rule, and more specifically, an embedding module for embedding a word converted according to the fourth expression rule Conventional embedding modules of various methods for converting normal information into vectors may be used, and in another embodiment of the present invention, the embedding module may be the same as the embedding module in the first expression rule.

As shown in FIG. 4 , in one embodiment of the present invention, the plurality of expression rules determine a part-of-speech for each of one or more morphemes included in a corresponding word based on preset third part-of-speech tag information. A fifth expression rule expressed by a word tag generated by merging is included, and the step of deriving the word embedding value (S10) derives a fifth embedding value by embedding the word tag for the corresponding word according to the fifth expression rule. can do.

Specifically, the fifth expression rule (Phrase PoS) converts a plurality of morphemes included in each word into a new single word tag for each word, and third part of speech tag information for the corresponding word according to preset third part of speech tag information. It corresponds to the expression rule for tagging with a single word tag according to . The fifth expression rule corresponds to an expression rule in consideration of the linguistic feature that one word consists of one or more morphemes in the case of Korean.

Accordingly, in the fifth expression rule, the maximum number of morphemes included in one word is set to a predetermined value, and the corresponding word is converted into a specific form according to the number of morphemes of the corresponding word. Preferably, the maximum number of morphemes in the fifth expression rule may be preset to four. That is, the fifth expression rule, as in the above-mentioned first expression rule, analyzes the morphemes of the corresponding word, and based on the analysis result of the number of one or more morphemes included in the corresponding word and each morpheme, the third part of speech tag information is obtained. Corresponds to the expression rule for conversion to a specific included word tag. An embodiment of the fifth expression rule ('Phrase PoS unit word feature' in FIG. 6) corresponding to this conversion rule is shown in FIG. 6.

That is, as shown in FIG. 6 , when a corresponding word is composed of a single morpheme, tag information on the corresponding morpheme may correspond to a word tag according to the fifth expression rule. For example, as shown in FIG. 6, if the word 'age' is morphologically analyzed, it can be displayed as 'age/NNG'. It may correspond to a word tag according to the 5th expression rule for the word 'age'.

If a word is composed of two morphemes, a result of merging tag information for each of the two morphemes may correspond to a word tag according to the fifth expression rule. For example, as shown in FIG. 6, if the morphological analysis of the word 'Gyeongshimi' is performed, it can be displayed as 'Gyeongshimi/NNP' and 'Eun/JK', and since the corresponding word has two words, , 'NNP+JK', which is a combination of tag information on two morphemes, may correspond to a word tag according to the 5th expression rule for the word 'Gyeongsimi'.

On the other hand, when a word consists of three morphemes, a result of merging a total of four tag information by repeatedly adding tag information for a second morpheme may correspond to a word tag according to the fifth expression rule. For example, as shown in FIG. 6, when the word 'your' is morphologically analyzed, it can be displayed as 'you/NP', 's/XSN', and 'of/JKG', and the corresponding word has three Since it has a word, 'NP+XSN+_XSN+JKG', in which the tag information of the second morpheme is added with '_', may correspond to a word tag according to the 5th expression rule for the word 'your'. .

Finally, when a word is composed of four morphemes, which is the preset maximum number of morphemes, a result of merging tag information on the four morphemes may correspond to a word tag according to the fifth expression rule. For example, as shown in FIG. 6, when the word 'five' is morphologically analyzed, it may be displayed as 'five/NR', 'this/VCP', 'da/EF', and './SF'. 'NR+VCP+EF+SF', which is a combination of tag information for each morpheme, may correspond to a word tag according to the fifth expression rule for the word 'five'.

In this way, by embedding the transformed result according to the fifth expression rule, a fifth embedding value for the corresponding word can be derived. Specifically, an embedding module stored in the DB 1400 may be used to embed the word tag converted according to the fifth expression rule, and more specifically, the embedding module for embedding the word tag converted according to the fifth expression rule is usually Conventional embedding modules of various methods for converting the information of h into vectors may be used, and in another embodiment of the present invention, the embedding module may be the same as the embedding module in the first expression rule.

Meanwhile, as described above, each of a plurality of words included in the input sentence is embedded as first to fifth embedding values based on the above-described first to fifth expression rules, and the word embedding value derivation step (S10) may derive a word embedding value for a corresponding word based on the first to fifth embedding values for the corresponding word.

As shown in FIG. 4 , in the step of deriving the word embedding value ( S10 ), the word embedding value may be derived by simply concatenating the first to fifth embedding values derived for the corresponding word. In addition, in another embodiment of the present invention, word embedding values to which different weights are applied to each of the first to fifth embedding values may be derived according to preset weights according to the accuracy of the dependent syntax analysis result according to each expression rule. .

4 to 6, in the present invention, not simply embedding a word according to one expression rule, but a plurality of expression rules, preferably the first embedding value derived according to each of the five expression rules. Since the final word embedding value for the corresponding word is derived by merging the fifth embedding value, an effect of deriving a more accurate dependency syntax analysis result can be achieved by considering various characteristics of the corresponding word.

7 schematically illustrates the process of performing the labeling step (S30) according to an embodiment of the present invention.

7 shows that the word embedding value derived for each word in the labeling step (S30) is input to each of a plurality of encoder cells 1431 included in the encoder 1430, and the output representation for each word outputted is included in the decoder 1440. Labeling results input to and output from each of the plurality of decoder cells 1441 are schematically shown.

In the present invention, the decoder 1440 includes a dominant element for each word with the corresponding word as a dependent element; and the name of the dependency on the corresponding dependent element and governing element; are regarded as one label and the labeling result is output. That is, in the sequence labeling problem, one output is generated for one input, whereas in the dependency syntax analysis, two values are output, a dominant place and a dependent relation name. Therefore, in the labeling step (S30) of the present invention, the sequence labeling problem In order to process, the name of the governing place and the dependent relationship are regarded as one label and labeled, and a detailed description of this will be described later.

ve positional encoding(레이블링#1)과 같이, 각 어절에 대한 지배소를 입력 문장 내에서의 절대적 위치로 표현하고, 이를 의존관계명과 하나의 레이블로 묶어 인코딩한다. 도 7에 도시된 바와 같이, 입력 문장인 '경심이는 금년 나이 스물 다섯이다'에서 입력 문장의 첫 번째 어절인 '경심이는'은 마지막 어절 '다섯이다'를 지배소로 하며 의존관계명은 'NP_SBJ'에 해당한다. 한편, 종래의 Na

ve positional encoding 방식의 경우 입력 문장에서의 각 어절을 절대적 위치 값으로 하므로, 지배소 '다섯이다'의 입력 문장에서의 절대적 위치 값인 '5'와 의존관계명 'NP_SBJ'를 병합한 '5+NP_SBJ'를 레이블로 인코딩한다.First, in the conventional dependent syntactic analysis method, the dominant place of an input sentence is expressed using an absolute position in the input sentence. For example, Na shown in FIG. 7

As with ve positional encoding (labeling #1), the dominant position for each word is expressed as an absolute position in the input sentence, and it is encoded by binding it with a dependency name and a label. As shown in FIG. 7, in the input sentence 'Kyungshim is twenty-five this year', the first word of the input sentence 'Kyungshimi' has the last word 'five' as the dominant element, and the dependency relation is 'NP_SBJ'. ' corresponds to. On the other hand, conventional Na

In the case of the ve positional encoding method, each word in the input sentence is an absolute position value, so '5', the absolute position value of the dominant element 'five' in the input sentence, and '5+NP_SBJ', which is a combination of the dependency name 'NP_SBJ'' as a label.

On the other hand, as shown in Figure 7, in one embodiment of the present invention, the labeling step (S30), in the plurality of words included in the input sentence, the position of the word corresponding to the corresponding output representation and the corresponding output representation distance information between the positions of the ruling places for the words corresponding to ; and a name of a dependency relationship with the governing station.

Specifically, in the present invention, unlike conventional methods, relative distances and names of dependent relationships in an input sentence of dependent and dominant elements are regarded as one label and labeled. For example, as in Relative positional encoding (labeling #2) shown in FIG. 7, the first word of the input sentence, 'Kyungsimi', is in the first position of the input sentence, and the dominant of 'Kyungsimi'. Since 'five' is in the fifth position of the input sentence, '4+NP_SBJ', which is a combination of '4', the position difference value between the corresponding words, and 'NP_SBJ', the dependency name, is encoded as a label.

In addition, as shown in FIG. 7, in one embodiment of the present invention, the labeling step (S30) includes the position of the word corresponding to the specific output representation and the specific output among a plurality of words included in the input sentence. Distance information considering the POS tag for the dominant point between the positions of the dominant point for the word corresponding to the representation; and a name of a dependency relationship with the governing station.

In the present invention, in addition to the above-described relative positional encoding method, a method of labeling a relative distance between dependent and dominant elements in an input sentence and a dependency relation name may be used as another method. Specifically, in another embodiment of the present invention, the labeling step (S30) utilizes the POS tag information for the dominant station, and determines the location of the dominant station to the relative distance and the dominant station to the dependent station based on the corresponding POM tag information. It is expressed as the part-of-speech tag information for the product, and it is labeled by considering it as one label combined with the name of the dependency relationship.

For example, as shown in FIG. 7 , a part-of-speech tag may be tagged for each dependent element and dominant element of an input sentence. As the expression rule to which the part-of-speech tag is tagged, either the third expression rule or the fourth expression rule described above may be used.

In the case of using the third expression rule, morphemes are analyzed for each word and a part-of-speech tag for the first morpheme is tagged. That is, when the third expression rule constructs the POS tag for the dependent and dominant points of the input sentence, 'Gyeongshimi' is a POS tag of 'NNP', 'this year' is a POS tag of 'NNG', 'Age' can be converted into a POS tag of 'NNG', 'twenty' can be converted into a POS tag of 'NR', and 'five' can be converted into a POS tag of 'NR'.

In this case, like XPoS of Relative PoS-based encoding (labeling #3) shown in FIG. The relative distance between (dependent elements and dominant elements) is expressed as the number of words among words that have the same POS tag as the dominant element ('it is five') based on the word to be labeled ('Gyeongsimi'). In FIG. 7, since the word having a part-of-speech tag of 'NR' is 'twenty' in the fourth position and 'five' in the fifth position, the relative distance between the dependent element 'Gyeongsimi' and the dominant element '+' 2', and thus the dependency 'hard core' can be encoded as '+2+NR+NP_SBJ' as a label.

Meanwhile, in the case of using the fourth expression rule, as described above, the part-of-speech converted by the third expression rule is converted into a specific part-of-tag included in the second part-tag information of the fourth expression rule. In other words, the dominant element 'five' for the word 'Kyungshimi' can be converted into a part-of-speech tag 'NOUN' according to the fourth expression rule.

In this case, like the UPoS of Relative PoS-based encoding (labeling #3) shown in FIG. The relative distance between (dependent elements and dominant elements) is expressed as the number of words among words that have the same POS tag as the dominant element ('it is five') based on the word to be labeled ('Gyeongsimi'). In FIG. 7, since there are three words having a part-of-speech tag of 'NOUN', 'this year', 'age', and 'five', the relative distance between the dependent element 'Kyungshimeun' and the dominant element 'five' is '+3'. , and thus the dependent element 'gyeongshimeun' can be encoded as '+3 + NOUN + NP_SBJ' as a label.

As such, in the present invention, as in the test result of dependency syntactic analysis described later, by labeling a value for a relative distance between a dependent element and a dominant element, rather than labeling a value for the absolute position of a dominant element in an input sentence, Compared to the method of , it can exert an effect of deriving improved prediction results for dependent syntax analysis.

8 schematically illustrates a framework to which an encoder 1430 and a decoder 1440 are connected according to an embodiment of the present invention.

As shown in FIG. 8, the dependent syntax analysis for each word included in the input sentence is performed by using the encoder 1430 in the output representation step (S20) and the decoder 1440 in the labeling step (S30). More specifically, in the present invention, dependency syntax analysis on an input sentence may be performed using a framework to which the encoder 1430 and the decoder 1440 are connected.

In addition, as shown in FIG. 8, in one embodiment of the present invention, the encoder 1430 has a Bidirectional LSTM (BLSTM) structure, the decoder 1440 has a LSTM structure, and a plurality of encoder cells 1431 ) each receives a hidden representation from the encoder cell 1431 adjacent to the first direction of the corresponding encoder cell 1431 and the encoder cell 1431 adjacent to the second direction of the corresponding encoder cell 1431, and the word embedding value and An output representation for a corresponding word embedding value may be output based on a plurality of hidden representations.

The encoder 1430 includes a plurality of encoder cells 1431, and each of the plurality of encoder cells 1431 sequentially proceeds in a first direction (sentence progression direction of an input sentence) and a second direction (sentence progression direction of an input sentence). backward), and the last encoder cell 1431 is connected to the first decoder cell 1441 among the plurality of decoder cells 1441 included in the decoder 1440, so that the first decoder cell 1441 A hidden representation can be provided.

The encoder 1430 is composed of a bidirectional LSTM, and the bidirectional LSTM will be described in detail with reference to FIG. 10 . Meanwhile, each of the encoder cells 1431 sequentially receives the word embedding value of each word according to the sentence order of the input sentence. In another embodiment of the present invention, the first encoder cell 1431 of the encoder 1430 may receive a word embedding value for the start token ($) of the input sentence. Meanwhile, each of the plurality of encoder cells 1431 may derive an output representation (S ₀ to S ₅ in FIG. 8) for the input word embedding value.

Meanwhile, the encoder 1430 further includes an attention module 1432 that receives a plurality of output representations from each of the plurality of encoder cells 1431, and the attention module 1432 receives the plurality of outputs. A plurality of attention weights input to each of the plurality of decoder cells 1441 may be derived based on the representation.

Specifically, the encoder 1430 further includes an attention module 1432, and the output representation output from each of the encoder cells 1431 is not only input to the decoder cell 1441, but also the attention module 1432. can be entered into The attention module 1432 receives output representations from each of the encoder cells 1431, derives and provides different attention weights for each decoder cell 1441, and the attention weights cause each decoder cell 1441 to input all Focus on a specific element (word) in a sentence. Through this, in the word embedding value derivation step (S10), the performance of the dependent syntax analysis result that may be caused by the loss of information about the word that may occur as it is expressed as the same size of the word embedding value regardless of the length of the word is reduced. can be improved

Meanwhile, the decoder 1440 includes a plurality of decoder cells 1441, and each of the plurality of decoder cells 1441 is sequentially connected in a first direction (sentence progression direction of an input sentence), and the encoder It consists of a one-way LSTM accompanied by the attention module 1432 included in (1430). The unidirectional LSTM will be described in detail with reference to FIG. 9 .

Each of the plurality of decoder cells 1441 included in the decoder 1440 sequentially receives an output representation for each word according to the sentence order of the input sentence. More specifically, each decoder cell 1441 does not receive only the output representation of the corresponding word corresponding to the sentence order, but the output representation of the corresponding word and the output representation of the adjacent word before and after the corresponding word are merged. Input information can be entered. On the other hand, in the case of the first decoder cell 1441, the output representation of the first word of the input sentence, the output representation of the second word adjacent to the first word, and the word adjacent to the first word in front of the first word do not exist. , input information in which the zero-padded values are merged is received. On the other hand, each decoder cell 1441 may output a labeling result for a dominant location and a dependent relation name for a corresponding word based on the received input information.

9 schematically illustrates the structure of a unidirectional LSTM configured in the decoder 1440 according to an embodiment of the present invention.

The neural network model shown in (A) of FIG. 9 corresponds to a diagram schematically showing the overall configuration of the Long-Short Term Memory (LSTM) neural network constituting the decoder 1440, and (B) in FIG. Corresponds to a diagram schematically showing one cell unit of , that is, one decoder cell 1441 .

As shown in (A) of FIG. 9, the LSTM neural network corresponds to a type of RNN and is suitable for processing sequence data in which values in the previous sequence may affect values in the next sequence. As shown in (A) of FIG. 9, the LSTM neural network includes a plurality of cell units, and each of the plurality of cell units is sequentially connected.

Values included in the sequence data are sequentially input to each of the sequentially connected cell units. For example, the X _t-1th value included _in the sequence data is input to the cell unit shown on the left of (A) of FIG. and the X _t+1th value included in the sequence data is input to the cell unit shown on the right. Here, the sequence data may correspond to input information obtained by merging the output representation of the word embedding value for the corresponding word and the output representation of the word embedding value adjacent to the corresponding word.

Meanwhile, the cell unit additionally receives a cell state value and a hidden state value output from the previous cell unit. For example, in the case of the cell unit shown in the center of FIG. 9 (A), the cell state value C _t-1 and the hidden state value h _t-1 output from the cell unit shown on the left are additionally input.

In this way, the cell unit uses the input value of the sequence data corresponding to the corresponding cell unit and the cell state value and hidden state value output from the previous cell unit to obtain the cell state value of the previous cell unit and the sequence input to the corresponding cell unit. It determines how much the input value of the data will be reflected, outputs the cell state value in the cell unit, and outputs the value obtained by filtering the input value of the sequence data input to the cell unit with the output cell state value to the corresponding cell unit. It is output as a hidden state value and an output value (feature value).

On the other hand, the cell state value and hidden state value output from the corresponding cell unit are input to the next cell unit, and in this way, each cell unit of the LSTM neural network reflects the output information of the previous cell unit to obtain output information from its own cell unit. Since it calculates, it corresponds to a neural network model suitable for processing sequentially correlated sequence data.

9(B) schematically shows the detailed configuration of a cell unit of an LSTM neural network.

As shown in (B) of FIG. 9, σ means a sigmoid function, tanh means a hyperbolic tangent function, and [Equation 1] and [Equation 1] below 2] Represents a sigmoid function and a hyperbolic tangent function, respectively, and 'x' and '+' mean pointwise multiplication and addition operations.

[Equation 1]

[Equation 2]

Meanwhile, f _t shown in (B) of FIG. 9 corresponds to a factor determining how much the previous cell state value C _t-1 is to be reflected, and i _t and corresponds to a factor to be updated to C _t , which is a cell state value to be output, and O _t corresponds to a factor for calculating h _t corresponding to an output value (feature value) and a hidden state value. Each of the factors described above may be expressed according to [Equation 3] to [Equation 8] below.

[Equation 3]

[Equation 4]

[Equation 5]

[Equation 6]

[Equation 7]

[Equation 8]

( Is second input value A weight vector value for Is th hidden state value, is the bias)

In this way, each cell unit in the LSTM neural network receives the cell state value C _t-1 and the hidden state value h _t-1 output from the previous cell unit, and the cell state value for X _t input to the cell unit Since C _t and hidden state value h _t are output, words are sequentially formed, and based on the output value h _t , in the labeling step (S30), control points and dependent names of words are labeled, so sequential It is possible to effectively derive feature values for text data in which there is an association between words connected by .

The above-described LSTM neural network may be included in the decoder 1440, and in another embodiment of the present invention, the decoder 1440 is an LSTM to which a peephole connection is added to the cell unit of the LSTM neural network shown in FIG. Like a neural network, it may include an LSTM neural network with additional elements added to the basic LSTM neural network structure.

10 schematically illustrates the structure of a bidirectional LSTM configured in the encoder 1430 according to an embodiment of the present invention.

The diagram shown in FIG. 10 corresponds to a diagram schematically showing the overall configuration of a Bidirectional LSTM (BLSTM) neural network. The BLSTM neural network is also a type of RNN, and has a structure in which two LSTM neural networks are connected.

Specifically, as shown in FIG. 10, in the first LSTM located at the top, as described in FIG. 9, sequence data having an order (Input[0] to Input[t] in FIG. 10) are sequentially connected to each cell unit. Cell state values (c[0] to c[t-1] in FIG. 10) and hidden state values (h[0] to h[ in FIG. 10) sequentially input along the first direction and updated in the previous cell unit t-1]) is input to output the first output representation (feature value). That is, in the 1st LSTM, the first output representation for the input value of the sequence data is derived by considering the cell state value and the hidden state value in the previous cell unit according to the forward direction of the sequence data.

On the other hand, in the second LSTM located at the lower end, a plurality of cell units are connected in the opposite order (second direction) to the above-described first LSTM, and each cell unit has sequential data (Input[t] to Input[ in FIG. 10). 0]) are sequentially input, and cell state values (c'[0] to c'[t-1] in FIG. 10) and hidden state values (h'[0] to h'[t-1]) is input and the second output representation (feature value) is output. That is, in the 2LSTM, the second output representation for the input value of the sequence data is derived by considering the cell state value and the hidden state value of the preceding cell unit according to the reverse direction of the sequence data.

On the other hand, the BLSTM neural network considers the first output representation and the second output representation output from each cell unit that receives the input value of the same sequence data in the 1st LSTM and the 2nd LSTM to obtain the final output representation (output[0 in FIG. 10] to output[t]). For example, the output representation that is finally output may be simply derived by combining the feature value output from the cell unit of the 1st SLTM and the feature value output from the cell unit of the 2nd LSTM, or the feature value output from the cell unit of the 1st LSTM. And a final feature value may be derived by assigning a predetermined weight to each feature value output from the second LSTM cell unit. In addition, in another embodiment of the present invention, the final output representation for the word may be output in consideration of the first output representation and the second output representation using the conventional SoftMax function.

As such, while the LSTM neural network shown in FIG. 9 has a structure that learns according to the forward direction of sequence data, the BLSTM neural network shown in FIG. 10 corresponds to a structure that learns by considering both the forward and backward directions of sequence data. A neural network may be included in the aforementioned encoder 1430.

Meanwhile, the number of cell units included in each neural network model in FIGS. 9 to 10 described above may correspond to the number of input values of sequence data. For example, if the sequence data is the word embedding value for each word included in the input sentence, and the input sentence contains five words, each neural network model additionally considers five cell units or sentence start tokens and six A number of cell units may be included.

Meanwhile, [Table 1] below schematically shows the performance change of the dependency segmentation analysis result according to the number of a plurality of expression rules of the present invention.

[Table 1]

As in [Table 1], when the word embedding value for each word derived using a specific single expression rule is used, when the word embedding value for each word derived using a plurality of expression rules is used Indicates the performance of dependent parsing results. Specifically, the first experimental result uses the word embedding value derived using only the third expression rule (XPoS), and the second experimental result uses the word embedding value derived using only the fifth expression rule (Phrase PoS). In the case of using, the third experimental result is when word embedding values derived using the third and fifth expression rules are used, and the fourth experimental result is derived using the third and fourth expression rules. When word embedding values are used, the fifth experimental result is when word embedding values derived using the 4th expression rule and the 5th expression rule are used, and finally, the sixth experimental result is the 3rd to 5th expression This corresponds to the case of using the word embedding value derived by using all the rules.

As shown in [Table 1], compared to using word embedding values derived by embedding values converted according to a single expression rule, a plurality of expressions derived by variously using a plurality of expression rules described in the present invention. It can be seen that a higher performance was achieved in the case of using the word embedding value generated by merging the embedding values of .

In addition, [Table 2] below schematically shows the performance change of the dependency syntax analysis result according to the labeling method in consideration of the relative distances of the dependent and dominant elements described in the present invention.

[Table 2]

As shown in [Table 2], when the above-described labeling method applying the absolute position of dominant elements in the conventional input sentence is used, when the labeling method applying the relative distance between dependent elements and dominant elements in the input sentence is used Indicates the performance of the dependence parsing result.

As shown in [Table 2], the performance of the dependent syntax analysis result when the labeling method using the relative distance between dependent and dominant points in the present invention is used compared to the conventional labeling method using the absolute location of the dominant points. was measured higher, and among the detailed labeling methods applying relative distance, it can be confirmed that the labeling method (UPoS in Table 2) using the relative distance using the POM tag of the governing station according to the 4th expression rule recorded the highest performance. there is.

As can be seen in the above [Table 1] and [Table 2], the encoder 1430 inputs the word embedding value obtained by embedding and merging the words converted according to the plurality of expression rules for each word of the input sentence. In the case of labeling a label to which a relative distance between dependent and dominant elements in a sentence is applied, the performance of a dependent syntactic analysis result for an input sentence can be improved compared to conventional methods.

11 schematically illustrates the internal configuration of a computing device according to an embodiment of the present invention.

The computing device 1000 illustrated in FIG. 1 described above may include components of the computing device 11000 illustrated in FIG. 11 .

As shown in FIG. 11, a computing device 11000 includes at least one processor 11100, a memory 11200, a peripheral interface 11300, an input/output subsystem ( It may include at least an I/O subsystem (11400), a power circuit (11500), and a communication circuit (11600). In this case, the computing device 11000 may correspond to the computing device 1000 shown in FIG. 1 .

The memory 11200 may include, for example, high-speed random access memory, magnetic disk, SRAM, DRAM, ROM, flash memory, or non-volatile memory. . The memory 11200 may include a software module, a command set, or other various data necessary for the operation of the computing device 11000.

In this case, access to the memory 11200 from other components, such as the processor 11100 or the peripheral device interface 11300, may be controlled by the processor 11100.

Peripheral interface 11300 may couple input and/or output peripherals of computing device 11000 to processor 11100 and memory 11200 . The processor 11100 may execute various functions for the computing device 11000 and process data by executing software modules or command sets stored in the memory 11200 .

The input/output subsystem can couple various input/output peripherals to peripheral interface 11300. For example, the input/output subsystem may include a controller for coupling a peripheral device such as a monitor, keyboard, mouse, printer, or touch screen or sensor to the peripheral device interface 11300 as needed. According to another aspect, input/output peripherals may be coupled to the peripheral interface 11300 without going through the input/output subsystem.

The power circuit 11500 may supply power to all or some of the terminal's components. For example, power circuit 11500 may include a power management system, one or more power sources such as a battery or alternating current (AC), a charging system, a power failure detection circuit, a power converter or inverter, a power status indicator or power It may contain any other components for creation, management and distribution.

The communication circuit 11600 may enable communication with another computing device using at least one external port.

Alternatively, as described above, the communication circuit 11600 may include an RF circuit and transmit/receive an RF signal, also known as an electromagnetic signal, to enable communication with other computing devices.

The embodiment of FIG. 11 is just one example of the computing device 11000, and the computing device 11000 may omit some components shown in FIG. 11, further include additional components not shown in FIG. It may have a configuration or arrangement combining two or more components. For example, a computing device for a communication terminal in a mobile environment may further include a touch screen or a sensor in addition to the components shown in FIG. , Bluetooth, NFC, Zigbee, etc.) may include a circuit for RF communication. Components that may be included in the computing device 11000 may be implemented as hardware including one or more signal processing or application-specific integrated circuits, software, or a combination of both hardware and software.

Methods according to embodiments of the present invention may be implemented in the form of program instructions that can be executed through various computing devices and recorded in computer readable media. In particular, the program according to the present embodiment may be composed of a PC-based program or a mobile terminal-specific application. An application to which the present invention is applied may be installed in the computing device 11000 through a file provided by a file distribution system. For example, the file distribution system may include a file transmission unit (not shown) that transmits the file according to a request of the computing device 11000 .

The device described above may be implemented as a hardware component, a software component, and/or a combination of hardware components and software components. For example, devices and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA) , a programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions. The processing device may run an operating system (OS) and one or more software applications running on the operating system. A processing device may also access, store, manipulate, process, and generate data in response to execution of software. For convenience of understanding, there are cases in which one processing device is used, but those skilled in the art will understand that the processing device includes a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that it can include. For example, a processing device may include a plurality of processors or a processor and a controller. Other processing configurations are also possible, such as parallel processors.

Software may include a computer program, code, instructions, or a combination of one or more of the foregoing, which configures a processing device to operate as desired or processes independently or collectively. The device can be commanded. Software and/or data may be any tangible machine, component, physical device, virtual equipment, computer storage medium or device, intended to be interpreted by or to provide instructions or data to a processing device. , or may be permanently or temporarily embodied in a transmitted signal wave. Software may be distributed on networked computing devices and stored or executed in a distributed manner. Software and data may be stored on one or more computer readable media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer readable medium. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. Program commands recorded on the medium may be specially designed and configured for the embodiment or may be known and usable to those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. - includes hardware devices specially configured to store and execute program instructions, such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of program instructions include high-level language codes that can be executed by a computer using an interpreter, as well as machine language codes such as those produced by a compiler. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

According to an embodiment of the present invention, in the word embedding value derivation step, a plurality of embedding values are derived for each word using a plurality of expression rules, and the word embedding value for each word generated by merging the plurality of embedding values is sequenced. By inputting to the encoder of the framework for performing the labeling-based dependency syntax analysis method, the prediction result of the dependency syntax analysis can be improved compared to the prediction result according to the conventional method.

According to an embodiment of the present invention, each of the plurality of decoder cells included in the decoder additionally receives an attention weight from the attention module included in the encoder and performs labeling on the word, so that the prediction result of dependency syntax analysis is obtained in the conventional case. Compared to the prediction result according to the method, an improvement effect can be exerted.

According to one embodiment of the present invention, in the labeling step, in order to label the dominant location for each word and the dependent relationship name with the corresponding dominant location as one label, the dependent relationship name and the relative distance between the dependent and dominant location are determined. By labeling with paired labels, an effect capable of achieving high performance compared to conventional methods can be exerted.

As described above, although the embodiments have been described with limited examples and drawings, those skilled in the art can make various modifications and variations from the above description. For example, the described techniques may be performed in an order different from the method described, and/or components of the described system, structure, device, circuit, etc. may be combined or combined in a different form than the method described, or other components may be used. Or even if it is replaced or substituted by equivalents, appropriate results can be achieved.

Therefore, other implementations, other embodiments, and equivalents of the claims are within the scope of the following claims.

Claims (11)

A dependent syntax analysis method based on sequence labeling for an input sentence performed in a computing device including one or more processors and one or more memories,
The word embedding value derivation unit receives an input sentence including a plurality of words, merges a plurality of embedding values derived according to each of a plurality of preset expression rules for each of the plurality of words, and obtains a value for each of the plurality of words. A word embedding value derivation step of deriving a word embedding value for a corresponding word by deriving an embedding value for each expression rule and merging the embedding values for each expression rule;
The output representation output unit inputs each of a plurality of word embedding values according to the word order in the input sentence to each of the encoder cells through an encoder including a plurality of encoder cells, and obtains each of the plurality of word embedding values. an output representation output step of outputting an output representation for; and
Through a decoder including a plurality of decoder cells, the labeling unit transmits input information obtained by merging each of a plurality of output representations according to the order of words in the input sentence and one or more output representations adjacent to the corresponding output representation to each of the decoders. A labeling step of inputting the input into a cell and labeling the ruling place for the word corresponding to the corresponding output representation and the name of the dependent relationship with the corresponding ruling office;
The word embedding value derivation unit, the output representation output unit, and the labeling unit are included in the computing device,
The plurality of expression rules,
A first expression rule for expressing the corresponding word in units of morphemes;
a second expression rule for expressing the corresponding word in units of syllables;
a third expression rule for expressing a corresponding word as a part of speech tag for each of one or more morphemes of the corresponding word based on first part of speech tag information preset for the language of the input sentence;
a fourth expression rule expressing a part of speech for a corresponding word based on predetermined second part of speech tag information in order to express a plurality of languages in the same form; and
A fifth expression rule for expressing the corresponding word as a single word tag generated by merging the part of speech tags for each of one or more morphemes included in the corresponding word based on the preset third part tag information;
The encoder is composed of a plurality of encoder cells in a bidirectional LSTM (BLSTM) structure, and each of the plurality of encoder cells is a cell unit of an LSTM neural network to which each of the plurality of encoder cells is connected in a first direction and each of the plurality of encoder cells A cell unit of an LSTM neural network connected in a second direction;
Each of the plurality of encoder cells receives a hidden representation from each of the encoder cells adjacent to the first direction of the corresponding encoder cell and the encoder cells adjacent to the second direction of the corresponding encoder cell, and based on the word embedding value and the plurality of hidden representations, corresponding Output the output representation for the word embedding value,
The decoder is composed of an LSTM structure, and includes a dominant element for each word as a dependent element; and the name of the dependency on the dependent and governing stations; are regarded as one label and the labeling result is output,
The labeling step is
distance information between a position of a word corresponding to a specific output representation in a plurality of words included in the input sentence and a position of a dominant location for a word corresponding to the specific output representation considering a POS tag for the dominant location; and a name of a dependency relationship with the dominant locus.
The method of claim 1,
The encoder,
Further comprising an attention module receiving a plurality of output representations from each of the plurality of encoder cells,
The attention module,
A sequence labeling-based dependency syntax analysis method for deriving a plurality of attention weights input to each of the plurality of decoder cells based on the plurality of input output representations.
The method of claim 1,
In the step of deriving the word embedding value,
Deriving a first embedding value by embedding the morpheme of the corresponding word according to the first expression rule, and deriving a first embedding value by embedding the first morpheme of the corresponding word when the corresponding word includes a plurality of morphemes, A dependency parsing method based on sequence labeling.
The method of claim 1,
In the step of deriving the word embedding value,
Dependence based on sequence labeling, wherein one or more syllables included in the corresponding word according to the second expression rule are embedded to derive one or more syllable embedding values, and a second embedding value is derived by merging the one or more syllable embedding values. Parsing method.
The method of claim 1,
In the step of deriving the word embedding value,
A third embedding value is derived by embedding the POS tag of the morpheme of the corresponding word according to the third expression rule, and when the corresponding word includes a plurality of morphemes, the POS tag of the first morpheme of the corresponding word is embedded to derive the third embedding value. A dependency parsing method based on sequence labeling that derives an embedding value.
The method of claim 1,
In the step of deriving the word embedding value,
A sequence labeling-based dependent syntax analysis method of deriving a fourth embedding value by embedding a part-of-speech for a corresponding word according to the fourth expression rule.
The method of claim 1,
In the step of deriving the word embedding value,
A method for analyzing dependency syntax based on sequence labeling in which a fifth embedding value is derived by embedding a word tag for a corresponding word according to the fifth expression rule.
delete delete A computing device including one or more processors and one or more memories that performs a dependent syntax analysis method based on sequence labeling for an input sentence, comprising:
Receiving an input sentence including a plurality of words, deriving an embedding value for each expression rule for each of the plurality of words by merging a plurality of embedding values derived according to each of a plurality of expression rules preset for each of the plurality of words; , a word embedding value deriving unit for deriving a word embedding value for a corresponding word by merging the embedding values for each expression rule;
Through an encoder including a plurality of encoder cells, an output that inputs each of a plurality of word embedding values according to the word order in the input sentence to each of the encoder cells and outputs an output representation for each of the plurality of word embedding values. representation output unit; and
Through a decoder including a plurality of decoder cells, input information obtained by merging each of a plurality of output representations according to the order of words in the input sentence and one or more output representations adjacent to the corresponding output representation is input to each decoder cell, and the corresponding output representation is input to each decoder cell. A labeling unit for labeling the ruling place for the word corresponding to the output representation and the name of the dependency relationship with the corresponding ruling office;
The plurality of expression rules,
A first expression rule for expressing the corresponding word in units of morphemes;
a second expression rule for expressing the corresponding word in units of syllables;
a third expression rule for expressing a corresponding word as a part of speech tag for each of one or more morphemes of the corresponding word based on first part of speech tag information preset for the language of the input sentence;
a fourth expression rule expressing a part of speech for a corresponding word based on predetermined second part of speech tag information in order to express a plurality of languages in the same form; and
A fifth expression rule for expressing the corresponding word as a single word tag generated by merging the part of speech tags for each of one or more morphemes included in the corresponding word based on the preset third part tag information;
The encoder is composed of a plurality of encoder cells in a bidirectional LSTM (BLSTM) structure, and each of the plurality of encoder cells is a cell unit of an LSTM neural network to which each of the plurality of encoder cells is connected in a first direction and each of the plurality of encoder cells A cell unit of an LSTM neural network connected in a second direction;
Each of the plurality of encoder cells receives a hidden representation from each of the encoder cells adjacent to the first direction of the corresponding encoder cell and the encoder cells adjacent to the second direction of the corresponding encoder cell, and based on the word embedding value and the plurality of hidden representations, corresponding Output the output representation for the word embedding value,
The decoder is composed of an LSTM structure, and includes a dominant element for each word as a dependent element; and the name of the dependency on the dependent and governing stations; are regarded as one label and the labeling result is output,
The labeling unit,
distance information between a position of a word corresponding to a specific output representation in a plurality of words included in the input sentence and a position of a dominant location for a word corresponding to the specific output representation considering a POS tag for the dominant location; and a name of a dependency relationship with the governing station.
A computer-readable medium for executing a computer program in a computing device including one or more processors and one or more memories to implement a sequence labeling-based dependent syntax analysis method for an input sentence performed by the computing device, wherein the A computer-readable medium stores instructions for causing the computing device to perform the following steps,
The following steps are
The word embedding value derivation unit receives an input sentence including a plurality of words, merges a plurality of embedding values derived according to each of a plurality of preset expression rules for each of the plurality of words, and obtains a value for each of the plurality of words. A word embedding value derivation step of deriving a word embedding value for a corresponding word by deriving an embedding value for each expression rule and merging the embedding values for each expression rule;
The output representation output unit inputs each of a plurality of word embedding values according to the word order in the input sentence to each of the encoder cells through an encoder including a plurality of encoder cells, and obtains each of the plurality of word embedding values. an output representation output step of outputting an output representation for; and
Through a decoder including a plurality of decoder cells, the labeling unit transmits input information obtained by merging each of a plurality of output representations according to the order of words in the input sentence and one or more output representations adjacent to the corresponding output representation to each of the decoders. A labeling step of inputting the input into a cell and labeling the ruling place for the word corresponding to the corresponding output representation and the name of the dependent relationship with the corresponding ruling office;
The word embedding value derivation unit, the output representation output unit, and the labeling unit are included in the computing device,
The plurality of expression rules,
A first expression rule for expressing the corresponding word in units of morphemes;
a second expression rule for expressing the corresponding word in units of syllables;
a third expression rule for expressing a corresponding word as a part of speech tag for each of one or more morphemes of the corresponding word based on first part of speech tag information preset for the language of the input sentence;
a fourth expression rule expressing a part of speech for a corresponding word based on predetermined second part of speech tag information in order to express a plurality of languages in the same form; and
A fifth expression rule for expressing the corresponding word as a single word tag generated by merging the part of speech tags for each of one or more morphemes included in the corresponding word based on the preset third part tag information;
The encoder is composed of a plurality of encoder cells in a bidirectional LSTM (BLSTM) structure, and each of the plurality of encoder cells is a cell unit of an LSTM neural network to which each of the plurality of encoder cells is connected in a first direction and each of the plurality of encoder cells A cell unit of an LSTM neural network connected in a second direction;
Each of the plurality of encoder cells receives a hidden representation from each of the encoder cells adjacent to the first direction of the corresponding encoder cell and the encoder cells adjacent to the second direction of the corresponding encoder cell, and based on the word embedding value and the plurality of hidden representations, corresponding Output the output representation for the word embedding value,
The decoder is composed of an LSTM structure, and includes a dominant element for each word as a dependent element; and the name of the dependency on the dependent and governing stations; are regarded as one label and the labeling result is output,
The labeling step is
distance information between a position of a word corresponding to a specific output representation in a plurality of words included in the input sentence and a position of a dominant location for a word corresponding to the specific output representation considering a POS tag for the dominant location; and a name of a dependency relationship with the governing station.

Images