KR102564692B1 - System and method for processing natural language using correspondence learning - Google Patents

Abstract

Disclosed is a natural language processing system and method using relational learning. A specific implementation example of this technology is to construct a learning model by generating a structure vector based on each sentence vector of a plurality of collected corpora, matching the generated structure vector with an input corpus, and constructing a learning model for the input corpus. By performing based learning on the built learning model, a number of similar structure vectors having similar sentence structures to the input corpus are estimated, and an answer sentence structure tag string is generated based on the distance function of the estimated similar structure vectors and sentence vectors. An input sentence can be encoded by reflecting the grammatical structure of the sentence, and a natural conversation can proceed accordingly.

Description

Natural language processing system and method using relational learning {SYSTEM AND METHOD FOR PROCESSING NATURAL LANGUAGE USING CORRESPONDENCE LEARNING}

The present invention relates to a natural language processing system and method using relational learning, and more particularly, to a sentence vector of an input sentence and a structure vector having a similar grammatical structure to the input sentence, which are estimated, and the estimated structure vector and sentence vector The present invention relates to a technology enabling real-time encoding of searching for sentences having similar grammatical structures by generating an answer grammar structure tag string as they become closer to each other in a vector space.

Natural language understanding is the process of mechanically analyzing language phenomena spoken by humans and structuring them into a form that computers can understand. Natural language understanding is one of the key technologies used for information extraction and document summarization.

A vector space learning method using mutual correlation, which is the relationship between sentences and meanings, was introduced, which is widely used in various fields of natural language understanding.

However, natural language understanding systems do not have a learning method using a vector space between a sentence and a grammatical structure for word order or sentence structure, which are important factors that determine the meaning of a sentence.

Accordingly, the present applicant models the correlation between a sentence and the grammatical structure of each sentence for a plurality of collected corpora, defines the mutual correlation between a sentence and the grammatical structure of each sentence as a distance in a vector space, I would like to suggest a way to conduct learning

Sangkeun Jung. Semantic vector learning for natural language understanding. Computer Speech & Language, 56:130 -145, 2019.

Therefore, the present invention performs learning modeling on the correlation between sentences and grammatical structures of sentences for a plurality of collected corpora, and correlates correlation between sentences and grammatical structures of each sentence on a vector space. Defined as a distance, learning is performed on the input corpus, then a number of structure vectors having a sentence structure similar to the input corpus are estimated, and a correct sentence structure tag string is generated based on the estimated structure vector and the distance function of the sentence vector. Accordingly, it is intended to provide a natural language processing system and method using relational learning that can perform encoding on an input sentence by reflecting the grammatical structure of the input corpus.

The object of the present invention is not limited to the above-mentioned object, and other objects and advantages of the present invention not mentioned above can be understood by the following description and will be more clearly understood by the examples of the present invention. It will also be readily apparent that the objects and advantages of the present invention may be realized by means of the instrumentalities and combinations thereof set forth in the claims.

A natural language processing system using relational learning according to an embodiment of the present invention,

A grammatical structure reader that expresses the collected corpus in the form of sentences through grammatical structure modeling and preprocessing, separates dependent phrases in the form of sentences into tokens through pre-learning BERT, and generates structure vectors with the embedding values of each token for learning modeling. ;

A text reader that separates tokens from the input corpus through a tokenizer of a pre-learning BERT (Bidirectional Encoder Representations from Transformers) language model and then generates sentence vectors with the embedding values of each token;

a grammar structure writer for generating a similar structure vector by performing learning through the learning modeling on the sentence vector generated by the text reader;

and a learning unit for estimating an answer structure vector based on a distance function between each similar structure vector among a plurality of similar structure vectors of the grammar structure writer and a sentence vector of the text reader.

Preferably, the grammatical structure reader

For a large number of collected sentences, dependencies between words are expressed in sentence form through dependency parsing and preprocessing, and the sentence form of the dependent phrase is separated through BERT tokenizer, and then embedded for each token. a dependent structure reader module that calculates a value and outputs it as a dependent syntax vector;

The phrase structure tree generated as a result of performing multiple phrase structure parsing on the collected sentences is expressed in sentence form, and then the phrase structure sentence form is separated through the BERT tokenizer, and for each token a spherical structure reader module that calculates an embedding value and outputs it as a spherical structure vector; and

Part-of-speech tagging is performed on multiple sentences, parts of speech tags according to each morpheme are expressed in sentence form, and then the morpheme analysis sentence form is separated through BERT tokenizer, and for each token a morpheme analysis reader module that calculates an embedding value and outputs it as a morpheme vector; may include at least one of them.

Preferably, the preprocessing of the grammar structure reader

For each sentence vector of the input text reader, a dependent syntax tree is created through analysis of each dependent syntax, and arc labels, which are the relationships between words in each generated tree, are converted into dependent syntax in the form of a sentence. Sentences can be derived through a pre-learning BERT language model.

Preferably the text reader

BERT talkizer that sequentially separates the input corpus; and

A sentence vector output unit for calculating an embedding value for each token and outputting the calculated embedding value as a sentence vector may be included.

Preferably the learning unit,

a distance cost derivation module for deriving a distance cost between the input sentence vector of the text reader and a similar structure vector of the sentence structure writer; and

a correct grammar structure derivation module for generating a grammatical structure tag string for the sentence vector and the similar structure vector having the smallest distance cost and outputting the generated grammatical structure tag string as a correct grammatical structure tag string; It may be provided to include.

Preferably the learning unit,

A learning controller for controlling learning performance based on the actually measured correct answer grammatical structure tag string and the generated correct answer grammatical structure tag string may be further included.

Preferably the learning controller,

a generation cost calculation module for calculating a generation cost based on correlation entropy between the actually measured correct answer grammatical structure tag sequence and the generated correct answer grammatical structure tag sequence; and

a loss cost calculation module for calculating a loss cost as a sum of the computed generation cost and distance cost; and

A learning performance control module for controlling learning performance based on the loss cost may be further included.

Natural language processing method using relational learning according to another embodiment of the present invention

Grammar structure reading that expresses the collected corpus in the form of sentences through grammatical structure modeling, pre-processes dependent phrases in the form of sentences, separates them into tokens through pre-learning BERT, and generates structure vectors with the embedding values of each token for learning modeling. step;

A text reader step of separating tokens from the input corpus through a tokenizer of a pre-learning BERT (Bidirectional Encoder Representations from Transformers) language model and then generating a sentence vector with an embedding value of each token;

a grammatical structure writing step of generating a similar structure vector by performing learning through the learning modeling on the generated sentence vector; and

and a learning step of estimating an answer structure vector based on a distance function between each similar structure vector among a plurality of similar structure vectors and a sentence vector of a text reader.

Preferably, the preprocessing of the grammatical structure reader,

For each sentence vector of the input text reader, a dependent syntax tree is created through analysis of each dependent syntax, and arc labels, which are the relationships between words in each generated tree, are converted into dependent syntax in the form of a sentence. Sentences can be derived through a pre-learning BERT language model.

Preferably, the learning step,

deriving a distance cost between the input sentence vector of the text reader and the similar structure vector of the sentence structure writer; and generating a grammatical structure tag sequence for the sentence vector and the similar structure vector having the smallest distance cost and outputting the generated grammatical structure tag sequence as a correct answer grammatical structure tag sequence.

Preferably, the learning step,

calculating a generation cost based on correlation entropy between the actually measured correct answer grammatical structure tag string and the generated correct answer grammatical structure tag string; calculating a loss cost as the sum of the calculated generation cost and the distance cost; and controlling learning performance with the loss cost.

According to an embodiment, a sentence reflecting a sentence structure in a vector form in which sentences having a similar sentence structure are arranged in a similar space on a vector space through a method of learning association between each sentence vector and a structure vector for a plurality of collected corpora. A learning model for the expression method is built, and a number of similar structure vectors having a similar sentence structure to the input corpus are estimated by performing based learning on the built learning model for the input corpus, and the estimated similar structure vector and sentence vector are By generating the correct sentence structure tag string based on the distance function, it is possible to encode the input sentence by reflecting the grammatical structure of the input sentence, which has the effect of allowing natural conversation.

The following drawings attached to this specification illustrate preferred embodiments of the present invention, and together with the detailed description of the present invention serve to further understand the technical idea of the present invention, the present invention is the details described in such drawings should not be construed as limited to
1 is a configuration diagram of a natural language processing system using relational learning according to an embodiment.
2 is a detailed configuration diagram of a grammar structure reader of the system according to an embodiment.
3 is an exemplary diagram showing a dependency syntax tree of a system according to an embodiment.
4 is an exemplary view showing the result of analyzing the sphere structure of the system according to an embodiment.
5 is an exemplary diagram showing a POS analysis result of a system according to an embodiment.
6 is a detailed configuration diagram of a text reader of the system according to an embodiment.
7 is a detailed configuration diagram of a learning unit of a system according to an embodiment.

Specific structural or functional descriptions of the embodiments are disclosed for illustrative purposes only, and may be modified and implemented in various forms. Therefore, the embodiments are not limited to the specific disclosed form, and the scope of the present specification includes changes, equivalents, or substitutes included in the technical spirit.

Although terms such as first or second may be used to describe various components, such terms should only be construed for the purpose of distinguishing one component from another. For example, a first element may be termed a second element, and similarly, a second element may be termed a first element.

It should be understood that when an element is referred to as being “connected” to another element, it may be directly connected or connected to the other element, but other elements may exist in the middle.

Singular expressions include plural expressions unless the context clearly dictates otherwise. In this specification, terms such as "comprise" or "have" are intended to designate that the described feature, number, step, operation, component, part, or combination thereof exists, but one or more other features or numbers, It should be understood that the presence or addition of steps, operations, components, parts, or combinations thereof is not precluded.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related art, and unless explicitly defined in this specification, it should not be interpreted in an ideal or excessively formal meaning. don't

Hereinafter, the present invention will be described in detail by describing preferred embodiments of the present invention with reference to the accompanying drawings.

1 is a configuration diagram of a natural language processing system using relational learning according to an embodiment, FIG. 2 is a detailed configuration diagram of the grammar structure reader of FIG. 1, and FIG. 3 is an exemplary diagram showing a dependency syntax tree of FIG. 4 is an exemplary view showing the result of analyzing the sphere structure of FIG. 2, FIG. 5 is an example view showing the POS analysis result of FIG. 2, FIG. 6 is a detailed configuration diagram of the text reader of FIG. 1, and FIG. This is a detailed configuration diagram of the learning unit.

Referring to FIGS. 1 to 7 , the natural language processing system using relational learning according to an embodiment provides sentences having similar sentence structures through a method for learning the association between each sentence vector and structure vector for a plurality of collected corpora. A learning model for the sentence expression method reflecting the vector-type sentence structure to be placed in a similar space on the vector space is built, and based learning is performed on the built learning model for the sentence vector for the input corpus to have a similar sentence structure It has a configuration for estimating a plurality of similar structure vectors and generating a correct sentence structure tag string based on a distance function of each of the estimated plurality of similar structure vectors and sentence vectors. ), a grammar structure writer 300, and a learning unit 400.

Here, the grammatical structure reader 100 is provided to construct a learning model by generating a structure vector for each of a plurality of collected sentence vectors, and thus, as shown in FIG. 2, the grammatical structure reader 100 has a dependent structure ( It may include at least one of a dependency structure reader module 110, a phrase structure reader module 120, and a morpheme structure (POS: Part-Of-Speech) reader module 130.

The dependency structure reader module 110 expresses the dependency relationship between each word in a sentence form through dependency parsing and preprocessing of a plurality of collected sentences, and converts the sentence form of the dependent phrase into a BERT tokenizer. After separation, the embedding value is calculated for each token, output as a dependent syntax vector, and learning modeling is performed with the output dependent syntax vector.

Here, in the preprocessing, a dependency syntax tree is generated through analysis of each dependent syntax for a plurality of collected sentences, arc labels, which are relationships between words in each generated tree, are converted into dependent syntax in the form of a sentence, and the converted dependent syntax is converted. Dependency structure vectors are output through the pre-learning BERT language model for the sentences of the syntax.

As an example, the dependency syntax for the collected corpus “on april first I need a flight from phoenix to san diego” is “prep-prbj-advmod-nsubj-ROOT-det_dobj_aci-prep-pobj-prep- compoung-pobj”.

Here, in the case of the collected corpus, the ETRI-dependent syntax analysis API published by AIOpen is used in the case of Korean, and in the case of English sentences, the Spicy open source library is used. Although not, it should be understood at the level of those skilled in the art.

In addition, the phrase structure reader module 120 expresses the phrase structure tree generated as a result of performing a plurality of phrase structure parsing on the collected sentences in a sentence form, and then converts the phrase structure sentence form into a BERT talk It is separated through a niger, and an embedding value is calculated for each token and output as a sphere structure vector to perform learning modeling.

As an example, the phrase structure for the collected sentence “on april first I need a flight from phoenix to san diego” is shown in FIG. 4, “(S (PP (IN) (NP (NP (NNP)) ( ADVP (RB)))) (NP (PRP)) (VP (VBP) (NP (NP (DT) (NN)) (VP (VBG) (PP (IN) (NP (NNP))) (PP (IN ) (NP (NNP) (NNP))))))”.

Meanwhile, the morpheme structure (POS tagging) reader module 130 performs part-of-speech tagging on a plurality of sentences, expresses the part-of-speech tag according to each morpheme in the form of a sentence, and then the morpheme analysis sentence The shape is separated through the BERT tokenizer, the embedding value is calculated for each token, output as a morpheme vector, and learning modeling is performed based on this.

As an example, the phrase structure for the collected sentence “on april first I need a flight from phoenix to san diego” is “ADP-PROPN-ADV-PRON-VERB-DET-NOUN-VERB-ADP-PROPN- ADP-PROPN-PROPN”.

Meanwhile, the input corpus is transmitted to the text reader 200, and the text reader 200 is provided to generate a sentence vector based on a pre-learning BERT (Bidirectional Encoder Representations from Transformers) language model and preprocessing, and thus the text reader 200 6 includes a BERT talkizer 210 and a sentence vector output unit 220.

Here, in the preprocessing, that is, the grammatical structure writer 300 generates a dependency syntax tree through analysis of each dependent syntax for the input corpus, and arc labels, which are relationships between words in each generated tree, into dependent syntax in the form of a sentence. converted, and the sentences of the converted dependent syntax are transmitted to the pre-learning BERT language model.

Accordingly, the BERT tokenizer 210 of the pre-trained BERT language model sequentially separates input sentences, and then the separated tokens are transmitted to the sentence vector output unit 220. Accordingly, the sentence vector output unit 220 calculates an embedding value for each token and converts the calculated embedding value into a sentence vector. output as

And, the output sentence vector is transmitted to the grammar structure writer 300. The grammar structure writer 300 is a sentence vector of the text reader 100. Sentence vector input by performing learning based on the learning model built for A number of similar structure vectors for outputs

For example, for an input sentence of “Can you tell me the fine dust information today?”, a Wizone syntax tree is output through dependency classification analysis, and the dependent syntax tree structure is as follows.

And, the arc level, which is the relationship between each word in the generated dependent phrase tree, is “NP_AJT-NP-NP_OBJ-VP” in the sentence form of the dependent phrase.

In addition, the dependent syntax in the form of a sentence is processed into a grammatical structure sentence form, and then a pre-learning BERT language model is performed to obtain a final number of similar structure vectors. is derived, and each of the derived similar structure vectors is transmitted to the learning unit 400.

The learning unit 400 generates each structure vector among a plurality of similar structure vectors of the grammar structure writer and the sentence vector of the text reader 100 It has a configuration for estimating the structure vector of the correct answer based on the distance function for , and thus, referring to FIG. 7, the learning unit 400 may include a distance cost derivation module 410 and a correct answer grammar structure derivation module 420. there is.

The distance cost derivation module 410 is a sentence vector of the input text reader 200 and each similar structure vector of the sentence structure writer 300 For, the distance cost as a predetermined cosine distance function derive

That is, the sentence vector with the closest distance in the vector space. and structure vector Syntax structure tag string for is created. in other words, and is derived as a cosine function. Since a series of processes of deriving a distance on a vector space using such a cosine function is similar to a process disclosed in prior literature, a detailed description thereof will be omitted.

Then, the correct answer grammatical structure derivation module 420 generates a grammatical structure tag sequence for the sentence vector and structure vector having the smallest distance cost and outputs the correct grammatical structure tag sequence as the generated grammatical structure tag sequence. Accordingly, natural language understanding is performed on the query sentence entered as the correct grammatical structure tag string, and then a response sentence is generated.

On the other hand, the natural language processing system using relational learning further includes a learning controller for controlling learning performance based on the actually measured answer grammatical structure tag string and the generated grammatical structure tag string, but the learning controller is generated as shown in FIG. It includes a cost calculation module 430, a consumption cost calculation module 440, and a learning performance control module 450.

That is, the generation cost calculation module 430 derives the generation cost by calculating the generation cost with the correlation entropy for the actually measured correct answer grammatical structure tag sequence and the generated grammatical structure tag sequence, and the derived generation cost is the loss cost calculation module 440 ) is transmitted to

The loss cost calculation module 440 calculates a loss cost as the sum of the calculated generation cost and the distance cost, and the calculated loss cost is transmitted to the learning performance control module 450 .

The learning performance control module 450 maximizes learning performance by adjusting learning variables to have a minimum loss cost.

Here, the learning variables are learning parameters of a general Recurrent Neural Network (RNN) language model, and an encoding device learned based on a Recurrent Neural Network (RNN) model includes an input layer and a hidden layer ( It consists of an encoder including a hidden layer and a decoder including a hidden layer and an output layer.

Models trained in the encoder and decoder continue to generate data during the learning process, and after learning is completed, a certain output can be generated by feeding the mathematical function distribution in the output format of the encoder to the decoder. That is, the RNN performs learning by using previous and/or subsequent data in a time-series manner in the neural network learning process. RNNs are based on the idea that an output value at time step t can also be influenced by input values from previous and/or subsequent time steps. For example, in order to fill in the word that most closely matches the blank in the Hangul problem, the context of the words after the blank can be identified based on the sentences preceding the blank. This recurrent neural network structure includes two RNNs in the case of a bidirectional RNN, and an output value is determined depending on the hidden layer of the two RNNs.

Hereinafter, a simulation that evaluates the answer sentence structure tag string derived based on the learning results for the Korean-based weather and nevi corpus and the English-based Atis and snips corpus, respectively, with predetermined evaluation criteria (BLEU and ROUGE) will be described.

Here, the evaluation criterion BLEU is a way to score references and measures how much overlapped ordered pairs are through n_gram, and ROUGE is a way to derive a score using the length of the most overlapping sentences between two sentences according to the number of words.

As an example, for the character vector and phrase vector for the input corpus of "Will you tell me the fine dust information today?", the similar dependency structure vector derived as a learning result is shown in Table 1 below.

[Table 1]

Referring to Table 1, for the corpus of “Can you tell me the fine dust information today?” and for the sentence vector of “NP_AJT_NP_NP_OBJ_VP”, dependent phrases such as “Tell me the fine dust level today”, “Tell me the weather in Dokdo tomorrow”, and “NP_AJT_NP_NP_OBJ_VP” When the dependent phrase vector is transmitted to the learning unit 400, a similar dependent phrase of “Tell me the highest temperature in Gyeongbuk tomorrow” and a dependent phrase vector of “NP_AJT_NP_NP_OBJ_VP” are derived according to the learning result.

Here, the sentence vectors are derived using BERT-base, Multilingual Cased, and BERT _base, Cased pre-learning models for the input Korean-based corpus and English-based corpus.

In addition, the structure vector is derived with BERT-base, Multilingual Cased for the Korean-based corpus, and BERT_base, Cased pre-learning models for the English-based corpus.

The evaluation results based on the predetermined evaluation criteria for these learning results are shown in Table 2 below.

[Table 2]

Referring to Table 2, for the weather, nevi, Atis, and snips corpus, performance is higher than the standard, and in the case of BLEU, it can be seen that the performance is improved the most in 1_gram.

A natural language processing method using relational learning according to another embodiment of the present invention,

Grammar structure reading that expresses the collected corpus in the form of sentences through grammatical structure modeling, pre-processes dependent phrases in the form of sentences, separates them into tokens through pre-learning BERT, and generates structure vectors with the embedding values of each token for learning modeling. step; A text reader step of separating tokens from the input corpus through a tokenizer of a pre-learning BERT (Bidirectional Encoder Representations from Transformers) language model and then generating a sentence vector with an embedding value of each token; a grammatical structure writing step of generating a similar structure vector by performing learning through the learning modeling on the generated sentence vector; and a learning step of estimating a structure vector of an answer based on a distance function for each similar structure vector among a plurality of similar structure vectors and a sentence vector of a text reader, wherein each step of natural language processing using relational learning, This is a function performed by the above-described grammatical structure reader 100, text reader 200, grammatical structure writer 300, and learning unit 400, and detailed reference is omitted.

Accordingly, in one embodiment, a sentence structure in the form of a vector in which sentences having similar sentence structures are arranged in a similar space on a vector space through a method of learning the association between each sentence vector and structure vector for a plurality of collected corpus is collected. A learning model for the reflected sentence expression method is built, and a number of similar structure vectors having a sentence structure similar to the input corpus are estimated by performing based learning based on the built learning model for the input corpus, and the estimated similar structure vector and sentence vector are estimated. By generating the correct sentence structure tag string based on the distance function of , the input sentence can be encoded by reflecting the grammatical structure of the input sentence, and natural conversation can proceed accordingly.

The embodiments described above may be implemented as hardware components, software components, and/or a combination of hardware components and software components. For example, the devices, methods and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate (FPGA). array), programmable logic units (PLUs), microprocessors, or any other device capable of executing and responding to instructions. A processing device may run an operating system (OS) and one or more software applications running on the operating system. A processing device may also access, store, manipulate, process, and generate data in response to execution of software. For convenience of understanding, there are cases in which one processing device is used, but those skilled in the art will understand that the processing device includes a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that it can include. For example, a processing device may include a plurality of processors or a processor and a controller. Other processing configurations are also possible, such as parallel processors.

Software may include a computer program, code, instructions, or a combination of one or more of the foregoing, which configures a processing device to operate as desired or processes independently or collectively. The device can be commanded. Software and/or data may be any tangible machine, component, physical device, virtual equipment, computer storage medium or device, intended to be interpreted by or provide instructions or data to a processing device. , or may be permanently or temporarily embodied in a transmitted signal wave. Software may be distributed on networked computer systems and stored or executed in a distributed manner. Software and data may be stored on one or more computer readable media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer readable medium. Computer readable media may include program instructions, data files, data structures, etc. alone or in combination. Program instructions recorded on a computer readable medium may be specially designed and configured for the embodiment or may be known and usable to those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. Included are hardware devices specially configured to store and execute program instructions, such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of program instructions include high-level language codes that can be executed by a computer using an interpreter, as well as machine language codes such as those produced by a compiler. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

As described above, although the embodiments have been described with limited drawings, those skilled in the art can apply various technical modifications and variations based on the above. For example, the described techniques may be performed in an order different from the method described, and/or components of the described system, structure, device, circuit, etc. may be combined or combined in a different form than the method described, or other components may be used. Or even if it is replaced or substituted by equivalents, appropriate results can be achieved.

100: grammar structure reader
200: text reader
300: grammar structure writer
4300: learning unit

Claims (12)

A grammatical structure reader that expresses the collected corpus in the form of sentences through grammatical structure modeling and preprocessing, separates dependent phrases in the form of sentences into tokens through pre-learning BERT, and generates structure vectors with the embedding values of each token for learning modeling. ;
A text reader that separates tokens from the input corpus through a tokenizer of a pre-learning BERT (Bidirectional Encoder Representations from Transformers) language model and then generates sentence vectors with the embedding values of each token;
a grammar structure writer for generating a similar structure vector by performing learning through the learning modeling on the sentence vector generated by the text reader;
A learning unit for estimating an answer structure vector based on a distance function between each similar structure vector among a plurality of similar structure vectors of the grammar structure writer and a sentence vector of the text reader,
The learning unit,
After calculating the generation cost with the correlation entropy for the actually measured correct answer grammatical structure tag string and the generated correct answer grammatical structure tag string, the calculated generation cost and the input distance cost between the sentence vector of the text reader and the similar structure vector of the sentence structure writer Calculate a loss cost as the sum of , and control the learning performance with the loss cost.
A natural language processing system using relational learning, characterized in that. The method of claim 1, wherein the grammar structure reader
For a large number of collected sentences, dependencies between words are expressed in sentence form through dependency parsing and preprocessing, and the sentence form of the dependent phrase is separated through BERT tokenizer, and then embedded for each token. a dependent structure reader module that calculates a value and outputs it as a dependent syntax vector;
The phrase structure tree generated as a result of performing multiple phrase structure parsing on the collected sentences is expressed in sentence form, and then the phrase structure sentence form is separated through the BERT tokenizer, and for each token a spherical structure reader module that calculates an embedding value and outputs it as a spherical structure vector; and
Part-of-speech tagging is performed on multiple sentences, parts of speech tags according to each morpheme are expressed in sentence form, and then the morpheme analysis sentence form is separated through BERT tokenizer, and for each token a morpheme analysis reader module that calculates an embedding value and outputs it as a morpheme vector; Natural language processing system using relational learning, characterized in that it comprises at least one of. The method of claim 2, wherein the preprocessing of the grammar structure reader
For each sentence vector of the input text reader, a dependent syntax tree is created through analysis of each dependent syntax, and arc labels, which are the relationships between words in each generated tree, are converted into dependent syntax in the form of a sentence. A natural language processing system using relational learning, characterized in that sentences are transmitted to a pre-learning BERT language model. The method of claim 3, wherein the text reader
BERT talkizer that sequentially separates the input corpus; and
A natural language processing system using relational learning, characterized in that a sentence vector output unit for calculating an embedding value for each token and outputting the calculated embedding value as a sentence vector is derived. The method of claim 4, wherein the learning unit,
a distance cost derivation module for deriving a distance cost between the input sentence vector of the text reader and a similar structure vector of the sentence structure writer; and
a correct grammar structure derivation module for generating a grammatical structure tag string for the sentence vector and the similar structure vector having the smallest distance cost and outputting the generated grammatical structure tag string as a correct grammatical structure tag string; Natural language processing system using relational learning, characterized in that it comprises a. The method of claim 5, wherein the learning unit,
A natural language processing system using relational learning, further comprising a learning controller for controlling learning performance based on the actually measured correct answer grammatical structure tag string and the generated correct answer grammatical structure tag string. The method of claim 6, wherein the learning controller,
a generation cost calculation module for calculating a generation cost based on correlation entropy between the actually measured correct answer grammatical structure tag sequence and the generated correct answer grammatical structure tag sequence; and
a loss cost calculation module for calculating a loss cost as a sum of the computed generation cost and distance cost; and
A natural language processing system using relational learning, comprising a learning performance control module for controlling learning performance at the loss cost. In the natural language processing method using relational learning performed based on the natural language processing system using relational learning including the grammatical structure reader, text reader, and learning unit of claim 1,
Grammar structure reading that expresses the collected corpus in the form of sentences through grammatical structure modeling, pre-processes dependent phrases in the form of sentences, separates them into tokens through pre-learning BERT, and generates structure vectors with the embedding values of each token for learning modeling. step;
A text reader step of separating tokens from the input corpus through a tokenizer of a pre-learning BERT (Bidirectional Encoder Representations from Transformers) language model and then generating a sentence vector with an embedding value of each token;
a grammatical structure writing step of generating a similar structure vector by performing learning through the learning modeling on the generated sentence vector; and
A learning step of estimating a structure vector of an answer based on a distance function between each similar structure vector among a plurality of similar structure vectors and a sentence vector of a text reader;
In the learning phase,
After calculating the generation cost with the correlation entropy for the actually measured correct answer grammatical structure tag string and the generated correct answer grammatical structure tag string, the calculated generation cost and the input distance cost between the sentence vector of the text reader and the similar structure vector of the sentence structure writer Calculate the loss cost as the sum of , and control the learning performance with the loss cost.
Natural language processing method using relational learning, characterized in that provided. The method of claim 8, wherein the pretreatment,
For each sentence vector of the input text reader, a dependent syntax tree is generated through analysis of each dependent syntax;
In each generated tree, the arc label arc label, which is the relationship between each word, is converted into a dependent phrase in the form of a sentence,
A natural language processing method using relational learning, characterized in that the sentence of the converted dependent syntax is transmitted to the pre-learning BERT language model. The method of claim 8, wherein the learning step,
deriving a distance cost between the input sentence vector of the text reader and the similar structure vector of the sentence structure writer; and
Generating a grammatical structure tag sequence for the sentence vector and similar structure vector having the smallest distance cost derived and outputting the generated grammatical structure tag sequence as a correct answer grammatical structure tag sequence. Natural language processing methods. The method of claim 10, wherein the learning step,
calculating a generation cost based on correlation entropy between the actually measured correct answer grammatical structure tag string and the generated correct answer grammatical structure tag string;
calculating a loss cost as the sum of the calculated generation cost and the distance cost; and
Natural language processing method using relational learning, characterized in that it further comprises the step of controlling the learning performance at the loss cost. A computer-readable recording medium on which a program for executing the natural language processing method using relational learning according to any one of claims 8 to 11 is recorded.