KR20230044914A - Korean relation extraciton model and method with entity positon information - Google Patents

Abstract

The present invention relates to a technology for predicting relationships between entities which are related to each other among entities existing in Korean sentences or documents. A Korean relationship extraction model according to the embodiment of the present application includes: a preprocessing unit which adds a special token to at least one location before or after the entity included in the Korean input sentence; and a Korean relationship extraction unit which predicts relationships between entities included in the Korean input sentence through the special token. The Korean relationship extraction model according to the present invention can accurately predict relationships between entities.

Description

Korean relationship extraction model and method using entity location information {KOREAN RELATION EXTRACITON MODEL AND METHOD WITH ENTITY POSITON INFORMATION}

The present invention relates to a technique for predicting a relationship between entities that are related to each other among entities existing in Korean sentences or documents.

As unstructured documents become massive, interest in information extraction, which automatically extracts important information as structured information, is increasing. Information extraction technology is an important technology for expanding the knowledge base because it can extract structured information.

Relation extraction, a sub-field of information extraction technology, aims to find entities that are related to each other among entities existing in a given sentence or document and predict the relationship. Research on relationship extraction is mainly conducted based on English, and there is almost no research on relationship extraction targeting Korean.

Moreover, unlike English, Korean has two different characteristics. First, Korean becomes a word with a substantial meaning when it consists of a root and an affix. Second, Korean is a postnuclear language, and predicates are mostly expressed at the end of sentences. Accordingly, in order to know whether a sentence is a declarative sentence or an interrogative sentence, you must check the sentence to the end. Due to these unique characteristics of Korean, there is a problem that the prediction is not accurate when the study on English-based relationship extraction is applied to Korean.

An object of the present invention is to provide a Korean relationship extraction model and a Korean relationship extraction method capable of accurately predicting relationships between entities existing in Korean sentences.

A Korean relationship extraction model according to an embodiment of the present application includes a pre-processor that adds a special token to at least one position before or after an entity included in a Korean input sentence; and a Korean relationship extraction unit that predicts a relationship between entities included in the Korean input sentence through the special token.

In an embodiment, the Korean input sentence includes a first entity and a second entity, and the preprocessor adds first and second special tokens before and after the first entity, respectively, and before the second entity. First and second special tokens are added after and respectively.

In an embodiment, the pre-processor adds a [CLS] token before the Korean input sentence and adds a [SEP] token after the Korean input sentence.

In an embodiment, the Korean language relation extracting unit may include a Korean language model for receiving an input sentence to which a special token is added from the pre-processing unit; an entity recognizer that operates in conjunction with the Korean language model and outputs a hidden state based on the special token; and a relationship classification unit that classifies relationships between entities based on the hidden state.

In an embodiment, the Korean language model is any one of KoBERT, HanBERT, KorBERT, KoELECTRA, and KcELECTRA.

In an embodiment, the entity recognizing unit includes K-EPICs that output hidden stays based on a special token added before an entity among special tokens.

In an embodiment, the entity recognizing unit further includes a K-EPICe that outputs a hidden stay based on a special token added after an entity among special tokens.

In an embodiment, the entity recognizing unit further includes a K-EPICse that outputs a hidden stay based on a special token added to both the front and back of the entity among the special tokens.

In an embodiment, the entity recognition unit further includes a K-EPICv outputting a hidden stay based on the [CLS] token added to the front of the Korean input sentence.

In an embodiment, the relationship classification unit predicts a relationship between entities by applying a softmax to the hidden state.

A Korean relationship extraction method according to an embodiment of the present application includes adding a special token including location information of an entity to a Korean input sentence; outputting a hidden state reflecting entity information based on the special token; and extracting relationships between entities by applying softmax to the hidden state.

In an embodiment, the adding of the special token may include adding the special token to at least one position of the front and back of the entity; and adding tokens before and after the Korean input sentence.

In an embodiment, the outputting of the hidden state outputs the hidden state by applying K-EPICs utilizing a special token added in front of the entity among the special tokens.

In an embodiment, the outputting of the hidden state outputs the hidden state by applying K-EPICe utilizing a special token added to the end of the entity among the special tokens.

In an embodiment, the outputting of the hidden state outputs the hidden state by applying K-EPICse utilizing special tokens added to both the front and back of the entity among the special tokens.

The Korean relationship extraction model and the Korean relationship extraction method according to the present invention add special tokens to Korean sentences to have an output that reflects information about entities well, and predict relationships between entities by using the special tokens. . Through this, the Korean relationship extraction model and the Korean relationship extraction method according to the present invention can accurately predict relationships between entities.

1 is a block diagram showing a Korean relationship extraction model 10 according to an embodiment of the present application.
FIG. 2 is a block diagram showing the Korean relationship extractor 200 of FIG. 1 in more detail.
FIG. 3 is a diagram showing an example of an operation of the pre-processing unit 100 of FIG. 1 .
4 shows an example of an operation related to the K-EPCIv 221 of the entity recognizing unit 220.
5 shows an example of an operation related to the K-EPCIs 222 of the entity recognition unit 220.
6 shows an example of an operation related to the K-EPCIe 223 of the entity recognition unit 220.
7 shows an example of an operation related to the K-EPCIse 224 of the entity recognition unit 220.
FIG. 8 is a diagram showing the results of testing the performance of the Korean relationship extraction model 10 of FIG. 1 in FIG. 1 .
9 is a block diagram showing a Korean relationship extraction model 20 according to another embodiment of the present application.
10 is a diagram showing the operation of the Korean relationship extraction model 20 of FIG. 9 as an example.
11 is a flow chart showing a Korean relationship extraction method according to an embodiment of the present application.
12 is a block diagram illustrating a computing environment including a computing device according to an exemplary embodiment.

Hereinafter, embodiments according to the technical idea of the present application will be described in more detail with reference to the accompanying drawings.

1 is a block diagram showing a Korean relationship extraction model 10 according to an embodiment of the present application. Referring to FIG. 1 , a Korean relationship extraction model 10 according to an embodiment of the present application includes a preprocessor 100 and a Korean relationship extraction unit 200.

The pre-processing unit 100 receives an input sentence (Sn) composed of Korean. The preprocessor 100 performs a preprocessing operation on the received input sentence Sn to generate a preprocessed input sentence SI.

Specifically, the pre-processing unit 100 may split the input sentence in blank units. At this time, each segmented object may be referred to as a token. The preprocessor 100 may add special tokens around tokens corresponding to the first entity (entity 1) and the second entity (entity 2) among the plurality of tokens. For example, the pre-processing unit 100 may add special tokens to both the front and back of the first and second entities. Since a special token added before and/or after an entity includes entity position information, the special token may be referred to as an entity position token.

The Korean relationship extractor 200 receives the preprocessed input sentence SI from the preprocessor 100. The Korean relationship extractor 200 may predict the relationship between the first entity and the second entity by using all or some of the special tokens included in the preprocessed input sentence SI.

For example, the Korean relationship extractor 200 may predict a relationship between the first entity and the second entity by using a special token added to the front of the first and second entities. As another example, the Korean relationship extractor 200 may predict a relationship between the first entity and the second entity by using a special token added after the first and second entities. As another example, the Korean relationship extractor 200 may predict a relationship between the first entity and the second entity by utilizing special tokens added to both the front and rear of the first and second entities.

As described above, the Korean relationship extraction model 10 according to an embodiment of the present application adds special tokens to Korean input sentences to reflect information about entities well, and predicts relationships between entities by utilizing special tokens. do. Through this, the Korean relationship extraction model 10 according to an embodiment of the present application can accurately predict relationships between entities.

FIG. 2 is a block diagram showing the Korean relationship extractor 200 of FIG. 1 in more detail. Referring to FIG. 2 , the Korean relationship extraction unit 200 includes a Korean language model 210, an entity recognition unit 220, and a relationship classification unit 230.

The Korean language model 210 receives preprocessed input sentences (SI). The Korean language model 210 may be a pre-trained language model trained to understand context in sentences based on a large corpus. For example, the Korean language model 210 may be a pretraining language model such as KoBERT, HanBERT, KorBERT, KoELECTRA, KcELECTRA, and mBERT. However, this is an example, and the pre-learning language model according to the technical idea of the present application is not limited thereto.

The entity recognition unit 220 operates in conjunction with the Korean language model and outputs a hidden state using tokens. The entity recognizer 220 includes K-EPICv 221 , K-EPICs 222 , K-EPICe 223 and K-EPICse 224 .

The K-EPICv 221 outputs a hidden state using a [CLS] token among tokens of the preprocessed input sentence (SI). Here, the [CLS] token refers to the token added at the beginning of the sentence.

K-EPICs (222), K-EPCIe (223), and K-EPICse (224) use at least two special tokens (special token) of the tokens of the pre-processed input armament (SI) to output the hidden state. For example, the K-EPICs 222 outputs a hidden state using an [e1sp] token added before a first entity and an [e2sp] token added before a second entity among special tokens. For example, the K-EPCIe 223 outputs a hidden state using an [e1ep] token added after a first entity and an [e2ep] token added after a second entity among special tokens. For example, the K-EPCIse 224 uses an [e1sp] token, an [e1ep] token, an [e2sp] token, and an [e2ep] token added before and after the first and second entities among the special tokens to hide hidden information. output state

Since different tokens are used, the hidden states of K-EPICv (221), K-EPICs (222), K-EPCIe (223), and K-EPICse (224) may be different.

The relationship classification unit 230 receives the hidden state from the entity recognizing unit 220 . The relation classification unit 230 may predict a final relation between the first entity and the second entity by performing relation classification by applying softmax to the received hidden state.

FIG. 3 is a diagram showing an example of an operation of the pre-processing unit 100 of FIG. 1 . In FIG. 3, it is exemplarily assumed that "KBS Radio 1 is a radio channel operated and broadcast by Korea Broadcasting Corporation, a broadcasting station in Korea" is provided as a Korean input sentence (S). In addition, the first entity (entity 1) is assumed to be "at the Korea Broadcasting Corporation", and the second entity (entity 2) is assumed to be "of the Republic of Korea".

The pre-processing unit 100 will split the input sentence S into space units and express it as a plurality of tokens. The pre-processing unit 100 will add a [CLS] token at the beginning of the input sentence (S), and will add a [SEP] token at the end of the input sentence (S). The preprocessing unit 100 will add an [e1sp] token and an [e1ep] token before and after the first entity "Korea", respectively. The pre-processing unit 200 will add an [e2sp] token and an [e2ep] token before and after the second entity "at Korea Broadcasting Corporation", respectively.

Through this preprocessing operation, the preprocessor 100 may generate the preprocessed input sentence SI as shown in FIG. 3 .

4 to 7 are diagrams showing examples of operations of the Korean language model 210 and entity recognition unit 220 of FIG. 2 . Specifically, FIG. 4 shows an example of an operation related to the K-EPCIv 221 of the entity recognition unit 220. 5 shows an example of an operation related to the K-EPCIs 222 of the entity recognition unit 220. 6 shows an example of an operation related to the K-EPCIe 223 of the entity recognition unit 220. 7 shows an example of an operation related to the K-EPCIse 224 of the entity recognition unit 220.

Referring to FIG. 4 , the preprocessed input sentence (SI) is provided to the Korean language model 210, and the K-EPICv 221 interlocks with the Korean language model 210 to output a hidden state (hv).

At this time, the K-EPICv 221 will output the hidden state (hv) using the [CLS] token among the tokens included in the preprocessed input sentence (SI).

Referring to FIG. 5 , the preprocessed input sentences (SI) are provided to the Korean language model 210, and the K-EPICs 222 interlock with the Korean language model 210 to output hidden states (hs).

At this time, the K-EPICs 222 use [e1sp] tokens and [e2sp] tokens located in front of the first entity and in front of the second entity among the tokens included in the preprocessed input sentence (SI), respectively. will output the state (hs).

Referring to FIG. 6 , the preprocessed input sentence (SI) is provided to the Korean language model 210, and the K-EPICe 223 interlocks with the Korean language model 210 to output a hidden state (he).

At this time, the K-EPICe 223 uses the [e1ep] token and the [e2ep] token located behind the first entity and the second entity among the tokens included in the preprocessed input sentence (SI), respectively. will output the state (he).

Referring to FIG. 7 , the preprocessed input sentence (SI) is provided to the Korean language model 210, and the K-EPICse 224 interlocks with the Korean language model 210 to output a hidden state (hse).

At this time, the K-EPICse 224 is [e1sp] token, [e1ep] token, and [e2sp] located before and after the first entity and the second entity, respectively, among the tokens included in the preprocessed input sentence (SI). We will use the token, the [e2ep] token, to output the hidden state (hse).

The hidden states (hv, hs, he, and hse) output in FIGS. 4 to 7 will have different output values. Accordingly, the final relationship between the first entity and the second entity predicted by the relationship classification unit 230 may differ according to the output hidden states hv, hs, he, and hse.

FIG. 8 is a diagram showing the results of testing the performance of the Korean relationship extraction model 10 of FIG. 1 in FIG. 1 .

As an example, experiments were conducted using the BERT-Ko-RE Dataset and the KLUE-RE Dataset. In addition, HanBERT, KLUE-BERT, KoBERT, KorBERT, and KoELECTRA, which are Korean language models trained with large amounts of Korean data, were applied to consider the characteristics of Korean, and compared with mBERT, which was trained by translating Korean.

In FIG. 8, EPIC_V, EPIC_S, EPIC_E, and EPIC_SE represent experimental results applying K-EPICv (221), K-EPICs (222), K-EPICe (223), and K-EPICse (224), respectively. non_EPIC represents the experimental results when K-EPICv (221) to K-PEICse (224) are not applied.

Also, in FIG. 8 , Micro means Micro-f1 and Weighted means Weighted f1. Specifically, relation extraction aims to predict relations, and the number of relations is more than 30, and the data distribution is different for each relation. Therefore, in the case of Micro, it means the average of the f1 results from all relationships, and Weighted f1 means the average of the f1 results derived from each relationship according to the data distribution.

Referring to FIG. 8, when the K-EPICv (221), K-EPICs (222), K-EPICe (223), and K-EPICse (224) described in FIGS. 1 to 7 are applied, performance is improved. You can check.

In particular, as confirmed in FIG. 8 , it can be seen that the performance results of EPIC_S and EPIC_SE are generally good. Therefore, among the special tokens, it can be confirmed that the tokens in front of the entity and the tokens in front and behind the entity generally reflect a lot of information about the entity.

Meanwhile, it will be understood that the description of FIGS. 1 to 8 is exemplary, and the technical spirit of the present application is not limited thereto. For example, in FIGS. 1 and 2, the entity recognition unit 220 is described as including K-EPICv 221, K-EPICs 222, K-EPICe 223, and K-EPICse 224. It became. However, this is exemplary, and the entity recognition unit 220 may be implemented by including only some of K-EPICv (221), K-EPICs (222), K-EPICe (223), and K-EPICse (224). can

In one embodiment, the entity recognizing unit 220 may be implemented to include only one K-EPIC. For example, the entity recognizing unit 220 may be implemented to include only K-EPICs 222 or only K-EPICse 224 .

In one embodiment, the entity recognizing unit 220 may be implemented to include two K-EPICs. For example, the entity recognizing unit 220 may be implemented to include only K-EPICs 222 and K-EPICse 224, and may be implemented to selectively apply one of them.

9 is a block diagram showing a Korean relationship extraction model 20 according to another embodiment of the present application. The Korean relationship extraction model 20 of FIG. 9 is similar to the Korean relationship extraction model 10 of FIGS. 1 and 2 . Accordingly, the same or similar components are denoted using the same or similar reference numerals, and overlapping descriptions will be omitted for brevity.

Referring to FIG. 9 , the Korean relationship extraction model 20 includes a preprocessing unit 100_1, a Korean language model 210, an entity recognition unit 220_1, and a relationship classification unit 230.

The entity recognizing unit 220_1 of FIG. 9 is implemented to include only one K-EPIC, unlike the entity recognizing unit 220 of FIG. 2 . For example, the entity recognizer 220_1 of FIG. 9 may be implemented to include only K-EPICs 222 among K-EPICs.

In addition, when adding a special token, the preprocessing unit 100_1 of FIG. 9 performs a preprocessing operation to add only the special token corresponding to the K-EPIC of the entity recognizing unit 220_1. For example, as shown in FIG. 9, when the entity recognition unit 220_1 includes only K-EPICs 222, the preprocessor 100_1 adds only [e1sp] tokens and [e2sp] tokens when adding special tokens, and , [e1ep] token and [e2ep] token may not be added.

10 is a diagram showing the operation of the Korean relationship extraction model 20 of FIG. 9 as an example.

As shown in FIG. 10 , the preprocessed input sentence SI generated by the preprocessor 100_1 does not include the [e1ep] token and the [e2ep] token among special tokens. The preprocessed input sentence (SI) is provided to the Korean language model 210, and the K-EPICs (220_1) outputs the hidden state by applying the [e1sp] token and the [e2sp] token.

As described above, the Korean relationship extraction model 20 according to the embodiment of the present application may be implemented to use one K-EPIC, and in this case, the pre-processing unit 100_1 is used in the K-EPIC. A preprocessing operation can be performed to add only the token. Therefore, it is possible to perform the preprocessing operation more efficiently.

11 is a flow chart showing a Korean relationship extraction method according to an embodiment of the present application.

In step S110, a preprocessing operation of adding special tokens for learning entity information to an input sentence is performed. For example, pre-processing operations can be performed to add special tokens before and/or after the two entities. In addition, the pre-processing operation may add [CLS] tokens and [SEP] tokens at the beginning and end of the input sentence.

In step S120, an operation of outputting a hidden state in which entity information is reflected is performed using a special token. For example, if the model is implemented by applying K-EPICs, the hidden state can be output using a special token added in front of each entity. As another example, when a model is implemented by applying K-EPICe, a hidden state may be output using a special token added at the end of each entity. As another example, when the model is implemented by applying K-EPICse, the hidden state may be output using special tokens added before and after each entity. In addition, when the model is implemented by applying K-EPICv, the hidden state may be output using only the [CLS] token, not the special token.

In step S130, a final relationship between entities is predicted by applying softmax to the output hidden state.

12 is a block diagram illustrating a computing environment including a computing device according to an exemplary embodiment.

In the illustrated embodiment, each component may have different functions and capabilities other than those described below, and may include additional components other than those described below.

Referring to FIG. 12, the computing environment includes a computing device 1000, which may be a component implemented to include the Korean relationship extraction model and/or the Korean relationship extraction method described in FIGS. 1 to 11. can The computing device 1000 includes at least one processor 1100, a computer readable storage medium 1200 and a communication bus.

The processor 1100 may cause the computing device 1000 to operate according to the Korean relationship extraction model and/or the Korean relationship extraction method described above. For example, the processor 1100 may execute one or more programs 1210 stored in the computer readable storage medium 1200 . The one or more programs may include one or more computer executable instructions, and the computer executable instructions, when executed by the processor 1100, cause the computing device 1000 to extract a Korean relationship model and/or a Korean relationship extraction method. It can be configured to perform operations according to.

Computer-readable storage medium 1200 is configured to store computer-executable instructions or program code, program data, and/or other suitable forms of information. The program 1210 stored in the computer readable storage medium 1200 includes a set of instructions executable by the processor 1100 . In one embodiment, computer readable storage medium 1200 may include memory (volatile memory such as random access memory, non-volatile memory, or a suitable combination thereof), one or more magnetic disk storage devices, optical disk storage devices, flash It may be memory devices, other forms of storage media that can be accessed by the computing device 1000 and store desired information, or a suitable combination thereof.

The communication bus interconnects the processor 1100, the computer readable storage medium 1200, and various other components of the computing device 1000.

Computing device 1000 may also include one or more input/output interfaces 1300 and one or more network communication interfaces 1400 that provide interfaces for one or more input/output devices 2000 . The input/output interface 1300 and the network communication interface 1400 are connected to a communication bus. The input/output device 2000 may be connected to other components of the computing device 1000 through the input/output interface 1300 . The exemplary input/output device 2000 includes a pointing device (such as a mouse or trackpad), a keyboard, a touch input device (such as a touchpad or touchscreen), a voice or sound input device, various types of sensor devices, and/or a photographing device. input devices, and/or output devices such as display devices, printers, speakers, and/or network cards. The exemplary input/output device 2000 may be included inside the computing device 1000 as a component constituting the computing device 1000, or may be connected to the computing device 1000 as a separate device distinct from the computing device 1000. may be

Although the present invention has been described in detail through representative examples above, those skilled in the art can make various modifications to the above-described embodiments without departing from the scope of the present invention. will understand Therefore, the scope of the present invention should not be limited to the described embodiments and should not be defined, and should be defined by not only the claims to be described later, but also those equivalent to these claims.

10, 20: Korean relationship extraction model
100, 100_1: pre-processing unit
200: Korean relationship extraction unit
210: Korean language model
220: entity recognition unit
230: relationship classification unit

Claims (15)

a pre-processing unit that adds a special token to at least one position before or after an entity included in a Korean input sentence; and
A Korean relationship extraction model using entity location information, including a Korean relationship extraction unit that predicts a relationship between entities included in the Korean input sentence through the special token. According to claim 1,
The Korean input sentence includes a first entity and a second entity,
Wherein the pre-processing unit adds first and second special tokens before and after the first entity, respectively, and adds first and second special tokens before and after the second entity, respectively. According to claim 2,
The Korean relationship extraction model, wherein the preprocessor adds a [CLS] token before the Korean input sentence and adds a [SEP] token after the Korean input sentence. According to claim 1,
The Korean relationship extraction unit
a Korean language model that receives an input sentence to which a special token is added from the pre-processing unit;
an entity recognizer that operates in conjunction with the Korean language model and outputs a hidden state based on the special token; and
A Korean relationship extraction model comprising a relationship classification unit that classifies relationships between entities based on the hidden state. According to claim 4,
The Korean language model is any one of KoBERT, HanBERT, KorBERT, KoELECTRA, and KcELECTRA, a Korean relation extraction model. According to claim 4,
The entity recognition unit
Korean relation extraction model, including K-EPICs that output hidden stays based on special tokens added in front of entities among special tokens. According to claim 6,
The entity recognition unit
A Korean relationship extraction model further comprising K-EPICe that outputs a hidden stay based on a special token added after an entity among special tokens. According to claim 7,
The entity recognition unit
Korean relationship extraction model, further comprising K-EPICse that outputs hidden stays based on special tokens added to both the front and back of the entity among the special tokens. According to claim 8,
The entity recognition unit
Korean relationship extraction model further comprising a K-EPICv that outputs a hidden stay based on the [CLS] token added to the front of the Korean input sentence. According to claim 4,
The Korean relationship extraction model, wherein the relationship classification unit predicts a relationship between entities by applying softmax to the hidden state. Adding a special token including entity location information to a Korean input sentence;
outputting a hidden state reflecting entity information based on the special token; and
A Korean relationship extraction method comprising the step of extracting a relationship between entities by applying softmax to the hidden state. According to claim 11,
The step of adding the special token is
adding a special token to at least one of the front and back of the entity; and
A Korean relationship extraction method comprising adding tokens before and after the Korean input sentence. According to claim 11,
The step of outputting the hidden state is
Korean relationship extraction method for outputting a hidden state by applying K-EPICs that utilize a special token added in front of the entity among the special tokens. According to claim 11,
The step of outputting the hidden state is
Korean relationship extraction method for outputting a hidden state by applying K-EPICe that utilizes a special token added to the end of the entity among the special tokens. According to claim 11,
The step of outputting the hidden state is
Korean relationship extraction method for outputting a hidden state by applying K-EPICse that utilizes special tokens added to both the front and back of the entity among the special tokens.

Images