KR20210085158A - Method and apparatus for recognizing named entity considering context - Google Patents

Abstract

As a method for a computing device operated by at least one processor to recognize an entity name, the method comprises: a step of receiving learning text tagged with an entity name label for a plurality of words; a step of learning each of a monomial entity name prediction model that learns a relationship between each word and an entity name label that is tagged corresponding to each word, and a binomial entity name prediction model that learns a relationship between an adjacent word pair constituting the learning text and the entity name label pair corresponding to each adjacent word pair so as to output a possibility that an arbitrary entity name label is adjacent to each other; and a step of generating an entity name prediction model by combining the learned monomial entity name prediction model and a label determination model at an output end of the learned binomial entity name prediction model. Therefore, the present invention is capable of improving a performance and accuracy of recognizing the entity name.

Description

Method and apparatus for recognizing entity names considering context information {METHOD AND APPARATUS FOR RECOGNIZING NAMED ENTITY CONSIDERING CONTEXT}

The present invention relates to a technology for recognizing an entity in consideration of context information.

Named Entity Recognition (NER) is to find an entity of interest in input text, for example, objects such as people, organizations, geographic locations in general news texts, or diseases in the field of biomedical science, It is to find objects such as proteins, genes, and chemical components. Accordingly, entity name recognition is the basis for Natural Language Processing (NLP), Information Extraction, and Knowledge Service.

In order to automatically perform entity name recognition in large-scale texts, conventionally, experts with expertise and experience in the relevant field manually design a rule or feature and substitute it into an algorithm for obtaining a label sequence.

Recently, through machine learning, a feature or representation suitable for object name recognition is extracted from a large-scale text as a vector of a fixed length, and deep is adopted and used in various fields including image processing and natural language processing. Entity name recognition is performed using learning technology.

For example, a deep learning model for describing the relationship between words constituting the input text and a deep learning model for describing the relationship between the characters constituting the word are combined, and the Conditional Random Field (hereinafter '' CRF') to derive the result of tagging each individual word constituting the text.

In this case, the CRF model plays a role of outputting a label sequence that reflects the relationship between neighboring labels using a transition matrix. The label sequence means a continuous permutation of individual labels corresponding to each word constituting the text.

The transition matrix is for modeling the relationship between neighboring labels for two consecutive input words in text. The transition matrix-based model performs machine learning only by relying on training data, and even when new text is input, it only refers to the value of the previously learned transition matrix, and the context of the newly input text is reflected in the relationship between neighboring labels. There is a limit to what you can't do.

On the other hand, in order to improve the object name recognition performance in Korean text, a method using the tendency of the object name to be located at the beginning of a word in Korean, which is an agglutinative language in which several morphemes are combined in one word, has been developed. In this method, vectors are learned in units of syllable-bigrams in Korean text, and one label is assigned to each syllable-bigram. Although there is a feature of using an artificial neural network to recognize the label for each syllable-bigram, it does not utilize the correlation of neighboring consecutive syllable-bigrams.

Therefore, there is a need for a method of predicting a label sequence by identifying not only individual words but also contextual relationships with neighboring words for input text, and reflecting the identified contextual relationships.

The task to be solved is a method of recognizing the entity names of each word in consideration of the independent characteristics of individual words included in the input text and the relationship between the individual words and neighboring words in order to predict the label sequence of the input text; to provide the device.

A method for recognizing an entity name by a computing device operated by at least one processor according to an embodiment, the method comprising: receiving an input of learning text in which entity name labels are tagged on a plurality of words; An object corresponding to a pair of adjacent words constituting the training text and each adjacent word pair so that a unary object name prediction model that learns the relationship between the tagged object name labels and the possibility that arbitrary object name labels are adjacent to each other are output Each step of training a binomial entity name prediction model that learns the relationship of name label pairs, and combining the learned unary entity name prediction model and the learned binomial entity name prediction model with the label determination model to create an entity name prediction model including the steps of

In the receiving of the input, the training text may be pre-processed into a word embedding model.

In the step of receiving the input, the plurality of words are tagged with the entity name label in an IOBES method or a BIO method, and in the IOBES method, if an arbitrary word corresponds to the start of an entity name composed of a plurality of words, B, entity If it corresponds to the middle of the name, I, if it corresponds to the end of the name of the entity, E, if it is a word that is not an entity name, it is O, if it corresponds to the entity name consisting of one word, it is indicated as S. If it corresponds to the beginning of an entity name composed of words, it may be indicated by B, if it corresponds to an entity name other than the beginning, then I, and if it is a word that is not an entity name, O may be indicated.

The training may include connecting separate label determination models to the output value of the unary entity name prediction model and the output value of the binary entity name prediction model.

In the entity name prediction model, the parameters of the Bidirectional Long Short-Term Memory (BiLSTM) layer included in the unary entity name prediction model and the parameters of the BiLSTM layer included in the binomial entity name prediction model are integrated. It may be a model that has been

The method may further include inputting text into the entity name prediction model, and predicting a label sequence in which entity name labels of words included in the text are listed using the entity name prediction model.

The label sequence may maximize a value obtained by multiplying a probability value output by the learned unary entity name prediction model by a probability value output by the learned binomial entity name prediction model.

The text may be a word embedding vector preprocessed by a word embedding model.

A method for recognizing an entity name by a computing device operated by at least one processor according to another embodiment, the method comprising: inputting text into a unary entity name prediction model and a binary entity name prediction model; using the unary entity name prediction model predicting the entity name label corresponding to each word constituting the text, and predicting the possibility of outputting the predicted entity name label pair corresponding to the adjacent word pair constituting the text using the binary entity name prediction model and outputting a label sequence in which entity names labels of words included in the text are listed based on the predicted results, wherein the unary entity name prediction model tags a plurality of words with entity name labels. It is a model that learns the relationship between each word and the entity name label tagged in correspondence to each word as a training text, and the binary entity name prediction model is the training text. It is a model that learns the adjacency relationship between the label of the entity that is tagged to be output.

The outputting may combine the prediction result of the unary entity name prediction model and the prediction result of the binary entity name prediction model into one label determination model.

A computing device according to an embodiment, comprising: a memory; and at least one processor executing instructions of a program loaded into the memory, wherein the program predicts an entity name with learning text tagged with an entity name label Learning the model, inputting text into the learned entity name prediction model, using the entity name prediction model, entity name labels of each word constituting the text, and entity names corresponding to neighboring words and predicting whether labels can be output next to each other, and outputting possible neighbor labels among the predicted entity name labels as a label sequence of the text.

The inputting may include generating words constituting the text as word embedding vectors using an arbitrary word embedding model, and inputting the generated word embedding vectors into the entity name prediction model.

The entity name prediction model determines whether the unary entity name prediction model for predicting the entity name label of each word constituting the text and entity name labels corresponding to the neighboring words constituting the text can be output next to each other. It may include a binomial entity name prediction model that predicts the likelihood of

The training may include connecting different label determination models to the unary entity name prediction model and the binary entity name prediction model, respectively.

According to the present invention, the performance and accuracy of entity name recognition can be improved by generating a label sequence that is a result of entity name recognition by reflecting not only characteristics of individual words but also correlation with neighboring words.

In addition, according to the present invention, it is possible to improve object name recognition performance for not only general texts such as news, but also specialized texts in the field of biology or medicine.

1 is a structural diagram of an apparatus for recognizing an entity name according to an embodiment.
2 is an exemplary diagram of a result of recognizing an entity name according to an exemplary embodiment.
3 is a flowchart of a method of operating an apparatus for recognizing an entity name according to an exemplary embodiment.
4 is a structural diagram of a unary deep learning model and a binary deep learning model according to an embodiment.
5 is an exemplary diagram of a transition matrix according to an embodiment.
6 is a diagram illustrating a structure in which a unary deep learning model and a binary deep learning model are combined according to an embodiment.
7 is a structural diagram of an entity name prediction model according to an embodiment.
8 is a hardware configuration diagram of a computing device according to an embodiment.

Hereinafter, with reference to the accompanying drawings, embodiments of the present invention will be described in detail so that those of ordinary skill in the art to which the present invention pertains can easily implement them. However, the present invention may be embodied in many different forms and is not limited to the embodiments described herein. And in order to clearly explain the present invention in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.

Throughout the specification, when a part "includes" a certain element, it means that other elements may be further included, rather than excluding other elements, unless otherwise stated. In addition, terms such as “…unit”, “…group”, and “module” described in the specification mean a unit that processes at least one function or operation, which may be implemented by hardware or software or a combination of hardware and software. have.

1 is a structural diagram of an apparatus for recognizing an entity name according to an embodiment, and FIG. 2 is an exemplary diagram of a result of recognizing an entity name according to an embodiment.

Referring to FIG. 1, the entity name recognition apparatus 1000 includes a preprocessing unit 100 for vectorizing an input sentence, and a learning unit for learning two separate deep learning models 210 and 220 using the preprocessed learning text, respectively ( 200) And predicting the entity name of the input sentence using the entity name prediction model 310 including the learned models 210 and 220 and the label determination model 311 and the prediction unit 300 for outputting a label sequence includes

For the sake of explanation, the preprocessor 100 , the learner 200 , and the prediction unit 300 are named and called, but these are computing devices operated by at least one processor. Here, the preprocessor 100 , the learner 200 , and the prediction unit 300 may be implemented in one computing device or distributed in separate computing devices. When distributed in a separate computing device, the preprocessor 100 , the learner 200 , and the prediction unit 300 may communicate with each other through a communication interface. The computing device may be any device capable of executing a software program written to carry out the present invention, and may be, for example, a server, a laptop computer, or the like.

Each of the preprocessing unit 100 , the learning unit 200 , and the prediction unit 300 may be one artificial intelligence model, or may be implemented as a plurality of artificial intelligence models. And the word embedding model 110, the unary deep learning model 210, the binary deep learning model 220, the unary label determination model 230, the binary label determination model 240, the label determination model 311, and the entity name The predictive model 310 may also be one artificial intelligence model or may be implemented as a plurality of artificial intelligence models. The entity name recognition apparatus 1000 may be one artificial intelligence model or may be implemented as a plurality of artificial intelligence models. Accordingly, one or a plurality of artificial intelligence models corresponding to the above-described configurations may be implemented by one or a plurality of computing devices.

When one sentence is input, the entity name recognition apparatus 1000 attaches a label, which is a result of entity name recognition, to each word constituting the sentence, and outputs a label sequence in which entity name labels are arranged. From the point of view of an AI model, a word can be called a node.

On the other hand, as a method of outputting the name of the entity constituting the label sequence, BIO tagging may be used. In BIO tagging, when displaying the object name recognition result, the starting word of an object name consisting of several words is displayed as B (Beginning), the word in the middle is I (Inside), and in the case of a word that is not an object name, O (Outside). This is a tagging method indicated by .

As another example, in addition to BIO, IOBES tagging may be used in which the letter at the end of the entity name is additionally expressed as E (End), and when one word itself is the entity name, additionally S (Singleton). The method of tagging the object name is not limited to any one.

Referring to FIG. 2 , for the first line of the input text, the entity name recognition result of each word constituting the first line is output as a label sequence. For example, since ureteric is the beginning of a name corresponding to Disease and obstruction is the end of a name corresponding to Disease, it can be seen that ureteric obstruction is a name corresponding to Disease. Also, it can be seen that caused and by are not individual names, and indinavir is an individual name composed of one word.

Returning again to FIG. 1 , roles of parts constituting the apparatus 1000 for recognizing an entity name will be described.

The preprocessor 100 converts the words of the training text and the input text into embedding vectors. The transformation method may be generated by, for example, searching a dictionary for an input word, or may use a value output through a pre-trained model. It is also possible to use the vector output from the model by putting the characters constituting the input word into a new deep learning model. Also, a plurality of methods may be used, but the present invention is not limited thereto.

In this specification, the word embedding model 110 is used and, as an example, a case using a pre-trained language model (Embeddings from Language Model, ELMo) for embedding words in consideration of the entire inputted sentence will be described.

The preprocessor 100 converts the training text into a word embedding vector and transmits it to the learner 200 , and converts the input text into a word embedding vector and transmits it to the predictor 300 .

The learning unit 200 trains the unary deep learning model 210 and the binomial deep learning model 220 using the training text converted into the word embedding vector and the two label determination models 230 and 240 , respectively.

The unary deep learning model 210 refers to a model that calculates the probability of individual labels of a word by considering only one word itself, and the binary deep learning model 220 considers the labels of neighboring words located around one word. It means a deep learning model that plays a role in calculating the probability of the label of the word.

The unary deep learning model 210 or the binary deep learning model 220 may be implemented as a suitable model according to the characteristics of the task to which the entity name recognition apparatus 1000 is applied. For example, it can be implemented with Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), Long Short-Term Memory (LSTM), Attention mechanism-based Transformer, etc. and is not limited to any one.

In this specification, a case in which the unary deep learning model 210 and the binomial deep learning model 220 is implemented with a bidirectional long short-term memory (BiLSTM) will be described. Detailed structures of the unary deep learning model 210 and the binary deep learning model 220 and the process of learning each model will be described with reference to FIG. 4 .

The prediction unit 300 combines the unary deep learning model 210 learned through the learning unit 200 and the learned binomial deep learning model 220 into one label determination model 311, and the entity name prediction model 310 ) is created. The structure of the entity name prediction model 310 in which the unary deep learning model 210 and the binary deep learning model 220 are combined through one label determination model 311 will be described in detail with reference to FIGS. 6 and 7.

Then, the prediction unit 300 outputs one label sequence as the entity name recognition result of the input text by using the generated entity name prediction model 310 .

The label determination model 311 collects the probability values calculated through the unary deep learning model 210 and the binomial deep learning model 220, and outputs one most suitable label sequence in consideration of the relationship with the neighboring labels. .

In the present specification, the label determination model 311 is implemented as a CRF, but is not necessarily limited thereto, and may be implemented as a Hidden Markov Model (HMM), a Maximum Entropy Markov Model (MEMM), or the like.

3 is a flowchart of a method of operating an apparatus for recognizing an entity name according to an exemplary embodiment.

Referring to FIG. 3 , the entity name recognition apparatus 1000 receives a training text ( S110 ). The field of the input learning text may be news, a biological field, a medical field, and the like, but is not limited thereto. The training text means data in which each word is tagged with IOBES.

The pre-processing unit 100 of the entity name recognition apparatus 1000 converts the training text into a word embedding vector by using the word embedding model 110 ( S120 ). The word embedding method for converting a word into a vector is not limited to any one, and as an example, Word2Vec, a distributed representation model that expresses words in a vector dimension, and ELMo ( Embeddings from Language Model) may be used as the word embedding model 110 .

The learning unit 200 connects the unary label determination model 230 to the output terminal to the unary deep learning model 210, and trains the unary deep learning model 210 with the word embedding vector converted in step S120 (S130). That is, the unary deep learning model 210 learns the relationship between each word constituting the training text and the tagged IOBES result.

In parallel with step S130, the learning unit 200 connects the binary label determination model 240 to the output end to the binary deep learning model 220, and uses the word embedding vector converted in step S120 to convert the binary deep learning model 220. to learn (S140). That is, the binary deep learning model 220 learns the relationship between the plurality of words constituting the training text and the output object names using the IOBES results tagged to the corresponding words.

The prediction unit 300 of the entity name recognition apparatus 1000 combines the unary deep learning model 210 and the binary deep learning model 220 learned through steps S130 and S140, and a label determination model 311 at the output end. ) to create the entity name prediction model 310 (S150). At this time, the two learned models are combined in the new label determination model 311, not the label determination models 230 and 240 used when training the two models. As described in FIG. 2, in the learning process, in order to facilitate parameter learning of the unary deep learning model 210 and the binomial deep learning model 220, the label determination models 230 and 240 for the two deep learning models, respectively. because it is connected to

Thereafter, the entity name recognition apparatus 1000 receives the input text, and the preprocessor 100 converts the input text into a word embedding vector ( S170 ). The word embedding method used may be the same as that of step S120.

The prediction unit 300 puts the transformed word embedding vectors into the entity name prediction model 310, and outputs a label sequence that is the entity name recognition result for the input text (S180). The outputted label sequence may be in the form of FIG. 2 .

Hereinafter, a method for the learning unit 200 to learn the unary deep learning model 210 and the binary deep learning model 220 in steps S130 and S140, respectively, will be described through the structure of each deep learning model.

4 is a structural diagram of a unary deep learning model and a binomial deep learning model according to an embodiment, and FIG. 5 is an exemplary diagram of a transition matrix according to an embodiment.

Figure 4 (a) shows the structure of the unary deep learning model 210, Figure 4 (b) shows the structure of the binary deep learning model 220.

When the training text is generated as a word embedding vector in the preprocessor 100 and input to the learning unit 200 , the learning unit 200 trains the unary deep learning model 210 and the binary deep learning model 220 , respectively.

In the prediction unit 300, two deep learning models are combined through one label determination model 311, but in the learning process, parameter learning of the unary deep learning model 210 and the binomial deep learning model 220 is facilitated. To do this, each deep learning model is trained individually. That is, the unary label determination model 230 and the binary label determination model 240 are used, respectively.

Each deep learning model (210, 220) proceeds to model learning to maximize the likelihood (Likelihood) for the training text using each individual label determination model (230, 240).

The unary deep learning model 210 includes a BiLSTM layer 211 and a binding layer 212 and is connected to the unary label determination model 230, and the binomial deep learning model 220 includes a BiLSTM layer 211 and a binding layer ( 212 ) and connected to the binary label determination model 240 .

Hereinafter, since the roles of the BiLSTM layer 211 and the BiLSTM layer 221 included in the unary deep learning model 210 and the binomial deep learning model 220, respectively, are similar, they will be described together, and in the unary deep learning model 210 The roles of the binding layer 222 and the unary label decision model 230 and the binary label decision model 240 in the binding layer 212 and the binomial deep learning model 220 will be described separately.

The BiLSTM layers 211 and 221 receive the word embedding vector generated by the preprocessor 100 and generate a state vector having an arbitrary dimension according to the input vector. Specifically, the state vector at time t generated through BiLSTM is generated through a simple concatenation operation of the state vector at time t of the forward LSTM and the state vector at time t of the backward LSTM.

In this case, the state vector at time t of the forward LSTM is generated through a function operation using the word embedding vector at time t and the state vector of the forward LSTM at the previous time t-1 as inputs. Similarly, the state vector at time t of the backward LSTM is generated through a function operation with the word embedding vector at time t and the state vector of the backward LSTM at future time t+1 as inputs.

A state vector having an arbitrary dimension generated through the BiLSTM layers 211 and 221 becomes an input vector of the upper coupling layers 212 and 222 . Meanwhile, the size of the vector output through the combining layers 212 and 222 may match the number of possible labels. For example, the problem of using the IOBES tagging method will be explained in order to recognize the individual names of the diseases and chemicals constituting the text. In this case, the labels that each word can have are I, B, E, and S for the disease and chemical type entity names, and O is possible, so the number of possible labels is 2*4+1, which can be a total of 9.

The combining layers 212 and 222 are composed of at least one linear transform operation, but are not necessarily limited thereto, and may be composed of a combination of a plurality of linear and non-linear transform operations in some cases. Hereinafter, the roles of the coupling layer 212 of the unary deep learning model 210 and the coupling layer 222 of the binomial deep learning model 220 will be described with reference to the drawings.

First, referring to FIG. 4A , the combination layer 212 of the unary deep learning model 210 generates a score vector through an arbitrary function.

The score means a numerical value for each word constituting the input text to correspond to an arbitrary label, and the score vector means that the score for each word is arranged in a vector format. For the example described above, each word in the text is “B-Disease”, “I-Disease”, “E-Disease”, “S-Disease”, “B-Chemical”, “I-Chemical”, “ A score corresponding to each label of "E-Chemical", "S-Chemical", and "O" may be output.

In this case, the combination layer 212 in the unary deep learning model 210 generates a score vector corresponding to each node, and the size of the score vector will be 9. In addition, the score vector of any other node will also contain a score for which the node word corresponds to each of the nine labels.

The unary label determination model 230 connected to the unary deep learning model 210 calculates the label sequence probability for the training text as shown in Equation 1.

는 출력되는 라벨 시퀀스의 확률 값이다.

는 출력 값을 확률로 정규화 시키기 위하여 가능한 라벨 시퀀스의 확률 값을 모두 더한 값이고, N은 학습 데이터에 포함된 단어의 개수이자 출력되는 라벨의 개수이다.

는 i번째 단어에 대해

라는 라벨을 할당하기 위한 단항 스코어이고,

는 전이 행렬 A의 i-1번째 행과 i번째 열에 해당하는 값을 의미하며, 도 5를 통해 자세히 설명한다.In Equation 1, X is the input training text, Y is the output label sequence,

is the probability value of the output label sequence.

is the sum of all probability values of possible label sequences in order to normalize the output value to the probability, and N is the number of words included in the training data and the number of output labels.

is for the ith word

is a unary score for assigning a label,

denotes values corresponding to the i-1 th row and i th column of the transition matrix A, which will be described in detail with reference to FIG. 5 .

는 다음 수학식 2를 통해 계산될 수 있다. Meanwhile

can be calculated through Equation 2 below.

Referring to FIG. 5 , the transition matrix refers to a matrix indicating all possible combinations of label pairs that can be combined into two labels. That is, each element (Entry) of the matrix is a score for mutual compatibility in which pairs of labels corresponding to each row and column appear adjacent to each other in the label sequence.

Since the transition matrix uses values created and stored in the machine learning process, it always provides the same value for any label pair even when new text is input. Accordingly, there is a disadvantage in that the relationship and compatibility with neighboring labels according to the context of the text cannot be considered. Therefore, in the present invention, the binary deep learning model 220 that considers the relationship between neighboring labels in the label sequence of the output stage for text input is used together.

Now, with reference to Fig. 4 (b), the role of the binding layer 222 of the binomial deep learning model 220 will be described. The combination layer 222 of the binary deep learning model 220 receives two state vectors of neighboring positions output from the lower BiLSTM layer 221 as inputs, and generates a score vector for the interoperability of neighboring label pairs. do.

In this case, the size of the score vector may be determined to include scores for all label pairs that can be combined. For example, the problem of using the IOBES tagging method will be described in order to recognize the individual names of the diseases and chemicals constituting the text described above. In this case, each word constituting the text has a corresponding score vector, and the size of the score vector will be 81 in 9*9.

The binary deep learning model 220 infers a relationship between two consecutive labels output from the label determination model 240 from the input text. In the model learning step, the binary label determination model 240 connected to the binary deep learning model 220 calculates a label sequence probability for the training text as shown in Equation 3.

는 라벨 시퀀스에 대한 확률 값이다.

는 라벨 시퀀스를 확률 값으로 정규화하기 위해, 생성 가능한 모든 라벨 시퀀스들의 스코어들을 모두 더한 값이고, N은 텍스트 입력에 포함된 단어의 개수이자 출력되는 라벨의 개수이다.

는 i-1번째 위치에서의 임의의 라벨과 i번째 위치의 임의의 라벨의 상호 호환성에 대한 스코어이다. 상호 호환성이란, 해당 위치에서 임의의 라벨 쌍이 나타날 수 있는 가능성을 의미한다. In Equation 3, X is the input training text, Y is one label sequence as an output corresponding to the training text,

is the probability value for the label sequence.

In order to normalize the label sequence to a probability value, N is the sum of the scores of all the label sequences that can be generated, and N is the number of words included in the text input and the number of output labels.

is a score for the interchangeability of any label at the i-1 th position and any label at the i th position. Interoperability refers to the possibility of any label pair appearing in that position.

는 다음 수학식 4를 통해 계산될 수 있다. Meanwhile

can be calculated through Equation 4 below.

Meanwhile, the calculation method for generating the score vector is not limited to any one. For example, a method of constructing neighboring vectors by simply concatenating them into one vector based on the vectors output from the neural network at the bottom, Element-wise multiplication of two neighboring vectors, or Hadamard Product ) to form a single vector can be used.

는

의 형태일 수 있다. 이때

는 Bilinear Model의 j번째 행렬을 가리킨다.Also, it is possible to use a method of forming a single vector by applying an absolute value to the difference between two neighboring vectors having the same size. In this case, Equation 3

is

may be in the form of At this time

is the j-th matrix of the bilinear model.

Meanwhile, although it has been described that only the relationship between adjacent state vectors is considered in the binary deep learning model 220, the present invention is not limited thereto.

6 is a diagram illustrating a structure in which a unary deep learning model and a binomial deep learning model are combined according to an embodiment, and FIG. 7 is a structural diagram of an entity name prediction model according to an embodiment.

는 시간 t-1에서 출력되는 라벨,

는 시간 t에서 출력되는 라벨을 의미한다.Referring to FIG. 6 , X is a word input to the unary deep learning model 210, and Y is an output label for each word constituting the input text. In other words

is the label output at time t-1,

denotes a label output at time t.

The unary deep learning model 210 is connected to individual labels output by the label determination model 311 , and the binary deep learning model 220 is connected between adjacent labels output by the label determination model 311 .

Referring to FIG. 7 , the prediction unit 300 integrates the learned unary deep learning model 210 and the learned binomial deep learning model 220 in one label determination model 311 to predict the entity name model 310 . create The entity name prediction model 310 provides one label sequence that is most suitable for the input text. In this case, the parameters of the BiLSTM layer 211 of the unary deep learning model 210 and the BiLSTM layer 221 of the binomial deep learning model 220 may be integrated with each other.

The label determination model 311 of the entity name prediction model 310 is a value calculated through Equation 5 using both the output value of the unary deep learning model 210 and the output value of the binary deep learning model 220. can provide

는 단항 딥러닝 모델(210)에서 계산된 해당 라벨 시퀀스의 확률 값이고,

는 이항 딥러닝 모델(220)에서 계산된 해당 라벨 시퀀스의 확률 값이다. In Equation 5, X is the input text, Y is the output label sequence,

is the probability value of the corresponding label sequence calculated in the unary deep learning model 210,

is a probability value of the corresponding label sequence calculated in the binary deep learning model 220 .

As an example, the entity name prediction model 310 may be obtained by multiplying the result of the unary deep learning model 210 and the result of the binary deep learning model 220 . In this case, predicting the entity label is a process of obtaining one label sequence that maximizes the multiplied value.

In this case, since the purpose is to obtain a label sequence that maximizes the probability value, not the probability value itself, the normalization process for the probability value may be omitted. Specifically, decoding techniques such as Viterbi decoding may be used.

8 is a hardware configuration diagram of a computing device according to an embodiment.

Referring to FIG. 8 , the preprocessing unit 100 , the learning unit 200 and the prediction unit 300 are instructions described to execute the operations of the present invention in the computing device 400 operated by at least one processor. Executes a program containing (instructions).

The hardware of the computing device 400 may include at least one processor 410 , a memory 420 , a storage 430 , and a communication interface 440 , and may be connected through a bus. In addition, hardware such as an input device and an output device may be included. The computing device 400 may be loaded with various software including an operating system capable of driving a program.

The processor 410 is a device for controlling the operation of the computing device 400, and may be various types of processors 410 that process instructions included in a program, for example, a central processing unit (CPU), an MPU (Central Processing Unit) Micro Processor Unit), MCU (Micro Controller Unit), GPU (Graphic Processing Unit), and the like. The memory 420 loads the corresponding program so that the instructions described to execute the operation of the present invention are processed by the processor 410 . The memory 420 may be, for example, read only memory (ROM), random access memory (RAM), or the like. The storage 430 stores various data, programs, etc. required for executing the operation of the present invention. The communication interface 440 may be a wired/wireless communication module.

The apparatus for recognizing entity names of the present invention may be utilized to recognize entity names for a person's name, an organization, a company or institution name, a geographical name, and the like in general news text.

In addition, the apparatus for recognizing entity names of the present invention can be utilized not only in news texts dealing with general fields, but also texts specific to each field. In particular, in the case of texts in the field of biology and medicine, the names of proteins, genes, chemical components, diseases, DNA, and RNA can be names of entities.

Individual names in specialized fields are newly discovered or created as each field is researched, there is no standard for naming rules, and there are cases in which names are used to describe or describe objects. Accordingly, the apparatus for recognizing entity names according to the present invention determines entity names and label sequences by recognizing contexts between neighboring words, and thus can be utilized even when recognizing entity names connected by a plurality of words in a specialized field.

The embodiment of the present invention described above is not implemented only through the apparatus and method, and may be implemented through a program for realizing a function corresponding to the configuration of the embodiment of the present invention or a recording medium in which the program is recorded.

Although the embodiments of the present invention have been described in detail above, the scope of the present invention is not limited thereto, and various modifications and improved forms of the present invention are also provided by those skilled in the art using the basic concept of the present invention as defined in the following claims. is within the scope of the right.

Claims (14)

A method for a computing device operated by at least one processor to recognize an entity name,
Step of receiving the input of the learning text tagged with the entity name label in a plurality of words,
A unary entity name prediction model for learning the relationship between each word and the entity name label corresponding to each word, and adjacent word pairs constituting the training text so that the probability that any entity name labels are adjacent to each other is output; Training each binary entity name prediction model for learning the relationship between entity name label pairs corresponding to each adjacent word pair, and
Creating an entity name prediction model by combining the learned unary entity name prediction model and the label determination model at the output of the learned binomial entity name prediction model
A method for recognizing an entity name, including In claim 1,
The step of receiving the input is
An entity name recognition method of pre-processing the training text into a word embedding model. In claim 1,
The step of receiving the input is
Tagging the entity name label to the plurality of words in an IOBES method or a BIO method,
In the IOBES method, if an arbitrary word corresponds to the beginning of an entity name composed of a plurality of words, B, if it corresponds to the middle of the entity name, I, if it corresponds to the end of the entity name, E, if it is a word other than the entity name, O, an entity composed of one word If it corresponds to a person, it is indicated by an S,
In the BIO method, if the arbitrary word corresponds to the start of the entity name composed of the plurality of words, B, if it corresponds to the entity name other than the beginning, I, and if the word is not the entity name, O is displayed. Way. In claim 1,
The learning step is,
A method for recognizing an entity name, in which separate label determination models are connected to the output value of the unary entity name prediction model and the output value of the binary entity name prediction model. In claim 1,
The entity name prediction model is,
Entity name recognition, which is a model in which the parameters of the Bidirectional Long Short-Term Memory (BiLSTM) layer included in the unary entity name prediction model and the parameters of the BiLSTM layer included in the binomial entity name prediction model are integrated Way. In claim 1,
inputting text into the entity name prediction model, and
Predicting a label sequence in which entity name labels of words included in the text are listed using the entity name prediction model
Further comprising, the object name recognition method. In claim 6,
The label sequence is
A method for recognizing an entity name such that a value obtained by multiplying a probability value output by the learned unary entity name prediction model by a probability value output by the learned binomial entity name prediction model becomes a maximum. In claim 7,
wherein the text is a word embedding vector preprocessed by a word embedding model. A method for a computing device operated by at least one processor to recognize an entity name,
inputting text into the unary entity name prediction model and the binary entity name prediction model;
The entity name label corresponding to each word constituting the text is predicted using the unary entity name prediction model, and the predicted entity name corresponding to the adjacent word pair constituting the text using the binary entity name prediction model predicting the likelihood that a label pair will be output, and
Outputting a label sequence in which entity name labels of words included in the text are listed based on the predicted result
including,
The unary entity name prediction model is a learning text in which a plurality of words are tagged with an entity name label, and is a model that learns the relationship between each word and the entity name label tagged corresponding to each word,
The binary entity name prediction model is a model that learns the adjacency relationship between tagged entity name labels so that the possibility that arbitrary entity name labels are adjacent to each other is output as the training text. In claim 9,
The output step is
A method for recognizing an entity that combines the prediction result of the unary entity name prediction model and the prediction result of the binary entity name prediction model into one label determination model. A computing device comprising:
memory, and
at least one processor executing instructions of a program loaded into the memory;
the program is
training the entity name prediction model with the entity name label-tagged training text;
Entering text into the learned entity name prediction model;
Using the entity name prediction model, predicting whether entity name labels of each word constituting the text and entity name labels corresponding to neighboring words can be output next to each other; and
Outputting possible neighboring entity name labels among the predicted entity name labels as a label sequence of the text
A computing device comprising instructions written to execute In claim 11,
The input step is
A computing device for generating words constituting the text as word embedding vectors using an arbitrary word embedding model, and inputting the generated word embedding vectors into the entity name prediction model. In claim 11,
The entity name prediction model is,
A unary entity name prediction model for predicting the entity name label of each word constituting the text and a binary entity predicting the possibility that entity name labels corresponding to neighboring words constituting the text can be output next to each other A computing device comprising a life prediction model. In claim 13,
The learning step is,
A computing device for connecting different label determination models to the unary entity name prediction model and the binary entity name prediction model, respectively.

Images