KR20220075807A - System and Method for correcting Context sensitive spelling error using Generative Adversarial Network - Google Patents

Abstract

The present invention simulates a real language model using a generative adversarial network, and uses this information to respond to various errors appearing in text-written general documents. An apparatus and method for correcting context-dependent spelling errors using a neural network, comprising: an input unit for inputting a sentence for correction; an error word inspection unit for examining an input sentence by word unit and searching for a context spelling error; a word to be corrected and a word in a word dictionary A candidate selection unit that selects a candidate word by calculating the editing distance between them; A language model generated using a generative adversarial neural network is used to calculate the distance between the entire surrounding context of the word to be corrected and the candidate words filtered by the candidate selection unit It includes; a prediction candidate generator; a correction word presenting unit that selects a final corrective word based on a distance calculated value between words calculated in a language model created in a generative adversarial neural network.

Description

System and Method for correcting Context sensitive spelling error using Generative Adversarial Network

The present invention relates to spelling error correction, and specifically, by using a generative adversarial network to simulate a real language model close to the actual language model, and using this information to solve various errors appearing in general documents written in text It relates to an apparatus and method for correcting context-dependent spelling errors using a generative adversarial neural network that enables a response to the suffix.

Recently, artificial intelligence technology such as deep learning has been actively researched in various computer-related fields around the world.

Among them, various researchers around the world such as Google Research, Facebook Research, and AllenNLP are developing deep learning technologies in relation to natural language processing, and the demand for deep learning-based programs is rapidly increasing in each field of industry. .

In natural language processing, the earlier the correct sentence is input in the information analysis process, the better the result is.

There are two main types of spelling errors: non-word spelling errors and context sensitive spelling errors.

A simple spelling error can be corrected more easily than a context-dependent spelling error, and the error word is judged by comparing whether the word to be corrected is included in the dictionary.

On the other hand, in the case of a context-dependent spelling error, the correction difficulty increases considerably. For example, in the sentences "look at your attention" and "look around you", "attention" becomes an error word. Since it is a word that exists, it is difficult to solve it by correcting a simple spelling error, so it must be solved by grasping the information of the surrounding context.

In the example, "attention" is an error word, but depending on the context, "around" can also be an error word.

The context-dependent spelling error correction method can be divided into a correction method using rules, a correction method based on statistical information, and a correction method using a neural network.

The rule-based proofreading method requires experts with advanced linguistics and computer science knowledge to create and verify rules, and it is practically impossible to make rules that reflect all linguistic phenomena in the real world.

In particular, a high frequency of occurrence or a standardized error has a high probability of being corrected by the rule-based method, but the unstructured error correction caused by an input error cannot be corrected only by the rule-based method, and the difficulty of correction is much higher.

The statistical correction method can be applied in a language environment where atypical errors frequently occur, and is a method that has been mainly proposed before neural network-based correction.

Compared to the development speed of neural network-based technology, it is difficult to find cases that are applied to context-dependent spelling error correction technology.

As one of the prior art methods, there is a method of correcting a context-dependent spelling error using a statistical model based on the frequency of appearance between each word of the proofing word pair and a word appearing in the surrounding context using a pre-constructed proofreading word pair. ( Republic of Korea Patent Registration No. 10-1495240)

Another method is to use the Korean lexical semantic network in the process of generalizing the rules to increase the reproducibility of the proofing rules. (Registration of Korean Patent No. 10-1500617)

As another method, a method for solving the data shortage problem that occurs in the process of calculating the correlation between each vocabulary of the proofreading vocabulary pair and the vocabulary displayed in the surrounding context has been proposed (Korean Patent Registration No. 10-1573854).

As a recently proposed method, a method for correcting context-dependent spelling errors by generating error candidates in real time for various errors has been proposed. (Korean Patent Publication No. 10-2019-0133624)

A common limitation of the statistical learning method and the deep learning method of the prior art is that the less the training data, the significantly lower the performance.

In certain circumstances, it may be difficult to obtain desired performance due to difficulties in obtaining training data, but natural language processing is no exception.

Therefore, there is a need to develop a technique for improving performance using low-capacity data, and a necessity for correcting spelling errors depending on context is also required.

Republic of Korea Patent No. 10-1495240 Republic of Korea Patent Registration No. 10-1500617 Republic of Korea Patent No. 10-1573854 Republic of Korea Patent Publication No. 10-2019-0133624

The present invention is to solve the problems of the prior art spelling error correction technology, and by using a generative adversarial network to simulate a real language model close to the actual language model, and use this information to create a general text An object of the present invention is to provide an apparatus and method for correcting context-dependent spelling errors using a generative adversarial neural network that enables responses to various errors appearing in documents.

The present invention provides an apparatus and method for correcting context-dependent misspellings using a generative adversarial neural network that enables efficient context-dependent spelling error correction using low-volume data in an environment in which it is difficult to obtain learning data due to difficulties in obtaining desired performance. There is a purpose.

The present invention uses a generative adversarial neural network to generate and provide learning data similar to sentences used by real people, so it is possible to provide abundant data for quantitative learning. and to provide a method.

It is an object of the present invention to provide an apparatus and method for correcting a context-dependent spelling error using a generative adversarial neural network capable of increasing correction accuracy for words that were not well corrected due to a lack of data.

Other objects of the present invention are not limited to the above-mentioned objects, and other objects not mentioned will be clearly understood by those skilled in the art from the following description.

In order to achieve the above object, an apparatus for correcting a context-dependent spelling error using a generative adversarial neural network according to the present invention includes an input unit for inputting a sentence for correction; Inspection unit; A candidate selection unit that selects a candidate word by calculating the edit distance between the word to be corrected and a word in the word dictionary; A generative adversarial neural network is used to calculate the distance between the entire surrounding context of the word to be corrected and the candidate words filtered by the candidate selector A prediction candidate generator that calculates using a language model generated using do.

To achieve another object, a context-dependent spelling error correction method using a generative adversarial neural network according to the present invention includes the steps of determining the possibility of a spelling error by examining an input sentence by word unit; in a language model to be corrected and a candidate word calculating the editing distance between the dictionary words of Calculating the word to be replaced; presenting the final correction word based on the ranked information; characterized in that it includes.

As described above, the apparatus and method for correcting a context-dependent spelling error using a generative adversarial neural network according to the present invention have the following effects.

First, by using a generative adversarial network, it is simulated close to the actual language model, and by using this information, it is possible to respond to various errors that appear in general documents written in text.

Second, in an environment where it is difficult to obtain the desired performance because it is difficult to obtain training data, the context-dependent spelling error correction is efficiently performed using low-volume data.

Third, by using a generative adversarial neural network to generate and provide learning data similar to sentences used by real people, it is possible to provide abundant data for quantitative learning.

Fourth, it is possible to increase the proofreading accuracy for words that were not well proofread due to the lack of data.

1A and 1B are diagrams illustrating a language model learning configuration using a context generated by a generative adversarial neural network according to the present invention.
2 is a block diagram of a language model learning algorithm using a context generated by a generative adversarial neural network according to the present invention.
3 is a configuration diagram of a context-dependent spelling error correction apparatus using a language model according to the present invention;
4 is a flowchart illustrating a context-dependent spelling error correction method using a language model according to the present invention.

Hereinafter, a preferred embodiment of an apparatus and method for correcting a context-dependent spelling error using a generative adversarial neural network according to the present invention will be described in detail as follows.

Features and advantages of the apparatus and method for correcting context-dependent spelling errors using a generative adversarial neural network according to the present invention will become apparent through detailed description of each embodiment below.

1A and 1B are diagrams illustrating a language model learning configuration using a context generated by a generative adversarial neural network according to the present invention, and FIG. 2 is a configuration diagram of a language model learning algorithm using a context generated by a generative adversarial neural network according to the present invention. to be.

An apparatus and method for correcting context-dependent spelling errors using a generative adversarial network according to the present invention closely simulates a real language model using a generative adversarial network, and uses this information to create text It was made possible to respond to various errors that appear in general documents.

To this end, the present invention generates a sentence through a generative adversarial neural network, learns a language model using the generated sentence, and identifies and corrects the relationship between the word to be corrected and the surrounding context of the word to be corrected through the language model. includes

A generative adversarial neural network is a method of solving data generation problems by approaching based on the minmax theorem of John Von Neumann, one of game theory, and in Figs. 1a and 1b. Similarly, a deep neural network composed of a data generation model and a discriminate model learns in an opposing way while repeating generation and discrimination dependent on each other.

As this learning is repeated, eventually, when the discriminator cannot distinguish between real and fake data, the game theory reaches the nash equilibrium, which means that any sentence generated by the generator can be faked. It means that the discriminator, which used to be classified by , cannot distinguish the sentences generated by the generator developed through learning.

Finally, the sentence generated by the creator becomes meaningful information as it becomes similar to the sentence written by a real person.

In the proofreading in the present invention, a language model close to reality is learned using a generated sentence, and correction is performed using the language model.

The present invention is also applied to generative adversarial neural networks of various structures, and is not limited to the structures shown in FIGS. 1A and 1B.

2 shows an example of an algorithm for generating a sentence using a generative adversarial neural network according to the present invention and reflecting it in a language model.

A context-dependent spelling error correction apparatus using a language model according to the present invention will be described in detail as follows.

3 is a block diagram of a context-dependent spelling error correction apparatus using a language model according to the present invention.

As shown in FIG. 3, the context-dependent spelling error correction apparatus using a generative adversarial neural network according to the present invention includes an input unit 301 for inputting a sentence for correcting an error, and an input unit 301 for inputting a word of a sentence input through the input unit 301. The error word checker 302 sequentially checks for errors, the candidate selector 303 selects correction candidates for the word to be corrected when it is determined that there is an error in the word, and the context and candidates around the location of the word to be corrected. It includes a prediction candidate generating unit 304 for calculating the distance values of words, and a proofing word presenting unit 305 for finally suggesting a candidate word close to the context.

The correction process using the context-dependent spelling error correction apparatus using a generative adversarial neural network according to the present invention having such a configuration will be described in detail for each step as follows.

All words used for proofreading are expressed using the generated language model, and the language model refers to a model used for understanding natural language.

Equation 1 represents a basic generative adversarial neural network, where D is a discriminant model (or discriminant) and G denotes a generative model (or generator).

은 초기에 생성자에서 생성된 가짜 데이터를 판단하여 0, 실제 데이터 x를 입력으로 하는

는 1로 계산된다.The generator takes a random vector z as input and generates fake data, and the discriminator outputs 1 when it is real and 0 when it is fake.

judging the fake data generated in the initial generator, 0 and real data x as input

is counted as 1.

는 실제 데이터 분포이고

는 생성자에서 생성한 데이터의 분포이며, E는 예상되는 출력이다.

is the actual data distribution

is the distribution of data generated by the constructor, and E is the expected output.

에 있어서 훈련 중 판별자는 진짜를 진짜로 가짜를 가짜로 판별해야 하므로 식의 총 출력을 최대화하려 하지만 생성자는 실제 데이터와 최대한 가까운 가짜 데이터를 만들어내어 식의 출력을 최소화하려 할 것이므로 생성적 적대 신경망은 훈련을 거듭하면서 생성자와 판별자의 신경망이 균형을 이루게 된다.

In training, the discriminator tries to maximize the total output of the expression because it has to discriminate between real and fake, but the generator will try to minimize the output of the expression by creating fake data as close as possible to the real data, so the generative adversarial neural network is trained Over time, the neural networks of generators and discriminators are balanced.

의 주변 문맥

,

,

,

에 대해 통계 후보어들 중 최대가 되는

의 문맥 확률을 계산한다.The error word checker 302 is an N-gram model that is a statistical language model, and Equation 2 is the position of the word to be corrected in which the statistical candidate word set T is substituted.

the surrounding context of

,

,

,

The largest among statistical candidates for

Calculate the context probability of

The statistical language model, used only in the error word checking stage, judges the presence of an error word by looking only at whether the word to be checked for error in the candidate word set T has a higher or lower probability than the statistical candidate words.

Statistical candidate words are different from proofreading words and are used only in the process of examining words to be corrected.

는 확률이 가장 높게 나온 예측 후보어이며,

는 실제 문서에서 나타난 단어 W대신 오류에 의해 쓰일 수 있는 단어 Y가 오용될 확률을 나타낸다.

is the predicted candidate word with the highest probability,

represents the probability of misuse of the word Y, which can be used by error instead of the word W appearing in the actual document.

As shown in Equation 3, statistical candidate words are obtained through a pre-built 3-gram dictionary, and 3-grams in the range of both two word clauses are searched based on the central word position '*'. The purpose of the search is to search all statistical candidate words that appear together with the surrounding context word at the central word position '*'.

The searched words belong to a candidate word set, and words close to each other are selected by calculating the editing distance from the word of the current target to be corrected.

The editing distance starts from 1 and serves as a standard for comparing the differences between words. The editing distance increases as the alphabet or phoneme of the comparison word is inserted, deleted, or exchanged from the reference word.

For example, in the reference word 'scissors' and the comparison word 'son-in-law', 'a' is replaced with 'ㅅ', so the edit distance for 'scissors' is 1 for 'scissors'.

The prediction candidate generator 304 uses a language model generated using a generative adversarial neural network.

와 선별부에서 선별된 교정 후보어 집합을 C라고 하고, 문맥의존 철자오류 교정 거리 값이

최대가 되는

를 선택하게 된다. Sentences input to the language model in Equation 4

and the set of candidate words for correction selected by the selection unit is called C, and the context-dependent spelling error correction distance value is

to be the maximum

will choose

를 기준으로 언어 모형의 전체 삽입 단어 사전과 설정된 편집거리를 만족하는 교정 후보어를 얻은 N개의 집합이다.C, which represents the set of candidate words for correction in Equation 4, is a central word using the edit distance calculation function EDF as in Equation 5.

Based on , it is a set of N obtained words that satisfy the entire inserted word dictionary of the language model and the set editing distance.

는 교정후보단어이고

은 교정후보단어의 주변 문맥의 크기이다.Equation 6 obtains the sum of the dot product values of the input context in Equation 4,

is a candidate word for correction

is the size of the surrounding context of the word for correction.

는 교정후보단어와 문맥단어간의 내적을 구하는 함수이며, 이 부분이 각 단어삽입 모형에서의 단어 간의 거리 값을 적용하는 부분이다.

is a function to find the dot product between the correction candidate word and the context word, and this is the part that applies the distance value between words in each word insertion model.

는 미등록어를 처리하기 위한 평탄화(smoothing)값이며, 언어 모형이 형태적으로 단어를 비교하여 미등록 언어의 특징에 따라서 내적을 비교한다면

를 따로 사용하지 않을 수도 있다.

is a smoothing value for processing unregistered words, and if the language model compares words morphologically,

may not be used separately.

The context-dependent spelling error correction method using the language model according to the present invention will be described in detail as follows.

4 is a flowchart illustrating a context-dependent spelling error correction method using a language model according to the present invention.

First, a context-dependent spelling error is searched for and the document to be corrected is input (S401), the word within the sentence in the document is sequentially checked (S402), and it is determined whether there is an error in the word (S403).

If there is no problem with the word, the next word is checked, and when it is determined that there is an error in the word, the word is determined as the word to be corrected and a tag (‘<target word>’) is applied to the word to be corrected. (S404)

In order to reduce the amount of calculation before calculating the distance from the context using the proofreading candidate word, the candidate word is determined as much as the set distance by calculating the edit distance from the entire dictionary word of the language model based on the proofreading word ( S405), the distance value between each candidate word and the surrounding context of the prediction target word is obtained based on the determined candidate word using a language model (S406), and the highest priority is selected as the final proofreading word (S407).

Here, if the proofreading word is the same as the proofreading word, proofreading has not been performed.

When the process of predicting the corrected word is finished, it is determined whether there is a next word or sentence (S408), and it is determined whether to terminate the system.

In the present invention, the context-dependent spelling error correction method using a language model sequentially checks errors from the first word to the last word based on the sentence unit.

As an example of error checking of the word to be corrected, when there is a sentence such as 'Hell, a man named mother-in-law came with scissors, but a hen can't...' It is assumed that 'I'm here', 'the seed hen', 'nail', ... are referred to as each word, and the central word currently being checked for errors is 'scissors'.

From the large 3-gram dictionary built in advance based on the central word 'scissors', a range of 2 words ('Mother-in-law', 'People', '*'), ('People', '*', ' Did you come'), ('*', 'I'm here', 'Sheen'), a set of candidate words in the context that can come in the position of '*' ', 'son-in-law', 'child', 'scissors', 'doro', etc., and the set of candidate words 'goose' that is close to the central word 'scissors' (edit distance 1 in the example) , 'son-in-law', 'children', 'scissors' and so on.

Next, the probability of forming a context within a two-word range is calculated based on the word to be searched for spelling errors. When the probability is calculated by substituting 'scissors' for each candidate word in the context of the sentence probability of 2 words, and the probability of the context including 'scissors' is the highest, it is judged that there is no error in the word and the candidate Among the words, if there is a candidate word with a higher probability of forming a context than that of 'scissors', it is judged as having an error.

When it is judged to be a word to be corrected, the first sentence in which the central word 'scissors' is replaced with '<target word>' 'Hell', 'Mother-in-law', 'People', '<target word>', 'Come here' , 'seed', 'nail', ... are put into the language model.

The language model is a language model learned in advance through deep learning, and takes the entire context set as the core word as input, and in the output, the embedding word in the corpus used in the learning process to predict the core word based on the entire context excluding the core word. Calculate the distance from the context for the entire dictionary.

In order to reduce the amount calculated through the proofreading language model, in the entire inserted word dictionary (a dictionary of candidate words obtained from the statistical model and other language models obtained in the previous example), 'scissors' and 'gooses' are close to the editing distance. ', 'son-in-law', 'dawi-ga', 'low-class', etc. are selected, and these correction candidates are ranked by calculating the distance value of the word to be corrected from the surrounding context in the language model. .

If the first-priority candidate word is the same as the word 'son-in-law' to be corrected, it is judged that there is no error in the word.

This process is sequentially repeated until the input document is finished to output the final proofreading result.

The apparatus and method for correcting a context-dependent spelling error using a language model according to the present invention described above enable processing of various errors by using the distance value between the correct word and context obtained from the language model in the spelling error correction step. will be.

The apparatus and method for correcting context-dependent spelling errors using a generative adversarial network according to the present invention described above closely simulate an actual language model using a generative adversarial network and use this information to It was made possible to respond to various errors that appear in general documents written in text.

As described above, it will be understood that the present invention is implemented in a modified form without departing from the essential characteristics of the present invention.

Therefore, the specified embodiments should be considered in an illustrative rather than a restrictive sense, the scope of the present invention is indicated in the claims rather than the foregoing description, and all differences within the equivalent scope are included in the present invention. will have to be interpreted.

301. Input
302. Error word checker
303. Candidate Selection Unit
304. Prediction candidate generator
305. Correction word presentation section

Claims (9)

an input unit for inputting a sentence for correction;
an error word checker that checks the input sentence by word unit and searches for context spelling errors;
a candidate selection unit for selecting a candidate word by calculating an editing distance between a word to be corrected and a word in the word dictionary;
a prediction candidate generator that calculates the distance between the entire surrounding context of the word to be corrected and the candidate words filtered by the candidate selector using a language model generated using a generative adversarial neural network;
A context-dependent spelling error correction apparatus using a generative adversarial neural network, comprising: a corrective word presenting unit that selects a final corrective word based on a distance calculated value between words calculated in a language model created in a generative adversarial neural network. The method of claim 1, wherein the generative adversarial neural network in the prediction candidate generator comprises:

is defined as
D is the discriminator, G is the generator,

is the actual data distribution

is the distribution of data generated by the constructor, E is the expected output,
The generator takes a random vector z as input and generates fake data, the discriminator outputs 1 when it is real and 0 when it is fake,

judging the fake data generated in the initial generator, 0 and real data x as input

A context-dependent spelling error correction apparatus using a generative adversarial neural network, characterized in that it is calculated as 1. The method of claim 1, wherein the error word checker is an N-gram model that is a statistical language model,
The position of the word to be corrected in which the statistical candidate word set T is substituted

the surrounding context of

,

,

,

The largest among statistical candidates for

Calculate the context probability of

is the probability of misuse of the word Y that can be used by error instead of the word W that appears in the actual document,
Statistical language model is a context-dependent spelling error correction device using a generative adversarial neural network, characterized in that it judges the presence or absence of an error word by only looking at whether a word to be checked for error in the candidate word set T has a higher or lower probability than the statistical candidate words. [4] The method of claim 3, wherein the statistical candidate words are obtained through a 3-gram dictionary built in advance, and 3-grams in the range of both two word clauses are searched based on the central word position '*',

is defined as
The purpose of the search is to search all statistical candidate words that appear together with the surrounding context words at the central word position '*', the searched words belong to the candidate word set, and the words and the edit distance of the current word to be corrected are calculated and close words are selected. A context-dependent spelling error correction device using a generative adversarial neural network, characterized in that The method of claim 4, wherein the prediction candidate generator uses a language model generated using a generative adversarial neural network,

using ,
Sentences input to the language model

and the set of correction candidates selected by the candidate selection unit is called C, and the context-dependent spelling error correction distance value is

to be the maximum

A context-dependent spelling error correction device using a generative adversarial neural network, characterized in that The method of claim 5, wherein C, which represents a set of proofreading candidates,

is defined as
The central word using the editing distance calculation function EDF

A context-dependent spelling error correction apparatus using a generative adversarial neural network, characterized in that it is a set of N sets obtained by obtaining the entire inserted word dictionary of the language model and the correction candidate words satisfying the set editing distance based on . The method of claim 6, wherein the corrective word presentation unit,

Using , find the sum of the dot product of the input context,

is a candidate word for correction

is the size of the surrounding context of the word for correction,

is a function to find the dot product between the correction candidate word and the context word, and is the part that applies the distance value between words in each word insertion model,

A context-dependent spelling error correction apparatus using a generative adversarial neural network, characterized in that is a smoothing value for processing unregistered words. The method of claim 1, wherein the corrective word presentation unit,
In order to predict the corrective word, the distance value of each candidate word from the surrounding context is obtained using the calculated value of the full word dictionary calculated from the language model generated by the generative adversarial neural network.
A context-dependent spelling error correction device using a generative adversarial neural network, characterized in that the highest value among the correction candidates is determined based on the calculated distance value, and the corresponding word is presented as a substitute. determining a possibility of a spelling error by examining the input sentence by word unit;
calculating an editing distance between the word to be corrected and the dictionary words in the language model to be a candidate word;
calculating a word to replace the word to be corrected by calculating the distance between the entire surrounding context of the word to be corrected and the candidate words filtered by the candidate selection unit using a language model generated using a generative adversarial neural network;
A context-dependent spelling error correction method using a generative adversarial neural network, comprising: presenting a final correction word based on the ranked information.

Images