KR102517983B1 - System and Method for correcting Context sensitive spelling error using Generative Adversarial Network - Google Patents

Abstract

The present invention closely mimics an actual language model using a generative adversarial network, and uses this information to respond to various errors in general text written documents. It relates to an apparatus and method for correcting context-dependent spelling errors using a neural network, comprising: an input unit for inputting a sentence to be corrected; an erroneous word check unit for inspecting an input sentence in word units and searching for contextual spelling errors; a word to be corrected and a word in a word dictionary. A candidate selection unit that selects candidate words by calculating an editing distance between them; Calculating the distance between the entire context surrounding the word to be corrected and candidate words filtered out by the candidate selection unit using a language model generated using a generative adversarial neural network. It includes a prediction candidate generation unit; a corrective word presentation unit that selects a final corrected word based on a distance calculation value between words calculated in a language model created in a generative adversarial neural network.

Description

System and method for correcting context sensitive spelling error using Generative Adversarial Network}

The present invention relates to spelling error correction, and more specifically, closely mimics an actual language model using a generative adversarial network, and uses this information to correct various errors appearing in text written documents. It relates to an apparatus and method for correcting context-dependent spelling errors using a generative adversarial network enabling response to

Recently, artificial intelligence technologies such as deep learning are being actively researched in various fields related to computers worldwide.

Among them, various researchers such as Google Research, Facebook Research, and AllenNLP are developing deep learning technology in relation to natural language processing worldwide, and the demand for deep learning-based programs is rapidly increasing in each field of industry. .

In natural language processing, the higher the correct sentence is initially input in the information analysis process, the higher the quality of the result. Therefore, context-dependent spelling error correction in natural language processing is an indispensable and important technique in the preprocessing process.

Spelling errors are largely divided into two types: non-word spelling errors and context sensitive spelling errors.

Simple spelling errors can be corrected more easily than context-dependent spelling errors. Error words are determined by comparing whether or not the word to be corrected is included in a dictionary.

On the other hand, in the case of context-dependent spelling errors, the degree of correction is significantly increased. For example, in the sentences "look at attention" and "look around", "attention" becomes an error word, and "attention" and "around" are Since it is a word that exists, it is difficult to solve it by correcting simple spelling errors, so it is necessary to solve it by identifying information in the surrounding context.

In the example, "attention" is an error word, but "around" may be an error word depending on the context.

Context-dependent spelling error correction methods can be divided into correction methods using rules, correction methods based on statistical information, and correction methods using neural networks.

The rule-based correction method requires experts with advanced linguistics and computer science knowledge to create and verify rules, and it is practically impossible to create rules that reflect all linguistic phenomena in the real world.

In particular, there is a high probability that errors that occur frequently or are standardized can be corrected by the rule-based method, but correction of unstructured errors caused by input errors is impossible only with the rule-based method and the correction difficulty is much higher.

The statistical correction method can be applied in a language environment where atypical errors frequently occur, and it was mainly presented before neural network-based correction.

Compared to the speed of development of neural network-based technology, it is difficult to find cases that have been applied to context-dependent spelling error correction technology.

As one of the prior art, there is a method of correcting context-dependent spelling errors using a statistical model based on the frequency of occurrence between each word of the corrected word pair and a word appearing in the surrounding context using a pre-constructed corrected word pair. ( Republic of Korea Patent Registration No. 10-1495240)

As another method, a method of using Korean lexical semantics in the process of generalizing rules to increase the recall of correction rules has been proposed. (Republic of Korea Patent Registration No. 10-1500617)

As another method, a method for solving the problem of lack of data occurring in the process of calculating the correlation between each word of the corrected word pair and the word appearing in the surrounding context is proposed. (Republic of Korea Patent Registration No. 10-1573854)

As a recently proposed method, a method for correcting context-dependent spelling errors by generating error candidates in real time for various errors has been proposed. (Republic of Korea Patent Publication No. 10-2019-0133624)

A common limitation of the prior art statistical learning method and the deep learning method is that the performance decreases considerably as the amount of training data decreases.

In certain situations, it may be difficult to obtain the desired performance because it is difficult to obtain training data, but natural language processing is no exception.

Therefore, there is a need to develop a technique for improving performance using low-volume data, and there is also a need for context-dependent spelling error correction.

Republic of Korea Patent No. 10-1495240 Republic of Korea Patent No. 10-1500617 Republic of Korea Patent No. 10-1573854 Republic of Korea Patent Publication No. 10-2019-0133624

The present invention is to solve the problems of the prior art spelling error correction technology, and uses a generative adversarial network to closely mimic an actual language model, and uses this information to create general texts. Its purpose is to provide an apparatus and method for correcting context-dependent spelling errors using a generative adversarial neural network that enables response to various errors appearing in documents.

The present invention provides an apparatus and method for correcting context-dependent spelling errors using a generative adversarial neural network that enables efficient correction of context-dependent spelling errors using low-volume data in an environment where it is difficult to obtain learning data and to obtain desired performance. It has a purpose.

The present invention uses a generative adversarial network to generate and provide learning data similar to sentences used by real people, so that rich data for quantitative learning can be provided. And to provide a method for that purpose.

An object of the present invention is to provide an apparatus and method for correcting context-dependent spelling errors using a generative adversarial network that can increase the accuracy of correction for words that have not been properly corrected due to a lack of data.

Other objects of the present invention are not limited to the above-mentioned objects, and other objects not mentioned above will be clearly understood by those skilled in the art from the description below.

In order to achieve the above object, an apparatus for correcting context-dependent spelling errors using a generative adversarial neural network according to the present invention includes an input unit for inputting a sentence to be corrected; Inspection unit; Candidate selection unit for selecting candidate words by calculating the editing distance between the word to be corrected and words in the word dictionary; The generative adversarial network calculates the distance between the entire context around the word to be corrected and the candidate words filtered out by the candidate selection unit. A prediction candidate generation unit that calculates using a language model generated using a generative adversarial neural network; a correction word presentation unit that selects a final corrected word based on a distance calculation value between words calculated from a language model created in a generative adversarial neural network. do.

In order to achieve another object, a method for correcting context-dependent spelling errors using a generative adversarial network according to the present invention includes the steps of examining an input sentence word by word and determining the possibility of a spelling error; Calculating an editing distance between dictionary words of the word to be corrected; Calculating the distance between the entire context surrounding the word to be corrected and the candidate words filtered out by the candidate selection unit using a language model generated using a generative adversarial neural network to determine the word to be corrected. Calculating a word to be replaced; Presenting a final correction word based on the ranked information; characterized in that it includes.

As described above, the apparatus and method for correcting context-dependent spelling errors using the generative adversarial neural network according to the present invention have the following effects.

First, a generative adversarial network is used to approximate an actual language model, and using this information, it is possible to respond to various errors appearing in general documents written in text.

Second, in an environment where it is difficult to obtain learning data and it is difficult to obtain the desired performance, context-dependent spelling error correction is performed efficiently using low-volume data.

Third, it generates and provides learning data similar to sentences used by real people using a generative adversarial network, so that rich data can be provided for quantitative learning.

Fourth, it is possible to increase the correction accuracy for words that have not been corrected well due to the lack of data.

1A and 1B are diagrams of language model learning using context generated by the generative adversarial neural network according to the present invention.
2 is a block diagram of a language model learning algorithm using a context generated by a generative adversarial network according to the present invention.
3 is a block diagram of an apparatus for correcting context-dependent spelling errors using a language model according to the present invention;
4 is a flowchart showing a method for correcting context-dependent spelling errors using a language model according to the present invention.

Hereinafter, preferred embodiments of an apparatus and method for correcting context-dependent spelling errors using a generative hostile neural network according to the present invention will be described in detail.

The characteristics and advantages of the apparatus and method for correcting context-dependent spelling errors using a generative adversarial neural network according to the present invention will become clear through detailed descriptions of each of the following embodiments.

1A and 1B are diagrams of language model learning using context generated by the generative adversarial neural network according to the present invention, and FIG. 2 is a diagram showing the configuration of a language model learning algorithm using context generated by the generative adversarial neural network according to the present invention. am.

An apparatus and method for correcting context-dependent spelling errors using a generative adversarial network according to the present invention closely mimics an actual language model using a generative adversarial network and uses this information to generate text written in text. This is to make it possible to respond to various errors that appear in general documents.

To this end, the present invention generates a sentence through a generative adversarial neural network, learns a language model using the generated sentence, and identifies and corrects the relationship between the word to be corrected and the surrounding context of the word to be corrected through the language model. includes

A generative adversarial neural network is a method of solving data generation problems by approaching based on von Neumann's minmax theorem, one of game theories. Similarly, a deep neural network composed of a generate model and a discriminate model of data learns in an opposing way while repeatedly generating and discriminating each other dependently.

As this learning is repeated, when the moment the discriminator cannot distinguish between real and fake data is reached, the Nash equilibrium, which is said in game theory, is reached, which means that a random sentence generated by a generator is fake This means that the discriminator, which used to be classified as , is no longer able to distinguish the sentences generated by the generator developed through learning.

Finally, the sentences generated by the generator become meaningful information as they become similar to sentences written by actual people.

In the correction in the present invention, a language model close to reality is learned using generated sentences, and correction is performed using the language model.

The present invention is also applied to generative adversarial networks of various structures, and is not limited to the structures in FIGS. 1A and 1B.

2 shows an example of an algorithm for generating sentences using a generative adversarial network according to the present invention and reflecting them in a language model.

A context-dependent spelling error correction device using a language model according to the present invention will be described in detail.

3 is a block diagram of an apparatus for correcting context-dependent spelling errors using a language model according to the present invention.

As shown in FIG. 3, the apparatus for correcting context-dependent spelling errors using a generative adversarial network according to the present invention includes an input unit 301 for inputting a sentence to correct an error, and a word of a sentence input through the input unit 301. An error word checking unit 302 that sequentially checks for errors, a candidate selection unit 303 that selects correction candidate words for the word to be corrected when it is determined that there is an error in the word, and a context surrounding the position of the word to be corrected and candidates It includes a prediction candidate generation unit 304 that calculates a distance value between words, and a correction word presentation unit 305 that finally presents a candidate word close to the context.

The correction process using the context-dependent spelling error correction device using the generative adversarial network according to the present invention having such a configuration will be described in detail for each step.

All words used for proofreading are expressed using the generated language model, and the language model refers to a model used for understanding natural language.

Equation 1 represents a basic generative adversarial network, where D is the discriminant model (or discriminator) and G is the generative model (or generator).

은 초기에 생성자에서 생성된 가짜 데이터를 판단하여 0, 실제 데이터 x를 입력으로 하는

는 1로 계산된다.The generator takes the random vector z as input and generates fake data, and the discriminator outputs 1 when it is real and 0 when it is fake.

determines the fake data initially generated by the constructor and takes 0 and the real data x as input

is counted as 1.

는 실제 데이터 분포이고

는 생성자에서 생성한 데이터의 분포이며, E는 예상되는 출력이다.

is the actual data distribution and

is the distribution of data produced by the constructor, and E is the expected output.

에 있어서 훈련 중 판별자는 진짜를 진짜로 가짜를 가짜로 판별해야 하므로 식의 총 출력을 최대화하려 하지만 생성자는 실제 데이터와 최대한 가까운 가짜 데이터를 만들어내어 식의 출력을 최소화하려 할 것이므로 생성적 적대 신경망은 훈련을 거듭하면서 생성자와 판별자의 신경망이 균형을 이루게 된다.

During training, the discriminator must discriminate real from real and fake from fake, so it tries to maximize the total output of the equation, but the generator tries to minimize the output of the equation by creating fake data as close as possible to the real data. Repeatedly, the generator and discriminator neural networks are balanced.

의 주변 문맥

,

,

,

에 대해 통계 후보어들 중 최대가 되는

의 문맥 확률을 계산한다.The error checking unit 302 is an N-gram model, which is a statistical language model, and Equation 2 is the position of the word to be corrected to which the set of statistical candidate words T is replaced.

the surrounding context of

,

,

,

The largest of the statistical candidates for

Calculate the context probability of

Statistical language models, which are used only in the error word checking step, determine the presence or absence of an error word by looking only at whether the probability of the word subject to error checking in the candidate word set T is higher or lower than that of the statistical candidate words.

Statistical candidate words are different from proofreading candidate words, and are used only in the process of examining the word to be proofread.

는 확률이 가장 높게 나온 예측 후보어이며,

는 실제 문서에서 나타난 단어 W대신 오류에 의해 쓰일 수 있는 단어 Y가 오용될 확률을 나타낸다.

is the prediction candidate with the highest probability,

represents the probability of misuse of the word Y, which could be used by error instead of the word W appearing in the actual document.

As in Equation 3, statistical candidates are obtained through a pre-built 3-gram dictionary, and 3-grams in the range of two words on both sides are searched based on the central word position '*'. The purpose of the search is to search all statistical candidate words appearing together with the surrounding context words of the central word position '*'.

The searched words belong to a set of candidate words, and close words are selected by calculating an editing distance from the word of the current target word to be corrected.

The edit distance starts from 1, and is a criterion for comparing differences between words. The edit distance increases as alphabets or phonemes of the comparison word are inserted, deleted, or exchanged from the reference word.

For example, in the standard word ‘scissors’ and the comparison word ‘scissors’, ‘ㄱ’ is exchanged for ‘ㅅ’, so ‘scissors’ has an editing distance of 1 for ‘scissors’.

The prediction candidate generation unit 304 uses a language model generated using a generative adversarial neural network.

와 선별부에서 선별된 교정 후보어 집합을 C라고 하고, 문맥의존 철자오류 교정 거리 값이

최대가 되는

를 선택하게 된다. Sentences input to the language model in Equation 4

and the set of correction candidate words selected in the selection unit is called C, and the context-dependent spelling error correction distance value is

maximum

will choose

를 기준으로 언어 모형의 전체 삽입 단어 사전과 설정된 편집거리를 만족하는 교정 후보어를 얻은 N개의 집합이다.C, which represents the set of correction candidate words in Equation 4, is the center word using the edit distance calculation function EDF as in Equation 5

Based on , N sets of correction candidates that satisfy the entire inserted word dictionary of the language model and the set editing distance are obtained.

는 교정후보단어이고

은 교정후보단어의 주변 문맥의 크기이다.Equation 6 calculates the sum of the dot products of the input contexts in Equation 4,

is a correction candidate word

is the size of the surrounding context of the correction candidate word.

는 교정후보단어와 문맥단어간의 내적을 구하는 함수이며, 이 부분이 각 단어삽입 모형에서의 단어 간의 거리 값을 적용하는 부분이다.

is a function that obtains the dot product between the correction candidate word and the context word, and this part applies the distance value between words in each word embedding model.

는 미등록어를 처리하기 위한 평탄화(smoothing)값이며, 언어 모형이 형태적으로 단어를 비교하여 미등록 언어의 특징에 따라서 내적을 비교한다면

를 따로 사용하지 않을 수도 있다.

is a smoothing value for processing non-registered words, and if the language model compares words morphologically and compares dot products according to the characteristics of unregistered languages,

may not be used separately.

A method for correcting context-dependent spelling errors using a language model according to the present invention will be described in detail.

4 is a flowchart illustrating a method for correcting context-dependent spelling errors using a language model according to the present invention.

First, search for context-dependent spelling errors and enter a document to be corrected (S401), sequentially inspect words in sentences in the document (S402), and determine whether there is an error in the word (S403).

If there is no problem with the word, the next word is checked, and when it is determined that the word has an error, the corresponding word is determined as the word to be corrected and a tag (‘<target word>’) is applied to the word to be corrected (S404).

In order to reduce the amount of calculation prior to calculating the distance from the context using the correction candidate word, the editing distance with the entire dictionary word of the language model is calculated based on the correction target word, and the candidate word is determined by the set distance ( S405), based on the determined candidate words, a distance value between the surrounding context of the target word to be predicted and each candidate word is obtained using a language model (S406), and the highest first rank is selected as the final correction word (S407).

Here, if the correction word is the same as the proofreading target word, proofreading is not performed, and if the proofreading word is different from the proofreading target word, correction is performed by substitution (S407).

When the process of predicting the correction word is finished, it is determined whether there is a next word or sentence (S408), and it is determined whether to terminate the system.

In the present invention, the method for correcting context-dependent spelling errors using a language model sequentially checks errors from the first word to the last word on a sentence-by-sentence basis.

As an example of the error check of the word to be corrected, when there is a sentence 'A mother-in-law came with scissors, but a hen couldn't...' It is assumed that 'Where's you from', 'The hen', 'Nail', ... are each word, and the central word that is currently being checked for errors is 'scissors'.

In the large-capacity 3-gram dictionary built in advance based on the central word 'scissors', the two-word range ('mother-in-law', 'person', '*'), ('person', '*', ' A set of candidate words in the context that can appear in the position of '*' by searching for ('*', 'Coming', 'Mr. ', 'Sawiga', 'Hawiga', 'Gawiga', 'Doroga', etc., and a set of candidate words 'goosega' with close editing distance (editing distance 1 in the example) to the central word 'Gawiga'. .

Next, based on the word to be searched for spelling errors, the probability of the context of the 2-word range is calculated. Here, the context of the 2-word range including the central word 'scissors' is 'The mother-in-law is the son-in-law' When comparing the calculated probability by replacing 'scissors' with each candidate word in the sentence probability of ' and the context of the two-word range, if the probability of the context including 'scissors' is the highest, it is determined that there is no error in the word, and the candidate Among the words, if there is a candidate word whose probability of forming with the context is higher than the probability of 'scissors', it is determined that there is an error.

When it is judged to be a word to be corrected, the first sentence in which the key word 'scissors' is replaced with '<target word>' is 'what the hell', 'mother-in-law', 'person', '<target word>', and 'Wongaundi' , 'the hen', 'nail', ... are put into the language model.

The language model is a language model pre-learned through deep learning, takes the entire context set as the central word as input, and predicts the central word based on the entire context except for the central word in the output. The embedding word of the corpus used in the learning process The distance from the context is calculated for the entire dictionary.

In order to reduce the amount calculated through the correction language model, 'gowiga' and 'goose', which have a close editing distance to 'scissors' in the entire inserted word dictionary (dictionaries of candidate words obtained from the statistical model and other language models in the previous example) ', 'Sawiga', 'Dawiga', 'Hawiga', etc. are selected, and among these correction candidates, the language model calculates the distance value from the surrounding context of the target word to be corrected and ranks them. .

If the first candidate word is the same as the word 'Sawiga' to be corrected, it is determined that there is no error in the word, and if it appears among other candidate words, it is determined that the word is erroneous and corrected.

This process is repeated sequentially until the input document is finished, and the final proofreading result is output.

The apparatus and method for correcting context-dependent spelling errors using the language model according to the present invention described above enable processing of various errors by using the distance value between the word to be corrected and the context obtained from the language model in the spelling error correction step. will be.

The apparatus and method for correcting context-dependent spelling errors using a generative adversarial network according to the present invention described above closely mimics an actual language model using a generative adversarial network, and uses this information to It is possible to respond to various errors that appear in general documents written in text.

As described above, it will be understood that the present invention is implemented in a modified form without departing from the essential characteristics of the present invention.

Therefore, the specified embodiments should be considered from an explanatory point of view rather than a limiting point of view, and the scope of the present invention is shown in the claims rather than the foregoing description, and all differences within the equivalent range are considered to be included in the present invention. will have to be interpreted

301. Input section
302. Error word check unit
303. Candidate selection department
304. Prediction candidate generator
305. Proofreading section

Claims (9)

an input unit for inputting sentences to be corrected;
an error word checking unit that checks the input sentence by word unit and searches for contextual spelling errors;
a candidate selector selecting a candidate word by calculating an editing distance between a word to be corrected and a word in the word dictionary;
a prediction candidate generation unit that calculates a distance between the entire context around the word to be corrected and candidate words filtered out by the candidate selection unit using a language model generated using a generative hostile neural network;
An apparatus for correcting context-dependent spelling errors using a generative adversarial network, comprising: a correction word presenter for selecting a final corrected word based on a distance calculation value between words calculated in a language model created in a generative adversarial neural network. The method of claim 1, wherein the generative adversarial neural network in the prediction candidate generator,

is defined as,
D is the discriminator, G is the generator,

is the actual data distribution and

is the distribution of data produced by the constructor, E is the expected output,
The generator takes a random vector z as input and generates fake data, and the discriminator outputs 1 when it is real and 0 when it is fake.

determines the fake data initially generated by the constructor and takes 0 and the real data x as input

A context-dependent spelling error correction device using a generative adversarial network, characterized in that is calculated as 1. The method of claim 1, wherein the error word check unit is an N-gram model, which is a statistical language model,
The position of the word to be corrected where the set of statistical candidate words T is replaced

the surrounding context of

,

,

,

The largest of the statistical candidates for

Calculate the context probability of

is the probability of misuse of the word Y, which could be used by error instead of the word W appearing in the actual document,
The statistical language model is a context dependent spelling error correction device using a generative adversarial network, characterized in that it determines the presence or absence of an error by only looking at whether the error test target word has a higher or lower probability than the statistical candidate words in the candidate word set T. The method of claim 3, wherein the statistical candidate words are obtained through a pre-built 3-gram dictionary, and 3-grams in the range of two words on both sides are searched based on the central word position '*',

is defined as,
The purpose of the search is to search for all statistical candidate words that appear together with the surrounding context words of the central word position '*', the searched words belong to the candidate word set, and select words close to the words of the current target word to be corrected by calculating the editing distance. A context-dependent spelling error correction device using a generative adversarial network, characterized in that. The method of claim 4, wherein the prediction candidate generation unit uses a language model generated using a generative adversarial network,

using,
Sentences input to the language model

and the set of correction candidates selected by the candidate selection unit is called C, and the context-dependent spelling error correction distance value is

maximum

Context-dependent spelling error correction device using a generative adversarial network, characterized in that for selecting. The method of claim 5, wherein C representing a set of correction candidates,

is defined as,
Using the editing distance calculation function EDF, the center word

A context-dependent spelling error correction device using a generative adversarial network, characterized in that N sets of correction candidate words that satisfy the entire insertion word dictionary of the language model and the set editing distance are obtained based on. The method of claim 6, wherein the correction word presentation unit,

Using , the sum of the dot products of the input context is obtained,

is a correction candidate word

is the size of the surrounding context of the correction candidate word,

is a function that obtains the dot product between the correction candidate word and the context word, and is a part that applies the distance value between words in each word embedding model,

A context-dependent spelling error correction device using a generative hostile neural network, characterized in that is a smoothing value for processing unregistered words. The method of claim 1, wherein the correction word presentation unit,
In order to predict the correction word, the distance value between the surrounding context and each candidate word is obtained using the calculated value of the entire word dictionary calculated in the language model generated by the generative adversarial neural network,
An apparatus for correcting context-dependent spelling errors using a generative adversarial network, characterized in that, based on the calculated distance value, the final corrected word is determined with the highest value among the corrected candidate words, and the corresponding word is presented as a substitute word. When a sentence to be corrected is input through the input unit, determining the possibility of a spelling error by inspecting the input sentence in word units in an erroneous word checking unit;
calculating an editing distance between a word to be corrected and dictionary words in a language model to be candidate words in a candidate selector;
Calculating a word to replace the word to be corrected by calculating the distance between the entire context around the word to be corrected in the prediction candidate generation unit and the candidate words filtered out in the candidate selection unit using a language model generated using a generative hostile neural network ;
A method for correcting context-dependent spelling errors using a generative adversarial neural network, comprising: presenting a final correction word based on the ranked information in the correction word presentation unit.

Images