KR102559687B1 - Method and apparatus for correcting grammatical error, and training method of deep learing model therefor - Google Patents

Abstract

A spelling correction method and apparatus using a deep learning model, and a method for learning a deep learning model therefor are disclosed. A spelling correction method according to an embodiment includes receiving a sentence and correcting the spelling of the sentence by inputting the sentence to a deep learning model, wherein the deep learning model is a deep learning model learned based on a normal sentence and an erroneous sentence generated by changing at least one of a word and a morpheme of the normal sentence.

Description

Spelling correction method using deep learning model, apparatus and learning method of deep learning model for the same

The present disclosure relates to a spelling correction method using a deep learning model, an apparatus, and a deep learning model learning method therefor.

Since the era of rapidly growing big data, access to text data is easy and it is widely applied in various fields. In this situation, an appropriate language processing method is required to process the vast amount of unorganized data. Various efforts are being applied to develop a Grammatical Error Correction (GEC) model for handling text data errors. However, spelling error correction models for Korean remain relatively unexplored.

As related prior art, there is Korean Patent Publication No. 10-2006-0043682 (Title of Invention: System and Method for Improved Spelling Check).

The present invention can provide a technique for easily generating a data set for learning a spelling correction model and learning a high-performance spelling correction model based on the generated data set.

However, the technical challenges are not limited to the above-described technical challenges, and other technical challenges may exist.

A spelling correction method according to an embodiment includes receiving a sentence and correcting the spelling of the sentence by inputting the sentence to a deep learning model, wherein the deep learning model is a deep learning model learned based on a normal sentence and an erroneous sentence generated by changing at least one of a word and a morpheme of the normal sentence.

When the word is a predicate, the erroneous sentence may be generated by changing the word based on the conjugation of a verb that can be made using the stem and ending of the word.

When the word is not a predicate, the erroneous sentence may be generated by changing the word into an error for the word included in the spelling error list.

The erroneous sentence may be generated by changing the morpheme to an error for the morpheme included in the spelling error list.

An apparatus for correcting spelling according to an embodiment includes a memory including instructions and a processor for executing the instructions. When the instructions are executed by the processor, the processor receives a sentence, inputs the sentence to a deep learning model, corrects the spelling of the sentence, and the deep learning model is a deep learning model learned based on a normal sentence and an erroneous sentence generated by changing at least one of a word and a morpheme of the normal sentence.

When the word is a predicate, the erroneous sentence may be generated by changing the word based on the conjugation of a verb that can be made using the stem and ending of the word.

When the word is not a predicate, the erroneous sentence may be generated by changing the word into an error for the word included in the spelling error list.

The erroneous sentence may be generated by changing the morpheme to an error for the morpheme included in the spelling error list.

A method of learning a deep learning model for spelling correction according to an embodiment may include generating an erroneous sentence including a spelling error based on at least one of a word and a morpheme of an input sentence, inputting the erroneous sentence to the deep learning model, and learning the deep learning model based on an output of the deep learning model and the input sentence.

The generating step may include determining whether the word is a predicate, changing the word into another conjugation if the word is a predicate, and changing the word to an error included in a spelling error list if the word is not a predicate.

The transforming may include extracting the stem and ending of the word, extracting conjugations that can be made using the stem and ending, and changing the word by selecting one of the conjugation forms based on a uniform distribution.

The generating may include changing the morpheme into an error for the morpheme included in a spelling error list.

The generating may include changing a morpheme having the highest error occurrence frequency among morphemes of the input sentence.

The learning method may further include a preprocessing step of dividing the input sentence into words.

1 illustrates a spelling correction device according to an embodiment.
2 shows a learning method of a spelling correction model.
Figure 3 shows the network structure of the spelling correction model.

Specific structural or functional descriptions of the embodiments are disclosed for illustrative purposes only, and may be changed and implemented in various forms. Therefore, the form actually implemented is not limited only to the specific embodiments disclosed, and the scope of the present specification includes changes, equivalents, or substitutes included in the technical idea described in the embodiments.

Although terms such as first or second may be used to describe various components, such terms should only be construed for the purpose of distinguishing one component from another. For example, a first element may be termed a second element, and similarly, a second element may be termed a first element.

It should be understood that when an element is referred to as being “connected” to another element, it may be directly connected or connected to the other element, but other elements may exist in the middle.

Singular expressions include plural expressions unless the context clearly dictates otherwise. In this specification, terms such as "comprise" or "have" are intended to designate that the described features, numbers, steps, operations, components, parts, or combinations thereof exist, but it should be understood that the presence or addition of one or more other features or numbers, steps, operations, components, parts, or combinations thereof is not precluded.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related art, and are not interpreted in an ideal or excessively formal sense unless explicitly defined herein.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. In the description with reference to the accompanying drawings, the same reference numerals are given to the same components regardless of reference numerals, and overlapping descriptions thereof will be omitted.

1 illustrates a spelling correction device according to an embodiment.

The spelling correction device 100 may correct spelling errors. The spelling correction device 100 may correct spelling errors in Korean sentences using a deep learning model.

A deep learning model for spelling correction may be trained using a denoising autoencoder method. For example, a spelling correction model can be trained to output input data based on input data with errors added.

The spelling correction device 100 may include a processor 130 and a memory 150 . The operation of the spelling correction device 100 may be performed by the processor 130 and the memory 150 .

Memory 150 may store instructions (or programs) executable by processor 130 . For example, the instructions may include instructions for executing an operation of the processor 130 and/or an operation of each component of the processor 130 .

The memory 150 may be implemented as a volatile memory device or a nonvolatile memory device. The volatile memory device may be implemented as dynamic random access memory (DRAM), static random access memory (SRAM), thyristor RAM (T-RAM), zero capacitor RAM (Z-RAM), or twin transistor RAM (TTRAM). Non-volatile memory devices include EEPROM (Electrically Erasable Programmable Read-Only Memory), Flash memory, MRAM (Magnetic RAM), Spin-Transfer Torque (STT)-MRAM (Spin-Transfer Torque (STT)-MRAM), Conductive Bridging RAM (CBRAM), FeRAM (Ferroelectric RAM), PRAM (Phase change RAM), Resistive RAM (RRAM), Nanotube RRAM (Nanotube RRAM), polymer It can be implemented as polymer RAM (PoRAM), nano floating gate memory (NFGM), holographic memory, molecular electronic memory device, or insulator resistance change memory.

The memory 150 may store a matrix on which an operation included in the neural network is to be performed. The memory 150 may store an operation result generated by processing by the processor 130 .

The processor 130 may process data stored in the memory 150 . The processor 130 may execute computer readable code (eg, software) stored in the memory 150 and instructions triggered by the processor 130 .

The processor 130 may be a hardware-implemented data processing device having a circuit having a physical structure for executing desired operations. For example, desired operations may include codes or instructions included in a program. For example, a data processing unit implemented as hardware may include a central processing unit, a graphics processing unit, a neural network processing unit, a multi-core processor, a multiprocessor, an application-specific integrated circuit (ASIC), and a field programmable gate array (FPGA).

2 shows a learning method of a spelling correction model, and FIG. 3 shows a network structure of a spelling correction model.

The spelling correction model 300 may be trained using Korean sentences. For example, Korean sentences may be acquired from online encyclopedias (eg, Wikipedia, Naver Encyclopedia). In addition, a corpus provided by the National Institute of the Korean Language can be used as a learning data set. In this case, the Korean sentences may be normal sentences that do not include spelling errors.

The spelling correction model 300 may be learned using a denoising autoencoder method.

The learning method 200 of the spelling correction model 300 may include a pre-processing step 210, a pre-training step 220, a training step 230, and a post-processing step 240.

The preprocessing step 210 may be a step of dividing an input sentence into subwords. For example, an input sentence input as training data to the spelling correction model 300 in the preprocessing step 210 may be separated into words through a sentencepiece source.

The pre-learning step 220 may be a step of adding errors to the input sentence. In the pre-learning step 220, an erroneous sentence including a spelling error may be generated based on at least one of a word and a morpheme of the input sentence.

An error sentence may be generated based on a word. It may be determined whether a word included in the input sentence is a predicate. For example, whether a word is a predicate may be determined based on a soylemma dictionary or a python dictionary for Korean NLP.

If the word is a predicate, an error sentence can be created by changing the word to another conjugation. Words that are predicate words can be separated into stems and endings, and conjugations of verbs that can be made based on the stems and endings can be extracted. The changed utilization type may be selected from utilization types extracted based on uniform distribution.

If the word is not a predicate, the word may be changed to an error included in the spelling error list. For example, the misspelling error list may include a list including frequent misspelling errors, and the like.

Error sentences may be generated based on morphemes. An error sentence may be generated by changing a morpheme included in an input sentence. An erroneous sentence may be generated based on a noise function that replaces correct morphemes of an input sentence with erroneous ones.

The noise function can change morphemes based on a misspelling list that includes correct morphemes and corresponding errors. The noise function can randomly select some of the morphemes included in the input sentence and replace them with errors included in the spelling error list. For example, the noise function may generate an error sentence based on the Korean learner corpus provided by the National Institute of the Korean Language. The Korean learner's corpus may include correct morphemes and corpora with errors corresponding to the morphemes.

The learning step 230 may be a step of learning the spelling correction model 300 based on the pair of input sentences and erroneous sentences. The learning method of the spelling correction model 300 can be expressed as Equation 1.

x and x+noise are input sentences and error sentences, respectively, Enc(*) and Dec(*) are encoder and decoder functions of the spelling correction model 300, and Loss(*) is a loss function.

The spelling correction model 300 may be a Transformer model. The spelling correction model 300 may include six block models of 512 to 2048 units including eight attention heads and pre-attention layer normalization.

The post-processing step 240 may be a step of changing the failed portion again when the spelling correction model 300 fails in correction. For example, in the post-processing step, an unknown word output from the spelling correction model 300 is displayed as “unk”, and the corresponding part may be changed to another word again.

The embodiments described above may be implemented as hardware components, software components, and/or a combination of hardware components and software components. For example, the devices, methods and components described in the embodiments may be implemented using a general purpose computer or special purpose computer, such as, for example, a processor, controller, arithmetic logic unit (ALU), digital signal processor, microcomputer, field programmable gate array (FPGA), programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and software applications running on the operating system. A processing device may also access, store, manipulate, process, and generate data in response to execution of software. For convenience of understanding, there are cases in which one processing device is used, but those skilled in the art will recognize that the processing device may include a plurality of processing elements and/or multiple types of processing elements. For example, a processing device may include a plurality of processors or a processor and a controller. Other processing configurations are also possible, such as parallel processors.

Software may include a computer program, code, instructions, or a combination of one or more of these, and may configure a processing device to operate as desired, or may independently or collectively direct a processing device. Software and/or data may be permanently or temporarily embodied in any tangible machine, component, physical device, virtual equipment, computer storage medium or device, or transmitted signal wave, to be interpreted by, or to provide instructions or data to, a processing device. Software may be distributed on networked computer systems and stored or executed in a distributed manner. Software and data may be stored on computer readable media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer readable medium. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination, and the program instructions recorded on the medium may be specially designed and configured for the embodiment or may be known and usable to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specially configured to store and execute program instructions such as ROM, RAM, and flash memory. Examples of program instructions include high-level language codes that can be executed by a computer using an interpreter, as well as machine language codes such as those produced by a compiler.

The hardware device described above may be configured to operate as one or a plurality of software modules to perform the operations of the embodiments, and vice versa.

As described above, although the embodiments have been described with limited drawings, those skilled in the art can apply various technical modifications and variations based on this. For example, even if the described techniques are performed in an order different from the described method, and/or components of the described system, structure, device, circuit, etc. are combined or combined in a different form from the described method, or replaced or substituted by other components or equivalents, appropriate results can be achieved.

Therefore, other implementations, other embodiments, and equivalents of the claims are within the scope of the following claims.

Claims (14)

In the spelling correction method performed by the spelling correction device,
receiving a Korean sentence by the spelling correction device; and
Correcting the spelling of the sentence by inputting the sentence to a deep learning model by the spelling correction device
including,
The deep learning model,
A denoising autoencoder-based deep learning model learned based on a normal sentence and an erroneous sentence generated by changing at least one of the word and morpheme of the normal sentence,
The error statement is
If the word is a predicate word, it is generated by changing the word based on the conjugation of a verb that can be made using the stem and ending of the word,
How to correct spelling.
delete According to claim 1,
The error statement is
If the word is not a predicate, the spelling correction method generated by changing the word to an error for the word included in the spelling error list.
According to claim 1,
The error statement is
A spelling correction method generated by changing the morpheme to an error for the morpheme included in a spelling error list.
memory containing instructions; and
A processor for executing the above instructions
including,
When the instructions are executed by the processor, the processor:
Receive a Korean sentence, input the sentence to a deep learning model to correct the spelling of the sentence,
The deep learning model,
A denoising autoencoder-based deep learning model learned based on a normal sentence and an erroneous sentence generated by changing at least one of the word and morpheme of the normal sentence,
The error statement is
If the word is a predicate word, it is generated by changing the word based on the conjugation of a verb that can be made using the stem and ending of the word,
Spell corrector.
delete According to claim 5,
The error statement is
If the word is not a predicate, the spelling correction device generated by changing the word to an error for the word included in the spelling error list.
According to claim 5,
The error statement is
A spelling correction device generated by changing the morpheme to an error for the morpheme included in a spelling error list.
In the learning method of a deep learning model for Korean spelling correction performed by a spelling correction device,
generating, by the spell correction device, an erroneous sentence including a spelling error based on at least one of a word and a morpheme of the input sentence;
inputting the erroneous sentence into the deep learning model by the spelling correction device; and
Learning, by the spell correction device, the deep learning model based on a denoising autoencoder based on the output of the deep learning model and the input sentence;
including,
The generating step is
extracting the stem and ending of the word when the word is a predicate;
extracting conjugations that can be made using the stem and the ending; and
Changing the word by selecting one of the conjugations based on uniform distribution
Including, learning method.
According to claim 9,
The generating step is
If the word is not a predicate, changing the word to an error included in a spelling error list.
Including, learning method.
delete According to claim 9,
The generating step is
Changing the morpheme to an error about the morpheme included in the spelling error list
Including, learning method.
According to claim 12,
The step of changing the morpheme to an error,
Changing a morpheme with the highest error occurrence frequency among morphemes of the input sentence
Including, learning method.
According to claim 9,
A pre-processing step of dividing the input sentence into words by the spelling correction device.
Further comprising, learning method.