KR101158331B1 - Checking meth0d for consistent word spacing - Google Patents

Abstract

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a spacing test method, and more particularly, to an automatic spacing method for easily checking a portion of a spacing performed inconsistently using syllable information and morpheme information. As described above, the present invention provides a spacing check method for checking a portion of a document that is inconsistently spaced in a document, and extracts a portion of left and right words centered on a blank (blank) in a document to form a spacing word (type A target string). Constructing a navigation key to confirm; Extracting words (type B target strings) pasted by substring search using the constructed search key; And displaying the spacing word and the extracted paste word corresponding to the search key as an inconsistent spacing word, wherein the extracting includes: a vocabulary for extracting a paste word that completely matches the search key; By morphologically analyzing the results retrieved through the matching model, the same syllable candidate matching model which extracts the pasted words by processing the same syllable candidate for the last syllable of the search key, or the same syllable candidate matching model. It is characterized in that any one of the morpheme matching models for extracting the same pasted phrases is performed.

Description

How to check for space spacing consistency {CHECKING METH0D FOR CONSISTENT WORD SPACING}

The present invention relates to a spacing coherence checking method, and more particularly, to a spacing coherence checking method that makes it possible to easily check a portion of a spacing performed inconsistently using syllable information and morpheme information.

Automatic spacing refers to the process of correctly restoring misplaced words in a sentence. Especially in Korean, it is very important not only for readability but also to convey accurate meaning. There are many errors in spacing in articles written by Internet users. Most of these spacing errors are caused by unintended mistakes or ignorance of correct spelling.

Spacing in Korean sentences improves readability and clarifies meaning. For proper spacing, the National Institute of Korean Language sets Hangul spelling and keeps all authors.

However, some of the spacing rules allow multiple rules, which may result in inconsistent spacing for the same word or word.

The following is a case of allowing multiple rules among the spacing rules of Korean spelling.

Article 47: Supplementary verbs shall be written in principle, but may be added in some cases.

Article 48: The first and last names, first and last names shall be pasted, and additional names, office names, etc. shall be used. However, if it is necessary to clearly distinguish the first and last names, the last name may be used.

Section 49: Proper nouns other than names are to be written word by word, but may be written unit by word.

Article 50: The terminology should be written in words, but can be pasted.

This plural rule treats both spellings as correct, but because different spacing for the same word or word in the same document is wrong in principle of document correction, it is modified to maintain consistency when publishing the document.

For example, if a word is written in a document such as 'document correction system', 'document correction system', 'document correction system', and so on, it is corrected and displayed in one unified space form.

Such inconsistent spacing actually occurs a lot, and corrective work for this is one of very troublesome tasks when writing or writing text contents.

In addition, since a translation unit or an index word is mostly selected as a spacing unit in an automatic translation system or an information retrieval system, spacing consistency may affect the performance of a natural language processing program.

For example, a compound word composed of two words is consistently used as a translation or index in the document when pasted or spaced consistently, but when used inconsistently, the pasted and spaced words are translated differently or Can be indexed and searched.

In particular, incorrect statistical information can be generated in automatic sorting in statistical translation systems or word similarity calculation in IR systems.

Thus, spacing consistency may be utilized for normalization of the input document.

Korean word processors currently on the market usually support automatic spacing but do not address spacing consistency.

In addition, although many researches on automatic spacing have been conducted in academia, no research has been conducted in terms of spacing consistency.

The spacing system is one of spelling corrections, and there is an automatic spacing system which is used to make a spacing correction, and pastes a document that is hardly spaced, and then respaces according to the rules.

First of all, the calibration system was mainly developed to correct the spacing between two words.

And the automatic spacing system was developed as a way to automatically space between three or more words or whole sentences.

Both of these systems are based on rules, statistics, and a combination of both.

The rule-based method first analyzes one word or two words, and then analyzes the types of misses (prefixes) and spaces (prefixes). Provide correct spacing as a basis.

At this time, if the morphological analysis fails, the error type is analyzed by separating the word into a compound word or by attempting a morphological analysis on the substring.

However, the method according to the above rule has a problem that the limitations of the morpheme analyzer remain intact. In other words, errors in the morpheme analyzer itself and processing limitations on unregistered words become limits of performance improvement.

Next, the statistical method is to divide words by using statistical information to avoid excessive morphological analysis. To this end, the number of divisions between syllables and the number of divisions between words within words and word boundaries are statistically calculated and used for separation, or words are separated using syllable characteristics such as survey and ending. Most statistical methods are used as a preliminary step in complex methods because of the lack of accuracy due to lack of statistical data. In the complex method, the accuracy of the word separated by the statistical method is verified through morphological analysis.

The spacing correction system allows multiple rules because it corrects incorrect spacing and therefore does not handle consistent spacing.

You can also think that most automatic spacing systems can be used for consistent spacing because all the words are pasted and the system automatically spaces them. However, at present the performance of these systems is usually not as perfect as word accuracy, which is 93%, so many errors can still be included. In addition, the author may ignore intentional spacing or pasting, and the system may misjudge the rule or statistical information and correct it.

Therefore, although human intervention is required after the automatic spacing process, the automatic spacing system does not provide a method for checking the inconsistent words entirely.

The present invention has been made in view of the above problems, and an object of the present invention is to provide a spacing consistency checking method for easily checking a portion of a spacing that is performed inconsistently using syllable information and morpheme information.

Another object of the present invention is to provide a spacing consistency checking method for performing a spacing consistency check at a lexical level to display and display a portion in which spacing is performed inconsistently.

The objects of the present invention are not limited to the above-mentioned objects, and other objects not mentioned can be clearly understood by those skilled in the art from the following description.

In the spacing consistency checking method according to an aspect of the present invention for achieving the above object, in the spacing check method for checking a portion inconsistent spacing in the document, the left and right words around the blank (blank) in the document Constructing a search key to extract a predetermined portion and identify a spaced word (type A target string); Extracting words (type B target strings) pasted by substring search using the constructed search key; And displaying the spacing word and the extracted paste word corresponding to the search key as an inconsistent spacing word, wherein the extracting includes: a vocabulary for extracting a paste word that completely matches the search key; By morphologically analyzing the results retrieved through the matching model, the same syllable candidate matching model which extracts the pasted words by processing the same syllable candidate for the last syllable of the search key, or the same syllable candidate matching model. It is characterized in that any one of the morpheme matching models for extracting the same pasted phrases is performed.

Here, the predetermined part of the left and right words, it is characterized in that the first word of the whole word and the right word of the left word.

Here, if the pasted word is not extracted, the method may further include removing the words found by the search key from the object of inconsistent spacing.

In this case, the same syllable candidate matching model, if the last syllable of the search key and the syllable compared in the target string belong to the same syllable candidate (syllables that are stemmed but may appear as utilization types), the string is treated as a match. It is characterized by.

Here, the morpheme matching model compares the parts of speech and the lexical level at the same time according to the results of morphological analysis of the written word and the attached word in consideration of the utilization form, and when the morpheme is a part of speech such as a noun or a verb, If the two words are determined to be the same, and the morphemes other than the noun or the verb are determined according to the morphological analysis result, if the parts of speech are the same, the two words are determined to be the same morpheme. Even if it matches any one of the candidates, the semantic is distinguished by the same word.

The method may further include providing a service for editing by connecting a hyperlink to a corresponding text position of a specific word displayed as the inconsistent spacing word.

A computer-readable recording medium having recorded thereon a program according to another aspect of the present invention is a computer that checks a portion of spaces written inconsistently in a document, and extracts a predetermined portion of left and right words centered on blank spaces in the document. Constructing a search key to check a spaced word (type A target string); Extracting words (type B target strings) pasted by substring search using the constructed search key; And displaying the spacing word and the extracted paste word corresponding to the search key as an inconsistent spacing word, wherein the extracting includes: a vocabulary for extracting a paste word that completely matches the search key; By morphologically analyzing the results retrieved through the matching model, the same syllable candidate matching model which extracts the pasted words by processing the same syllable candidate for the last syllable of the search key, or the same syllable candidate matching model. A program for executing one of the morpheme matching models for extracting the same pasted phrases is recorded.

The present invention has the effect of effectively checking the spacing using syllable information and morpheme information.

In addition, by performing a lexical-level spacing consistency check, the user can check the areas where spacing has been performed inconsistently and display them, so that the user (the author) can easily identify the parts that may be inconsistent spacing. There is an effect that the sentence can be easily modified by the corresponding spacing so that the meaning to be expressed can be transmitted.

In addition, by searching for inconsistent spacing candidates first by a simple string comparison method, and performing morphological analysis only on the retrieved candidates, there is an effect of searching for a portion where spacing has been performed more accurately.

In addition, by using only vocabulary and morpheme information, it is possible to select only the parts with high possibility to check the spacing, so that the spacing test can be flexibly performed in response to the inspection request of the user.

1 is an exemplary view for explaining a spacing consistency checking method according to the present invention.
2 is an exemplary view showing an example of a post file according to an embodiment of the present invention.
3A and 3B are flowcharts illustrating a spacing consistency checking method according to an embodiment of the present invention.
4 is an exemplary view showing a screen displayed by the spacing consistency check method according to an embodiment of the present invention.

According to an embodiment of the present invention, a spacing check method according to an embodiment of the present invention first proposes an inconsistent spacing candidate by a simple string comparison method, and then proposes a technical configuration of an efficient method of morphological analysis of only these candidates.

At this time, the present invention proposes a lexical matching model, a syllable candidate matching model, and a morpheme matching model. Here, the lexical matching model means how to find the target string that matches the search key. In addition, the syllable candidate matching model between the same words means a method of finding a target string by treating the syllable candidate between the same words with respect to the last syllable of the search key. In addition, the morpheme matching model means a method of morphological analysis of the results found by the syllable candidate matching model between identical words to find a target string having the same basic morpheme.

In the following description, terms of a lexical matching model, a syllable candidate between identical words, and a morpheme matching model will be used.

The spacing coherence checking method of the present invention is a technology built in a computer or a portable terminal and searches for a portion in which a spacing is performed inconsistently under a general control of a microprocessor (main controller) of the device configuration and displays it on the screen. Make a point. At this time, each process (step) of checking a space is performed by a microprocessor.

The present invention also proposes a computer-readable recording medium having recorded thereon a program for realizing the spacing consistency checking method.

In the following description, specific details of the spacing test method of the present invention are presented to provide a more general understanding of the present invention. It is to be understood that the present invention may be readily implemented without these specific details and by modification thereof. It will be apparent to those of ordinary skill in the field.

Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the accompanying drawings, with reference to the parts necessary for understanding the operation and operation according to the present invention.

First, the spacing consistency checking method according to the present invention proposes a three-step processing level of 1. vocabulary level, 2. type level, and 3. semantic level, and focuses on the practically usable vocabulary level.

1. Consistency check at lexical level

Consistency checking at the lexical level means checking that words with the same basic vocabulary are uniformly used in a single space in the document. Here, the same basic vocabulary means that the nouns and verbs have the same vocabulary except for ending changes and investigations.

The following is the syntax of [Example 1] to help understand the automatic spacing method according to the present invention.

[Example 1]

(1) computer science

(2) in computer science

(3) information science

(4) going out

(5) turned off

(6) living

In [Example 1], (1) and (2) have the same first word as "computer" and the same word "science". Therefore, the word "science" and the word before it should be used consistently.

However, in (2) and (3) of [Example 1], since the first word is different, the word "science" and the words before it are not necessarily used consistently.

Also, in case of (4) and (5) of [Example 1], it is changed from 'to go off', and the basic vocabulary is the same, and there is no spacing consistency between the main and auxiliary verbs. Here, the correctness and wrongness of spacing are not determined, only the consistency of spacing is checked.

In other words, (4) in [Example 1] puts an auxiliary verb of 'going' after the main verb of 'get off', and (5) of [Example 1] adds an auxiliary verb of 'liver' after the main verb of 'off' In the case of pasting without a space, even if the basic vocabulary is the same, there is a lack of consistency, so that the space is consistent and unified.

2. Type-level consistency check

Type-level spacing consistency checking means checking that words made with the same type of vocabulary are used in a unified spacing form in a document.

For example, the words "computer" and "information" used in (2) and (3) of [Example 1] are compound words in which vocabularies representing a kind of science are used together with "science."

In this case, to be consistent at the type level, the spacing must match. However, in the case of such compound words, if the word is a compound word according to the spacing rules, the word is pasted.

In other words, if 'computer science' is a compound word, paste it. If 'information science' is not a compound word, use it.

In addition, as in (4) and (6) of the [Example 1], the two words are used differently in the case of the same auxiliary verb. Therefore, there is a problem in the spacing consistency of the same type, but it must be handled differently depending on whether each vocabulary is a compound word.

Since the spacing of compound words differs from case to case in each case, the criteria are usually determined by whether the standard word is listed as a compound word in the standard Korean dictionary. However, the list of compound words listed in the Korean Standard Dictionary is constantly changing, and in some cases, opinions on whether or not it is a compound word from the point of view of Korean linguistics or linguistics are still difficult to apply to the type-level spacing consistency test.

3. Consistency Level Consistency Check

The semantic level spacing consistency check means to check the contextual meaning of the spacing.

In other words, if the author intentionally puts or pastes to change the meaning differently, in the case of compound words in grasping and processing them, two words gather together to form a word with a new meaning, so it is distinguished from the case where it is written.

For example, ‘great father’ means dad's brother, but ‘great father’ means dad who is tall or tall.

Therefore, in order to identify cases where they are intentionally used to clarify their meanings and those which do not, they must maintain the spacing consistency for the same meaning. In these cases, you need to understand the context before and after. However, even in this case, it is difficult to handle the current natural language processing technology because it is not a level to grasp the entire context.

Accordingly, the spacing coherence check according to the present invention will be described in detail with respect to a method of checking a portion in which spacing is performed inconsistently so that spacing can be performed consistently using a lexical level.

4. Vocabulary Level Spacing Consistency Checking Model

4.1. Navigation keys  Candidate group extraction

In order to find a case where a compound word is inconsistently spaced as in (1) and (2) of [Example 1], the noun part must first be separated, and the cases where the nouns are pasted and spaced are found.

To do this, we can morphologically analyze every word and find each case based on these morphemes.

However, in this case, it is inefficient because morphological analysis is included including all the words which have no possibility of inconsistent spacing.

The present invention proposes an efficient method of finding inconsistent spacing candidates first by a simple string comparison method and performing morphological analysis on only those candidates for efficient searching. This method basically extracts a part of left and right words around a blank and uses it as a search key.

Candidates found with the search key should contain all inconsistent words. To do this, there are several navigation keys. In the present invention, the search key is preferably set as the first letter of the word before the blank and the word after the blank.

In other words, as shown in (1) to (6) in [Example 1], when a compound word or auxiliary word that exhibits inconsistency in spacing is used, the first word is usually unchanged while the first word is usually the same and thereafter. The reason is that it may change due to affinity or irradiation.

Of course, since one letter of the back word can be changed as in the example of (5), three models are proposed in the next section.

When the searched string is called a target string, and the spaced string is a type A target string and the pasted string is a type B target string, the search keys for (4) and (5) in [Example 1] are shown in FIG. Is defined as:

That is, it is defined as 'off' as a type A target string, 'off' as a search key, and 'off' as a type B target string.

Search keys are basically extracted from spaced words, so you build all kinds of search keys that are possible from spaced words (type A target strings) in the first pass of your document. These search keys are used to extract words (type B target strings) pasted by substring search on the second pass. If the pasted word is not found, the word found by the search key is removed from the object of inconsistent spacing. By this method it can be obtained in the form of a post file as shown in FIG.

4.2. Proposed model

The model is divided into three types according to the search method and the post-processing method using search keys. The first is a lexical matching model, abbreviated as T1. This model is a simple model that finds only substrings that match the search key exactly, and is used for comparison with other models.

The second is the 'Same Syllable Candidate Matching Model', abbreviated as Tc and is a model created to complement the T1 model. In the case of supplemental verbs, unlike in the case of nouns, the syllables in the beginning of a word may change as the ending changes. This is not the case with the T1 model, which allows only exact syllable matching. As in (4) and (5), it should be compared with the search string in consideration of the usage of auxiliary words. In other words, the sub-language stem liver should be identified as a form of 'ga' and should be treated as 'going away'.

According to the National Institute of Korean Language, "Korean Language Learning Materials," the National Institute of Korean Language's website, http://www.korean.go.kr, there are 33 kinds of auxiliary words.

go Hold Want to have Seem No have look like not be there be worth come go out roll have It's me eat give Pay Can not do it die put Abandon Pretend Sweet see pretend put Obvious clean up give want Do

Although the stems of these supplementary words are changed by the mother's change, the number of syllables is limited.

In the present invention, the case of changing the first syllable of the auxiliary words by the change of the ending of these words was examined and summarized as shown in Table 2 below.

Auxiliary words Syllable Candidates Go, have, have Go, Live, Go, Persimmon, Gap, Gone It's me Me, I, Me, M, Lead, Woke Pay Mine, embellish, snatch, sniff, synapse, snuggle Sweet Moon, multi, sweet, fence, answer put Two, dull, two, dum, dum, put Hold Guys, whatever, hard, petting, knot roll Say, only, do, thank you see Boy, seen, seen, spring, bob, saw come Oh, come, come, ohm, opt, come give Share, gave, give, zoom, picked up Do Ha, han, do, ha, sum, did

As shown in [Table 2], the syllables that can appear in the same stem but the utilization type are defined as 'same syllable candidates'. Also, if the last syllable of the search key and the syllable compared in the target string belong to the same syllable candidate, the string is treated as a match.

The third model is the 'morphological match model', abbreviated Tm. This model morphologically analyzes the candidate words found in the Tc model and finds inconsistent spacing words based on them.

For example, all of the cases (1) and (2) below [Example 2] are extracted as inconsistency candidates. Therefore, "parts" must be correctly analyzed and the noun part "parts" must be separated and treated differently from "from" the investigation described in (2) of [Example 2]. This requires morphological analysis.

[Example 2]

(1) Memory parts

(2) From memory

In addition, in the case of the Tc model that is searched in consideration of the utilization form, an incorrect result may be output by treating 'liver' and 'ga' shown in (1) and (2) of [Example 3] below as syllable candidates of the same language. have. In order to solve this problem, it is necessary to morphologically analyze the preceding word and the next word to distinguish the auxiliary verb from other parts of speech.

Example 3

(1) between registers

(2) registers

The results of morphological analysis on the two words to be compared are compared at the parts of speech and lexical level.

In other words, if the morpheme is a part-of-speech such as a noun or a verb, the vocabulary should be the same. If the other morphemes are the same, the two words are determined to be the same. Moreover, when the result of morphological analysis comes out with various candidates, even if it matches only one candidate, the semantic is discriminated with the same word.

For example, (1) and (2) in [Example 4] below are stemming the 'above the bus' and 'above the bus' to check the spacing consistency.

Example 4

Stemming by using both cases shows that two grammatical elements are separated.

Both of the first elements are analyzed as nouns (N) on the bus and have the same vocabulary. The second element is different from the vocabulary but is examined (j).

Therefore, the word is expressed as a word with inconsistent spacing.

Also, in the case of (3) and (4), the ending verb is changed to 'see' and 'to see', but the verb stem (V) of the main verb and the auxiliary verb coincides with the ending (e). Regardless of the vocabulary changes, the same type of word is used to indicate that the spacing is inconsistent.

(5) and (6) are spacing consistency checks for two words, "for memory" and "even for memory." In this case, since the nouns have different vocabularies, they are discriminated by different kinds of words and excluded from the comparison of spacing consistency.

The above three models are summarized as follows.

The lexical matching model (T1) means finding a target string that perfectly matches the search key, and the syllable candidate matching model (Tc) treats the same syllable candidate for the last syllable of the search key as the target string. The morpheme matching model (Tm) means morphological analysis of the results found by the model Tc to find a target string having the same basic morpheme.

3A and 3B are flowcharts illustrating a spacing test method according to an exemplary embodiment of the present invention. Here, FIGS. 3A and 3B illustrate a spacing test method for checking a portion of spaces inconsistently spaced in a document.

Referring to FIGS. 3A and 3B, first, a search key is constructed to extract a predetermined portion of left and right words centered on a blank space (blank) in a document to identify a spaced word (type A target string) (S310).

By using the constructed search key, the words (type B target strings) pasted by the substring search are extracted (S320).

In response to the search key, the spacing word and the extracted paste word are displayed as inconsistent spacing words (S330).

The model (method) of extracting an object to which consistency is compared by the above-described step S320 is as follows, and any one of steps 321 (S321) to 325 (S325) described below is performed.

That is, when executing the lexical matching model (T1), the pasted words that completely match the search key is extracted (S321).

However, when executing the syllable candidate matching model Tc between the same words, the extracted syllables are extracted by processing the same syllable candidates as the same for the last syllable of the search key (S323). Here, the syllable candidate matching model between the same words is a model to complement the lexical matching model.

However, when the morpheme matching model (Tm) is executed, the results found through the syllable candidate matching model between the same words are morphologically analyzed to extract words with the same basic morphemes (S325). Here, the morpheme matching model is a model that can extract the most accurate result.

5. Experiment and Evaluation

5.1. Experiment

Some of the corpus constructed by the National Institute of Korean Language for the experiment were selected by field, and five randomly selected papers among recent papers published in the Journal of the Society were converted into text files to create a test document set as shown in Table 3 below. For the convenience of experiments, the size of books, novels and papers were selected as short stories or short stories around 5,000 words.

If possible, they were selected as books or novels from various fields (records, education, humanities, science, art, etc.) and various authors. Writing is a relatively short writing by college students, and news articles have also gathered short articles from newspapers, totaling about 25,000 words.

Word count Document Count Average word count per document book 26804 5 5361 writing 24802 39 636 News articles 24836 100 248 novel 25915 5 5183 Paper 25888 5 5178 synthesis 128245 154 833 (average)

The experiment implemented and evaluated the three models described in the previous chapter. The lexical matching model (Tm) is a publicly available morpheme analyzer, the results of performing this lexical matching model is shown in FIG. In FIG. 4, '/' means line break display.

As shown in FIG. 4, the left and right context information and the file name are displayed based on the inconsistency of the spacing so that the user can make a final judgment. Each line is defined as 'inconsistent spacing' (hereinafter referred to as 'case'), and cases requiring the same spacing are separated by dotted lines. It is defined as' (hereinafter referred to as' group ').

4 shows an example of inconsistent spacing between 'business' and 'business', 'quality requirements' and 'quality requirements'. To make this program even more useful, you can make it easier to edit by linking hyperlinks to the text locations that actually appear in each case.

5.2. evaluation

For evaluation purposes, a set of answer cases were manually created for a set of test documents. The syllable candidate congruence model (Tc) between the same words was used when preparing the correct case set.

That is, the theoretical syllable candidate matching model can theoretically extract all possible inconsistent spacing cases. Therefore, the correct among the inconsistent spacing found in the same syllable candidate matching model is manually selected and grouped by group. I created a set.

Table 4 below summarizes the number of inconsistent spacing errors found by the field.

Average word count per document Error cases Error cases per document Word Count per Error Case Error group count book 5361 497 99 54 109 writing 636 140 4 177 44 News articles 248 350 4 71 125 novel 5183 655 131 40 181 Paper 5178 881 176 20 107 Average 833 ⁺ 505 83 74 113

The average number of spacing errors in all areas of the test document averaged 1 per 74 words.

Generally speaking, inconsistent spacing is likely to occur in long documents.

As shown in [Table 4], a lot of inconsistent spacing appeared in books, novels and papers with a long actual document length. In papers, one error per 29 phrases was found, showing the highest error rate among the test areas. Exceptionally, news articles were relatively inconsistent in spacing even in short articles.

Model evaluation was largely divided into case and group results.

As described above, instances represent each substring including inconsistent words, and groups represent sets of cases with conflicting spacing. The evaluation of the two was again calculated by dividing the accuracy, recall and F value for each. For the accuracy and reproducibility of the case using the following [Equation 1], [Equation 2], the accuracy and reproducibility for the group using the following [Equation 3], [Equation 4]. The F value can be calculated differently according to the evaluation ratio of the accuracy rate and the reproducibility rate. Equation 5 can be used to calculate the case F value and the group F value, respectively.

In [Equation 3] and [Equation 4] of the group evaluation equation, 'the number of groups that match the correct answer in the output' is included in the count when all cases appearing in the correct answer group are included in the output group.

For example, suppose there are groups A1, A2, B, C, and D, in which cases a, b, c, d, and e are elements as shown in Example 5 below.

When the sets A1 and A2 are the correct answer groups, B is the group that matches the correct answer, and C and D are the groups that do not match. In case B, a mismatched case “f” is included, but it can be reviewed to correct any inconsistency.

However, C may be missing a case 'a', which may prevent consistent spacing correction. This is defined as an ungroup error.

In addition, D should be divided into two groups, A1 and A2. If the number of cases is not divided, or the number of mixed groups is large, it may be difficult to review by hand, which is treated as an error. This case is defined as an overgroup error.

Example 5

A1 = {a, b, c}

A2 = {d, e}

B = {a, b, c, f}

C = {b, c}

D = {a, b, c, d, e}

The spacing consistency was checked only for consistency in one document, and the evaluation results of each experimental model are shown in Table 5 below.

case group Recall Accuracy F value Recall Accuracy F value T1 95.8 70.9 81.2 92.5 68.5 78.5 Tc 100.0 67.4 80.0 99.6 67.5 80.0 Tm 93.2 93.2 93.2 86.0 86.0 86.0

The results of the evaluation of the case showed that the lexical matching model (T1) had a lower reproducibility than the identical syllable candidate model (Tc), but the F value was slightly higher than that of the same syllable candidate model (Tc). The morphological match model (Tm) has a 93.2% recall, which is relatively lower than the syllable candidate model, but its accuracy is 93.2%, which is 93.2%, which is more significant than other models with 81.2% and 80.0%. Would be excellent.

In other words, it can be seen that the morpheme matching model using morpheme information is much better than the lexical matching model using only syllable information and the syllable candidate matching model between identical words.

The evaluation results of the group were generally similar to the case evaluation results, but unlike the case evaluation, the F value of the same syllable candidate model was higher than that of the lexical agreement model. This is because the group repetition rate is relatively low because the lexical matching model often divides 'inconsistent spacing cases' belonging to the same group into different groups. In addition, the reason why the group reproducibility of the same syllable candidate model is not 100% is that the overgroup error is included. This means that you can actually find all groups by hand.

Each model can be selected and used according to its needs.

That is, since the syllable candidate matching model between the same words has a high reproducibility, it can be used even if the user's time is required to check the spacing coherence error in all cases. On the other hand, the Tm model has a relatively high accuracy and can be used to correct inconsistent spacing with little effort.

In addition, the syllable candidate matching model between the same words is a model that reduces the number of morphological analysis targets as a previous step of the morpheme matching model. The total number of experimental words in the experimental data was 128,245, and the number of cases of inconsistent candidates suggested by the syllable candidate model was the same as 3,792 cases. It could be processed more efficiently.

5.3. Error analysis

After analyzing some of the data types used in the experiment and analyzing the error type of the Tm model, 80% of the errors (missing errors) that could not be found and 20% of the wrong errors (false extraction errors) were found. It was about.

Missing errors are about 50% of missing errors extracted by colloquial, which is a non-standard language used in novels, etc., and others are caused by errors in the stemming itself.

False extraction errors were mainly caused by pasting into a different meaning. In the current morpheme matching model, two words are added with a spaced word to check whether they are identical, and then morphologically analyzed and compared. For example, in 'Look at my head', 'na' and '이' can be analyzed as pronouns and tubular pronouns, respectively. Therefore, it was extracted incorrectly by matching the word that contained the noun “naiga”.

In general, inconsistent spacing occurs in compound words and auxiliary words that allow multiple spacing rules. However, if you look at the inconsistency in the actual text, if the spacing rule is incorrect or noun is followed by 'have' or 'become' and becomes a part-prescriptive verb with a verb, then 'then', ' On this day, the case may be added as a quasi-synthetic word or may be used separately.

While the present invention has been described in connection with what is presently considered to be the most practical and preferred embodiment, it is to be understood that the invention is not limited to the disclosed embodiments, but is capable of various modifications within the scope of the invention. Therefore, the scope of the present invention should not be limited by the illustrated embodiments, but should be determined by the scope of the appended claims and equivalents thereof.

Claims (8)

In the spacing method of checking for inconsistent spacing in a document,
Constructing a search key so that a predetermined portion of left and right words is extracted based on a blank space in the document to identify a spaced word (type A target string);
Extracting words (type B target strings) pasted by substring search using the constructed search key; And
Including a spacing word and the extracted pasted word corresponding to the search key as an inconsistent spacing word,
The extracting may include: a lexical coincidence model extracting pasted words that completely match the search key, and extracting pasted words by processing the same syllable candidates for the last syllable of the search key. Spacing consistency check method characterized in that for performing the morphological analysis of the result retrieved through the candidate matching model or the syllable candidate matching model between the same words to extract the pasted words with the same basic morpheme.
The method of claim 1, wherein the predetermined portion of the left and right words,
Spacing consistency check method characterized in that the first letter of the left word and the right word.
The method of claim 1, further comprising removing the words found by the search key from the subject of the inconsistent spacing if the pasted word is not extracted.
The method of claim 1, wherein the same syllable candidate model
If the last syllable of the search key and the syllable to be compared in the target string belong to the same syllable candidate (syllable but syllables that can appear as utilization), the spacing consistency check method is treated as a match.
The method of claim 4, wherein the same syllable candidate
Spacing consistency check method, characterized in that the same stem as shown in Table 6 below, but may appear as a conjugation type.
Auxiliary words Syllable Candidates Go, have, have Go, Live, Go, Persimmon, Gap, Gone It's me Me, I, Me, M, Lead, Woke Pay Mine, embellish, snatch, sniff, synapse, snuggle Sweet Moon, multi, sweet, fence, answer put Two, dull, two, dum, dum, put Hold Guys, whatever, hard, petting, knot roll Say, only, do, thank you see Boy, seen, seen, spring, bob, saw come Oh, come, come, ohm, opt, come give Share, gave, give, zoom, picked up Do Ha, han, do, ha, sum, did
The method of claim 1, wherein the morpheme matching model,
In accordance with the results of morphological analysis of the written and pasted words, the parts of speech and the lexical level are compared at the same time, and if the morphemes are the same parts of nouns or verbs, the words must be the same. and,
In the case of morphemes other than the nouns and verbs according to the morphological analysis, if the parts of speech are the same, two words are determined to be the same morpheme.
And if the morphological analysis results in a variety of candidates, even if one of the candidates matches, the semantic is distinguished by the same word.
The method of claim 1, further comprising: providing a hyperlink to edit a text link corresponding to a specific word marked as the inconsistent spacing word.
On computers that check for inconsistent spaces in documents,
Constructing a search key so that a predetermined portion of left and right words is extracted based on a blank space in the document to identify a spaced word (type A target string);
Extracting words (type B target strings) pasted by substring search using the constructed search key; And
Including a spacing word and the extracted pasted word corresponding to the search key as an inconsistent spacing word,
The extracting may include: a lexical coincidence model extracting pasted words that completely match the search key, and extracting pasted words by processing the same syllable candidates for the last syllable of the search key. A computer-readable program that records a program for executing one of the morphological matching models extracting pasted words having the same basic morpheme by morphologically analyzing the results searched through the candidate matching model or the syllable candidate matching model between the same words. Record carrier.

Images