KR100897718B1 - Device and method for correcting errors of colloquial type sentence - Google Patents

Abstract

An error correction apparatus and method for colloquial sentences which can efficiently correct various errors in spoken sentences appearing in a communication environment are provided. The error correcting apparatus extracts colloquial character strings from the input data to generate input data according to a user's operation, removes repeated phrases from the colloquial character strings, combines disassembled syllables, and corrects strings based on syllables. A colloquial proofing unit for performing colloquial correction by selectively applying the process, and a correction data return unit for returning corrected input data by reflecting the corrected colloquial string on the input data. The error correction method includes generating input data according to a user's operation, extracting colloquial strings included in the input data, removing repeated phrases from the extracted colloquial strings, combining disassembled syllables, and syllables as basic correction units. Selectively applying a process such as string correction to perform colloquial correction, and including the corrected colloquial string in the input data to generate the corrected input data.

Network, spelling, colloquialism

Description

DEVICE AND METHOD FOR CORRECTING ERRORS OF COLLOQUIAL TYPE SENTENCE}

The present invention relates to language processing in a communication network, and more particularly, to an apparatus and method for correcting errors in spoken sentences in a communication network.

Conventional natural language processing techniques, such as morphological analysis and syntax analysis, use sentences that are grammatically correct and have no non-verbal elements such as emoticons. For reference, emoticons are various facial expressions made by combining symbols within a range that can be input through an input means such as a keypad or a mouse.

However, there are some spelling or grammatical errors in the language used by the actual user, and many sentences include misspelling and spacing errors. These errors are a major cause of degrading the performance of natural language processing techniques and applications using them.

In order to solve this problem, various error correction techniques such as spell correction and spacing correction have been proposed.

However, the conventional error correction technique targets written sentences such as newspaper articles, novels, reports, and essays. Errors in these sentences are usually unintentional errors caused by user error or incorrect information, lack of grammatical knowledge, and the like. For example, marking "I can" as "I can" or "San Francisco" as "San Francisco" can be attributed to a simple mistake or lack of knowledge of the correct notation.

However, in the colloquial sentence used in a communication environment such as the Internet or a mobile communication network, there are various types of error elements different from the general written sentence. That is, sentences used in a communication environment contain error types that are very different from those targeted by conventional error correction techniques.

Therefore, colloquial sentences used in a communication environment are difficult to correct by applying a conventional error correction technique. The reason for this is described in more detail as follows.

First, in colloquial sentences used in a communication environment such as the Internet or a mobile communication network, a user intentionally causes an error for emphasis, visual effects, and fun.

For example, suppose a user uses the sentence "I'm sick of -.-". This sentence is used to convey the meaning of "I'm annoyed," and the user separates consonants and vowels, gives a large visual effect, and emoticons (-. -) Was also used together.

As described above, the colloquial sentence of the communication environment includes not only an error not intended by the user but also an error input intentionally.

This is the result of reflecting the characteristic that the main users who use colloquial sentences in the communication environment are teenagers in their 20s and 20s who prefer fashion-sensitive, personal, and unique things.

Second, errors intentionally entered by the user are different from unintentional errors such as spacing and spelling errors.

For example, an error intended by a user usually appears in several letters, and the letters used differ depending on the type of error. In the case of an emoticon, a symbol is mainly used. In the case of the aforementioned "ㅈㅈ", a consonant, a consonant, and a vowel are used.

In addition, in situations where an error occurs, that is, when an error occurs when entering a word or sentence, some tendency (e.g., an arrangement of keyboards) can be found in a conventional unintended spelling / offset error. The only difference is that the user intended error can be any case.

Third, since colloquial sentences in a communication environment have various types of errors and many types of errors appear together in one sentence, it is difficult to correct only one error. Therefore, several errors must be corrected together in one process, module.

Accordingly, an aspect of the present invention provides an error correction apparatus for efficiently recognizing and correcting various errors in spoken sentences appearing in a communication environment, thereby improving performance of colloquial analysis or natural language processing technology and improving user satisfaction. To provide a way.

Technical problems to be achieved by the present invention are not limited to the above-mentioned technical problems, and other technical problems not mentioned above may be clearly understood by those skilled in the art from the following description. There will be.

According to an aspect of the present invention, there is provided an apparatus for correcting a spoken sentence, comprising: an input unit generating input data according to a user operation; A colloquial text extracting colloquial character strings from the input data and selectively performing one or more processes of removing a phrase, combining disassembled syllables, and correcting a syllable based on syllables as a basic correction unit. Correction unit; And a correction data return unit which returns the corrected input data by applying the corrected colloquial character string to the input data.

On the other hand, the error correction method of spoken sentences according to the present invention, generating the input data according to the user operation; Extracting a colloquial character string included in the input data; Generating a corrected colloquial string by selectively performing one or more processes of removing a phrase, combining disassembled syllables, and correcting a syllable based on syllables based on the colloquial string; And generating the corrected input data by including the corrected colloquial character string in the input data.

The error correction apparatus and method according to the present invention made as described above can efficiently recognize and correct various errors of spoken sentences appearing in a communication environment.

In addition, the error correction apparatus and method according to the present invention can improve the performance of colloquial analysis technology and increase user satisfaction.

In addition, if the error correction apparatus and method according to the present invention is applied to the pre-processing process of spoken sentences, and combined with the written language-based natural language processing technology, even if the written language and error type is different and the colloquial language is difficult to correct, Maintain overall performance.

Hereinafter, an error correction apparatus according to a preferred embodiment of the present invention will be described in detail with reference to FIGS. 1 and 2.

1 is a block diagram showing an error correction device of spoken sentences according to an embodiment of the present invention, for convenience of description illustrating an error correction device in the form of a mobile phone. In addition to a mobile phone, as an error correction device, various types of information processing devices that can be used in a communication environment, such as a computer, a personal digital assistant, a notebook, and a server, may be applied.

Referring to FIG. 1, an apparatus for correcting a spoken sentence according to an embodiment of the present invention includes a control unit 100, an input unit 110, a display unit 120, a data storage unit 131, and a mode / state storage unit 132. ), A signal processor 140, a colloquial correction unit 150, a correction data return unit 160, and a post-process correction unit 170.

The controller 100 controls the overall operation of the mobile phone based on the state flag stored in the mode / state storage 132. The natural language processing function is performed by driving the colloquial correction unit 150, the correction data return unit 160, and the post-processing correction unit 170 to correct an error of the input data input by the user. Is displayed on the display unit 120.

The input unit 110 generates input data corresponding to the original message input by the user, and a keypad, a touch pen, a mouse, and the like may be used as the input means. The keypad 110 has a numeric key for inputting a number such as a telephone number, a character key for inputting a character, a function key for performing a specific function, and a direction key. When the user presses a specific key, the input unit 110 generates input data corresponding to the pressed key and transmits it to the control unit 100.

The display unit 120 displays a state of use of the battery, a reception strength of radio waves, a date and time, an operation state of the mobile phone, etc. in the standby state of the mobile phone. Then, when the natural language processing function is performed, the original message and a correction message correcting the error of the original message are visually displayed on the display unit 120 in the message editing window.

The data storage unit 131 acts as a data buffer when the mobile phone operates, and temporarily stores data input by the input unit 110 or receives a text or an image received from the external mobile phone. Save the data.

The mode / state storage unit 132 stores the current operation mode of the mobile phone selected by the input unit 110 as a status flag (0, 1, 2, ...). The controller 100 updates the mode / state storage unit 132 by assigning a unique state flag for each mode to distinguish whether the cellular phone is in a reception mode, an outgoing mode, a storage mode, a call mode, or the like.

The signal processor 140 is a processor that codes or decodes a voice signal and performs an equalizer function and a sound data processing function for removing multipath noise.

The colloquial correction unit 150 determines whether or not a colloquial character string is included in the input data generated according to a user operation, and extracts a colloquial character string to be corrected from the input data. The colloquial correction is performed on the extracted colloquial string to generate a corrected colloquial string.

The calibration data return unit 160 receives the corrected colloquial character string from the colloquial collation unit 150, and reflects the collated spoken character string to the input data to return the corrected input data.

The error correction apparatus may further include a post-processing corrector 170 for performing a subsequent process of natural language processing through the colloquial body correcting unit 150. The post-processing correction unit 170 performs subsequent written word correction on the returned input data.

Here, the post-processing corrector 170 performs a language processing function on a written word, that is, a written word based sentence (eg, a sentence used in a report, a newspaper article, etc.) that does not include an intentional error of a user. Is a natural language processing engine.

The colloquial correction unit 150 analyzes various errors shown in spoken sentences in a communication environment, classifies the types, and devises a correction method suitable for each type to correct the corresponding errors. This converts it to sentence expressions that can be used directly in written language-based natural language processing engines such as morphological analysis and syntax analysis. That is, not only basic correction such as spacing and spelling errors, but also repeated phrase removal, decomposed phoneme combination, emoticon recognition, slang recognition and correction are performed on the colloquial sentence input by the user.

Therefore, if a technique for correcting a spoken error is applied to a preprocessing process of a spoken sentence, and using the corrected string output as a result of the corrected spoken error as an input to the post-processing corrector 170, a written language-based natural language processing engine is used. It can be applied to colloquial language without degrading performance.

As the post-processing corrector 170, various natural language processing applications such as a morpheme analyzer, a syntax analyzer, a dialogue engine, and a machine translation engine may be used.

FIG. 2 is a detailed configuration diagram of some components shown in FIG. 1, and illustrates a detailed configuration of the colloquial correction unit 150.

The colloquial error is explained again as follows.

The colloquial sentences used in the communication environment often contain artificial errors, unlike general documents or sentences used in real life, and the types thereof are various as well as conventional typos or spacing errors.

For example, phrases such as "AAA ANOYA", "I'm sick," "pretty," include colloquial errors, and these errors are used to emphasize the user's feelings or use them for fun. This is a rare error in written sentences.

In addition, as the content of the sentence itself is often simple and light, the user is relatively insensitive to spelling or grammatical errors. Therefore, even if a sentence is incorrectly input, the user does not necessarily correct it or inputs a phonetic sound instead of the correct word. In the case of the expression 'pretty', 'pretty' is a correct expression, but it is often used as a phonetic input.

The colloquial correction unit 150 analyzes various error phenomena frequently occurring in a spoken language, and classifies them into three types such as repetitive phrase removal, disassembled syllable combination, and string correction, and performs appropriate correction for each.

When the string entered by the user is received as input data, various error corrections are performed and the corrected string is returned as a result.

Referring to FIG. 2, the colloquial body correcting unit 150 includes a repeating phrase removing unit 151, a disassembled syllable combining unit 152, and a string correcting unit 153.

The repetitive phrase removing unit 151 extracts a repetitive phrase that is an unnecessary part of the spoken text string, and removes the extracted repetitive phrase to make the sentence clear and converts the sentence into a sentence suitable for language processing.

The disassembled syllable combination unit 152 extracts disassembled syllables from colloquial strings, and combines consonants and vowels of disassembled syllables to restore correct syllables.

The string corrector 153 learns and stores correction rules including syllables based on syllables based on the test strings including colloquial errors and the correction strings correcting the errors of the test strings and based on the stored correction rules. Correct errors in colloquial strings.

As shown in FIG. 2, the character string corrector 153 includes a verification data dictionary 153_1, a learning performer 153_2, a correction rule dictionary 153_3, and a correction performer 153_4.

The verification data dictionary 153_1 stores demonstration strings and correction strings correcting errors of the demonstration strings. Here, it is good to register a large number of demonstration strings and calibration strings to increase the reliability of the calibration rule. In addition, since errors in spoken sentences have many different types from generalized natural language processing techniques, it is recommended that designers work manually to improve the efficiency of error proofing of test strings.

The learning performer 153_2 learns calibration rules for the test strings and the correction strings for correcting the errors. The correction rule is a correction rule in syllable string units. The learning performing unit 153_2 compares the test string with the calibration string corrected for the error of the test string, extracts different parts, corrects the error part of the test string for the syllable string, and corrects the error correction part for the calibration string. After considering it as, a plurality of preliminary correction rules are generated by including 1 to n left / right syllables, that is, left syllable context and right syllable context, to the left and right of the syllable string. Subsequently, the preliminary correction rule is selected by measuring the performance of the preliminary correction rules and using a small context, and then the correction rule dictionary with the selected syllable sequence and the left / right syllable context is included in the selected correction rule (153_3). Register at The process of learning the calibration rule is described in more detail in FIG. 6.

The calibration rule dictionary 153_3 stores the calibration rules learned through the learning performer 153_2.

The calibration performing unit 153_4 searches for a calibration rule of a type corresponding to the colloquial string among the calibration rules registered in the calibration rule dictionary 153_3 (for example, whether there is a target syllable string that matches some or all of the colloquial string). Error correction is performed by applying the found correction rule to a given colloquial string (for example, replacing part or all of the colloquial string with a target syllable string and a left / right syllable context registered as a correction rule).

In this way, the colloquial correction unit 150 defines three types of errors frequently occurring in a communication environment, and effectively corrects errors appearing in the spoken language of the communication environment through a correction algorithm suitable for each error type, and processes the language. The performance of the technology can be improved to improve user satisfaction.

For repetitive phrase elimination and decomposing syllable combinations, a heuristic algorithm suitable for each error may be applied. Heuristic algorithms are empirical methodologies that draw conclusions based on the subjective judgments of the designer (decision maker).

In addition, rule-based learning algorithms can be used for string corrections, including spelling correction, spacing error correction, emoticon recognition and correction, slang recognition and correction. Error correction is made.

The colloquial correction unit 150 may selectively apply repetitive phrase removal, disassembly syllable combinations, and string correction processes to arbitrary colloquial strings, or may be applied at the same time. Therefore, in the colloquial body correcting unit 150, the repeated phrase removing unit 151, the disassembled syllable combination unit 152, and the string correcting unit 153 may be selectively constructed or all together.

On the other hand, the colloquial correction unit 150 determines in advance the correction method (for example, repeated phrase removal, disassembly syllable combination, string correction) to be applied to the extracted colloquial string, and performs colloquial correction according to the determined correction scheme. You may. In the case of determining the correction method in advance, the colloquial correction unit 150 checks whether or not a repeated phrase exists using the extracted colloquial string, checks whether or not the dismantled phoneme exists, or is stored in the verification data dictionary. It checks whether there is a corresponding part of the character string, and determines whether to apply the calibration method according to the inspection result.

When there is a repetitive phrase in the colloquial string, the repetitive phrase removing process is performed, and when there is a dismantled phoneme, a disassembly syllable combining process is performed. Similarly, when there is a part corresponding to the string stored in the verification data dictionary 153_1 in the colloquial string, the string correction process is performed.

Hereinafter, an error correction method according to a preferred embodiment of the present invention will be described in detail with reference to FIGS. 3 to 7. For convenience, the error correction method of FIGS. 3 to 7 will be described assuming the error correction apparatus of FIGS. 1 and 2.

3 is a flowchart illustrating an error correction method of a spoken sentence according to an embodiment of the present invention.

First, in step S100, when a user inputs an original message using the input unit 110, input data is generated correspondingly, and the control unit 100 transmits the generated input data to the colloquial correction unit 150.

In operation S110, the colloquial correction unit 150 determines whether oral collation is to be performed by checking whether or not the colloquial string to be corrected is included in the received input data. Proceed to

In operation S120, the colloquial correction unit 150 extracts a colloquial character string from the input data for error correction.

In operation S130, the colloquial correction unit 150 determines a calibration method according to the extracted colloquial character string, and performs at least one or more of steps S140 to S160 according to the determination result.

At this time, the colloquial correction unit 150 examines the extracted colloquial string and determines in advance the correction method (for example, repeated phrase removal, disassembly syllable combination, string correction) to be applied to the string, and then determines the corrected correction method. So that colloquial corrections can be performed. In order to determine the correction method, the colloquial correction unit 150 checks whether or not there is a repeated phrase using the extracted colloquial string, checks whether or not the dismantled phoneme exists, or compares the string with the string stored in the verification data dictionary. It is checked whether there is a corresponding part, and it is determined whether to apply each calibration method shown in steps S140 to S160 according to the inspection result.

In steps S140 to S160, error correction is performed on the colloquial strings extracted by the calibration method determined through step S130. Steps S140 to S160 that perform colloquial correction on any colloquial character string may be selectively applied in some cases and may be overlapped at the same time.

Step S140 is a step of removing the repeated phrases, the repeated phrase removal unit 151 in the colloquial correction unit 150 extracts and removes the repeated phrases that are unnecessarily repeated in the colloquial string.

Step S150 is a step of performing the disassembly syllable combination process, after the disassembly syllable combination unit 152 in the colloquial body correcting unit 150 extracts the disassembled syllables in the colloquial string, combines the consonants and vowels of the disassembled syllables to correct the correct syllables. Restore

The step S160 is a step of performing a string correction process, and may simultaneously perform string correction such as spell correction, spacing correction, emoticon correction, and slang correction using the verification data dictionary 153_1 and the correction rule dictionary 153_3. Here, the string correction unit 153 in the colloquial correction unit 150 learns the correction rules for the verification data dictionary 153_1 and stores the correction rules in the correction rule dictionary 153_3, thereby constructing the correction rule dictionary 153_3. In the verification data dictionary 153_1, demonstration strings including colloquial errors and correction strings correcting errors of the demonstration strings are stored.

The step S130 of determining the correction method in advance according to the extracted colloquial character string may be omitted.

In operation S170, when the error-corrected colloquial string is output through the colloquial correction unit 150, the calibration data return unit 160 includes the corrected colloquial string in the input data to generate the corrected input data.

4A to 6 are detailed flowcharts of some of the steps shown in FIG. 3 and illustrate a repetitive phrase removing process, a disassembled syllable combining process, and a string correction process, respectively.

The repeated phrase removing process performed by the repeated phrase removing unit 151 is performed through a series of algorithms as shown in FIG. 4A.

First, when the colloquial string is received, the repeating phrase removing unit 151 searches for the colloquial string received from the leftmost side to the rightmost side through the search for the repeated phrase in steps S141 and S142 and checks whether there is a repeated portion. . If there is a repeated portion in the colloquial string, step S143 is performed to extract an unnecessary phrase that is repeatedly repeated in the colloquial string.

N strings that form colloquial strings w ₁ ,… When expressed as w _n , iterative search is performed from left to right for a given string. An iterative phrase search algorithm is described in more detail as follows.

The first step is that for any letter w _i (1 ≦ i <n) between w ₁ to w _n , the same letter is w _{i + 1} ,... , check whether w _n exists.

The second step, the same letters _{w j (i <j ≤ n} ) and w _i is present, the substring w _i, if ... , w _{i + k} and w _j ,… , w Check for _{j + k} matches with each other. Where k is the distance from w _i to w _j and is defined as k = ji-1.

The third step is substring w _i ,... , w _{i + k} and w _j ,… If w _{j + k} matches, then w _i ,… , w _{i + k} as repeating string elements, w _j ,… , w Specify _{j + k} as a repeating phrase.

Fourth, the string w _{j + k + 1,} that follow w _j ... , w _n is repeated from the second process described above to check whether there are more repetitive phrases.

This process is shown in Figure 4b.

For example, given a colloquial string of 'Now bye bye', the repeating phrase removing unit 151 finds a part of the wording bye bye. At this time, 'by' is a repeating string element, and k is 2. Bye-bye is a case where the phrase "bye" is repeated three times.

In the repetitive phrase removing unit 151, a repetitive phrase dictionary 151_1 in which character strings subject to correction exceptions are stored is constructed. The repetitive phrase dictionary 151_1 is for performing repetitive phrase removal only on strings that do not belong to the correction exception object.

The repetitive phrase removing unit 151 searches for the repetitive phrase dictionary 151_1 in the process of checking whether an exception is performed in steps S144 and S145, and determines whether or not the repetitive phrase extracted from the colloquial string is in the repetitive phrase dictionary 151_1.

When the extracted repetitive phrase is in the repetitive phrase dictionary 151_1, it is determined that the repetitive phrase is not a correction target and the process of removing the repetitive phrase ends. For example, a repetitive phrase dictionary 151_1 is constructed for a case in which a repetitive phrase is a word such as 'snapping', and a string searched through a repetitive phrase search process is stored in the repetitive phrase dictionary 151_1. In that case, no repetitive phrase removal process is attempted. The repetitive phrase dictionary 151_1 storing the correction exception objects may be manually constructed as in the case of collecting colloquial sentences in a communication environment.

If the extracted repetitive phrase does not exist in the repetitive phrase dictionary 151_1, step S146 corresponding to the repetitive phrase removing process is performed to remove unnecessary repetitive phrases in the colloquial string. If the searched repetitive phrase is not subject to correction exception, the sentence is converted so that the word (repeated string element) appears only once in the extracted repetitive phrase by removing unnecessary parts.

At this time, if a predetermined phrase is repeated N times (N is a natural number of 2 or more) in the colloquial string, the repeated phrase removing unit 151 sequentially reduces the number of repetitions one by one and removes the repeated phrase until only one phrase is left. To perform. That is, the repetitive phrase removing unit 151 removes only one repeated string element at a time in order to reduce the possibility of error, and then checks the repetitive phrase dictionary 151_1 again.

For example, if the user enters 'by-bye', first modify the string from 'by-bye' to 'by-by' because 'by-bye' is a repetitive phrase and 'by' is a repetitive string element. After that, it is checked whether the corresponding string is in the repetitive phrase dictionary 151_1.

If the 'bye' does not exist in the repetitive phrase dictionary 151_1, the repeated string element is removed from the string and modified to 'by'. This process is repeated until only one repeating string element remains in the string corresponding to the repeating phrase.

When the "bye" is included in the repetitive phrase dictionary 151_1, the repetitive phrase removal process for the word is not performed again, and the repetitive phrase removal process is terminated as it is.

5 illustrates a combination algorithm of disassembled syllables for restoring disassembled syllables. Disorganized syllables are when consonants and vowels are input correctly when the Korean characters are input on an information processing device that implements a communication environment such as a computer or a mobile phone. (E.g., 'hehyan').

Referring to FIG. 5, a combination algorithm of disassembled syllables implemented through the disassembled syllable combination unit 152 will be described.

The disassembly syllable combination unit 152 sets a check pointer through step S151 to designate the N (N is a natural number) character at an arbitrary position in a colloquial string as a reference character. Here, the disassembly syllable combination unit 152 may designate a reference character in such a manner as to increase the check pointers that designate the position of the character to be examined for disassembly syllables one by one. The disassembly syllable test is performed one by one from the leftmost side to the rightmost side of the spoken character string.

In step S152, the disassembly syllable combining process is terminated when the entire sentence constituting the spoken character string is finished, otherwise the subsequent process of steps S153 to S158 is performed.

In step S153, the disassembled syllable combination unit 152 checks whether the reference character of the current position corresponds to the Hangul consonant. For example, w ₁ ,. For a letter w _i at any position in a sentence of n letters up to, w _n , check whether the corresponding reference letter corresponds to a Korean consonant.

If the reference letter is not a consonant, the disassembled syllable combination unit 152 proceeds to step S154 and checks whether the next letter in the N + 1th position is consonant again. If the reference letter is a consonant, step S155 is performed to check whether the N + 1 th next letter is a vowel. In an example, if w _i is a consonant, it is checked whether the next letter w _{i + 1} is a vowel.

When the N th reference letter is a consonant and the N + 1 th next letter is a vowel, the disassembled syllable combination unit 152 combines the reference letter and the next letter which are the vowels and vowels in step S156 to form one syllable. Make. In addition, the process returns to step S154 to check whether the consonants are repeated again from the N + 2 th letter. In the example, if w _{i + 1} is a vowel, a combination of w _i and w _{i + 1} creates a letter, and w _{i + 2} starts consonant checking again.

If the Nth reference letter is a consonant, but the N + 1th next letter is not a vowel, the disassembly syllable combination unit 152 performs the S157 step, where the next N + 1th letter is the same as the Nth reference letter. Check if it is. In an example, if w _{i + 1} is not a vowel, it is checked whether w _i is the same consonant.

If the N th reference letter and the N + 1 th next letter are the same consonant, the disassembled syllable combination unit 152 recognizes the current consonant as one current consonant by combining the reference letter and the next letter through step S158, and then proceeds to step S155. We go back and check again whether the next N + 2th character is a vowel. In an example, if w _i and w _{i + 1} are both the same consonant, the two consonants are combined to form one double consonant. For example, if w _i = 'ㅈ', w _{i + 1} = 'ㅈ', the combination of the two creates a 'ㅉ'. Then it is checked whether w _{i + 2} is a vowel. If w _{i + 2} is a vowel, the previously combined double consonant and w _{i + 2} are combined to form a letter.

If the N th reference letter and the N + 1 th next letter are not the same consonants, the disassembled syllable combination unit 152 returns to step S154 to determine whether the N + 2 th next letter is the new consonant or not. Will be checked again.

Through such a series of processes, the disassembly syllable combination process is sequentially performed for each letter from the leftmost side to the rightmost side of the spoken character string.

6 is a flowchart illustrating a string calibration process.

The string correction process simultaneously corrects spelling errors, spacing errors, and emoticon recognition based on the same structured dictionary and rule-based learning algorithms. The string correction unit 153 for performing this is associated with the verification data dictionary 153_1, the learning execution unit 153_2 for learning the correction rule largely, the correction rule dictionary 153_3 for storing the learned correction rules, and stored corrections. It consists of a calibration execution unit 153_4 that performs actual calibration by applying the rule (see FIG. 2).

In step S161, the verification data dictionary 153_1 is constructed. The designer pre-stores a plurality of test strings including colloquial errors and correction strings correcting errors of each test string in the verification data dictionary 153_1.

The step S162 is a step of constructing the correction rule dictionary 153_3 through the learning process of the correction rule, and may be subdivided into steps S162_1 to S162_4.

The string calibration unit 153 learns calibration rules for the demonstration strings stored in the verification data dictionary 153_1 and calibration strings thereof, and stores the learned calibration rules as the calibration rule dictionary 153_3. The calibration rule dictionary 153_3 stores calibration rules for string calibration, and the format of each calibration rule may be expressed as follows.

"Target syllable sequence → left syllable context correction syllable sequence right syllable context"

Here, the target syllable string is a string including the syllable string to be corrected and its left / right syllable context and is used to find a part to be corrected in the original message. The left syllable and right syllable contexts determine the left and right contexts of post-fertilized syllable strings. For example, if the original message is 'you are very pretty', assuming that 'pretty' is corrected to 'pretty', the correction rule can be constructed as follows.

"_ Pretty → _ Pretty"

Here, underscores (_) means blanks. However, in the case of emoticons, an arbitrary mark of 'M:' is used because it must be extracted and displayed separately in a sentence. 'M:' means to separate the syllable strings from the sentence. The example below shows the result of applying the correction rule and correction rule to recognize the emoticon '-.-'.

"Calibration Rule: -.- → Hmm M: [-.- Anger / Toy] *" (where * means applicable to all contexts)

"Original message: Hmm ..-What are you going to do?"

"Calibration Message: Hmm what are you doing?"

"Extract Syllables: [-.- Angry / Toy]"

This correction rule can be applied not only to the recognition of emoticons, but also to removing unnecessary words, symbols, swear words, and slang words from sentences.

In order to learn the calibration rules to be contained in the calibration rule dictionary 153_3, the designer collects a large number of corpus written in communication environment colloquial language, and then builds the verification data manually corrected for each sentence. By using rule-based learning algorithm, useful rules can be extracted.

An algorithm that can be used here is the Candidate-elimination Learning Algorithm. In the alternative elimination learning algorithm, the initial set consists of all expressible concepts, but given training examples, the Candidate Concepts that conflict with it are removed from the version space. Through this elimination process, the last remaining concept of concept becomes the concept to find.

The learning process of the calibration rule is described in detail by stages as follows.

First, in step S162_1, the demonstration string and the calibration string stored corresponding thereto are compared with each other. For example, a sentence A containing an error and a sentence A 'corrected in the sentence are compared with each other.

In step S162_2, the string correction unit 153 extracts different parts of the demonstration string and the correction string according to the comparison result. That is, in the example, the string corrector 153 finds the different parts a and a 'in the two sentences A and A'. Here, a means a substring included in A, and a 'means a substring included in A' and corresponds to a. If 'a and a' are displayed in the error sentence 'A' your girlfriend is pretty 'and' correct your girlfriend 'in the correction sentence' A 'corrected as follows:

In step S162_3, different portions extracted from the demonstration string and the correction string are regarded as the target syllable sequence and the post-corrected syllable sequence, respectively. Here, when there are more than one different part of the demonstration string and the calibration string, the longest part is regarded as the different part.

In step S162_4, the string correction unit 153 generates a preliminary correction rule by optimizing the left syllable context and the right syllable context that are located on the left and right sides of the corrected syllable string based on the demonstration string.

In the example, the string correction unit 153 considers a and a 'as the target syllable string and the corrected syllable string, respectively, and extracts the left syllable context and the right syllable context from sentence A to generate a correction rule. At this time, the left syllable context and the right syllable context may be diversified, so that correction rules for various contexts may be generated. An example of a calibration rule that can be generated in the above example is as follows.

Girlfriend → Girlfriend (Left context: 0, Right context: 0)

Girlfriend → _ Girlfriend (Left context: 1, Right context: 0)

Girlfriend → Girlfriend _ (Left Context: 0, Right Context: 1)

Girlfriend → _ Girlfriend _ (Left context: 1, Right context: 1)

Girlfriend → Yes_ Girlfriend _ (Left context: 2, Right context: 1)

Girlfriend → _ Girlfriend _ Yes (Left context: 1, Right context: 2)

Next, in step S162_5, the string correction unit 153 measures the performance of the generated preliminary calibration rule and adds the most high-performance calibration rule to the calibration rule dictionary 153_3. That is, the string correction unit 153 applies the generated preliminary correction rules to the hold-out data and measures each performance. Table 1 shows performance examples of the correction rules according to the left and right contexts.

These performance measures show which calibration rules perform the same, using less context. In Table 1, the rules with left and right contexts of (1,1) and (1,2) respectively have the same performance with 86%. In this case, the rule using less context, in the above example, the rule with left and right contexts (1,1) is considered to be a more general rule and added to the correction rule dictionary 153_3.

Using this learning algorithm, only the more general, but more general, correction rules that exhibit the same performance can be extracted and added to the correction rule dictionary 153_3.

Step S163 is a step of applying a correction rule, and performs a string correction using the correction rule dictionary 153_3.

The application process of the correction rule is largely composed of three steps: S163_1 to search for the target syllable sequence, S163_2 to compare the left / right syllable context, and S163_3 to replace the spoken character string with the modified syllable sequence.

When the colloquial string is received, the string corrector 153 searches for a correction rule of a type corresponding to the colloquial string received from the calibration rules stored in the calibration rule dictionary 153_3 through steps S163_1 to S163_3. The colloquial string is corrected and returned by applying the found correction rule.

In operation S163_1, the string correction unit 153 searches whether the target syllable string matches the colloquial string in the verification data dictionary 153_1, and if there is a target syllable string, the target object from the verification data dictionary 153_1. Information about the syllable string is returned.

In this step, the character string corrector 153 checks whether the same character string as the target syllable string listed in advance is output while checking the syllable string from the leftmost to the rightmost side of the sentence.

For example, if the colloquial string given by user input is "ABCDEFG (alphabet represents any syllable)" and the target syllable string is "A", "AB", "ABC", "ABCD", the longest "ABCD" is selected as the target syllable sequence.

When the matching syllable string is found, the position of the syllable string on the sentence given as the spoken string, the left syllable context, the right syllable context, and the modified syllable sequence information are returned.

In operation S163_2, the character string corrector 153 determines whether or not the correction rule is applicable to the spoken character string by checking the left and right context of the spoken character string based on the returned information about the target syllable string.

The steps are broken down as follows.

First, the string correction unit 153 recognizes a colloquial string to be corrected as a target syllable string, and checks whether the left portion of the recognized target syllable string matches the left syllable context.

Then, it is checked whether the right part of the recognized syllable sequence matches the right syllable context.

If the left and right parts of the colloquial string match the returned left syllable context and the right syllable context, it is determined that the correction rule is applicable, and the subsequent steps for replacing the given colloquial string with the modified syllable sequence are performed.

Therefore, when it is determined that the correction rule is applicable, step S163_3 is performed to replace the spoken word string with the modified syllable string, and the correction rule of the target syllable string is extracted from the correction rule dictionary 153_3, and extracted to the spoken string. Correction rules are applied resulting in a corrected colloquial string.

In this step, the string correction unit 153 replaces the target syllable sequence recognized in the comparison step of the left / right syllable context with the corrected syllable sequence recorded in the correction rule dictionary 153_3.

To help understand, an example of emoticon recognition and deletion during string correction is as follows. For example, suppose a user enters the following sentence.

"Original message: Hmm ..-What are you going to do?"

The text string correcting unit 153 first searches the original message from the leftmost side to the rightmost side, and compares it with the target syllable sequence recorded in the correction rule dictionary 153_3.

Then, it is confirmed that the target syllable sequence of the preliminary correction rule included in the target syllable sequence matches a part of the original message.

"Pre-calibration rule: -.- → Hmm M: [-.- Anger / Toy] *"

The preliminary correction rule above is 'blem' with the length of the left syllable context 1, and the right syllable context includes all cases (*). In addition, since the target syllable string to be corrected starts with "M:", it is not simply to correct, but to extract the corresponding character string and display it separately.

Next, check whether the left syllable context of the preliminary correction rule appears in the original message. In the example, the word 'hmm' is on the left side of '-.-', so we can see that the left syllable contexts coincide with each other.

Next, check that the dominant verse context of the preliminary correction rule appears in the original message. In the example, the circumferential context is '*', so it is applicable in all cases, so it can be said to match.

Next, since the left / right symptomatic context coincides, correction is performed. In our example, we simply replace the string, delete the '-.-' part of the original message, and return the following <correction message, extracted string> pairs.

"Calibration Message: Hmm what are you doing?"

"Extract String: [-.- Angry / Toy]"

In this way, string corrections can be performed that include spelling correction, spacing correction, emoticon recognition, and correction.

This string correction process has the following advantages.

First, it can handle all four types of spelling error correction, spacing error correction, emoticon recognition and correction, slang recognition and correction. Therefore, four kinds of error recognition and correction operations can be performed at the same time by the same correction algorithm, and creation and management of correction rules can be simplified.

Second, each item of the correction rule, that is, the target syllable sequence, the left syllable context, the corrected syllable sequence, and the right syllable context are all composed of syllable units, so the accuracy and reproducibility are high. For example:

"Ipu → Pretty": Ipuda (pretty), Ipu (pretty), Ipu (pretty), Ipupona (pretty)

"Eat rice → rice _ eat": eat (eat), eat (eat), eat (eat)

Third, incorrect correction may frequently occur when only the string to be modified is considered without considering the context, whereas the accuracy of the correction is increased when the context is considered as in the present invention. For example:

"Editing rules: Fa ~~ → By ~~", "Estuary → and"

"Correct correction: study well pa ~~ → study well. Bye ~~"

"Bad correction: brother ~~ You're so smart → Oh ~~ You're so smart"

FIG. 7 is a diagram illustrating a screen that appears in an error correcting process of a spoken sentence according to an embodiment of the present invention. FIG.

Referring to FIG. 7, in the first screen D100, an error correction apparatus displays a screen that allows a user to select an error correction function after inputting an original message (for example, 'study well ~~'). do. The user may proceed to the second screen D110 by inputting an original message by operating a keypad corresponding to the input means on the first screen D100 and selecting an error correction function by pressing the correction button D101.

When the error correction function is executed, error correction is performed on the original message (for example, 'not good at studying ~~'), and the message corrected as shown in the second screen (D110) (for example, 'study well'). And, bye ~~ ') is displayed visually.

Although embodiments of the present invention have been described above with reference to the accompanying drawings, those skilled in the art to which the present invention pertains may implement the present invention in other specific forms without changing the technical spirit or essential features thereof. I can understand that.

Therefore, since the embodiments described above are provided to completely inform the scope of the invention to those skilled in the art, it should be understood that they are exemplary in all respects and not limited. The invention is only defined by the scope of the claims.

The error correction apparatus and method according to the present invention can efficiently recognize and correct various errors in spoken sentences appearing in a communication environment, and as a result, improve the performance of the natural language processing technology, thereby improving user satisfaction.

1 is a block diagram showing an error correction apparatus of spoken sentences according to an embodiment of the present invention.

FIG. 2 is a detailed configuration diagram of some components shown in FIG. 1. FIG.

3 is a flowchart illustrating a method for correcting errors in spoken sentences according to an embodiment of the present invention.

4A-6 are detailed flowcharts of some of the steps shown in FIG.

7 is an exemplary view showing a screen during an error correction process of a spoken sentence according to an embodiment of the present invention.

*** Explanation of symbols for the main parts of the drawing ***

110: input unit 150: colloquial correction unit

151: repeat phrase removal unit 152: disassembled syllable combination unit

153: string correction unit 160: correction data return unit

Claims (14)

An input unit for generating input data according to a user operation; A colloquial text extracting colloquial character strings from the input data and selectively performing one or more processes of removing a phrase, combining disassembled syllables, and correcting a syllable based on syllables as a basic correction unit. Correction unit; And Correction data return unit for returning the corrected input data by reflecting the corrected colloquial character string to the input data Error correction device of spoken sentences comprising a. The method of claim 1, wherein the colloquial correction unit, A repetitive phrase removal unit for extracting and removing an unnecessary repetitive phrase in the colloquial string; A disassembled syllable combination unit for extracting disassembled syllables from the colloquial string and restoring correct syllables by combining consonants and vowels of the disassembled syllables; And Learning and storing correction rules using syllables as basic correction units for a plurality of demonstration strings including errors and a plurality of correction strings for correcting errors of the plurality of demonstration strings, and based on the stored correction rules String correction unit to correct errors in strings Error correction device of spoken sentences comprising at least one of. The method of claim 2, wherein the string correction unit, A verification data dictionary for storing the plurality of demonstration strings and the plurality of calibration strings; A learning execution unit learning the calibration rules for the plurality of demonstration strings and the plurality of calibration strings; A calibration rule dictionary for storing the learned calibration rules; And A correction performing unit for searching for a type of correction rule corresponding to the colloquial character string among the learned correction rules and applying the searched correction rule to correct the spoken character string Error correction device of spoken sentences comprising a. The apparatus of claim 1, further comprising a post-processing corrector configured to perform written correction on the returned input data. Generating input data according to a user operation; Extracting a colloquial character string included in the input data; Generating a corrected colloquial string by selectively performing one or more processes of removing a phrase, combining disassembled syllables, and correcting a syllable based on syllables based on the colloquial string; And Generating the corrected input data by including the corrected colloquial character string in the input data Error correction method of spoken sentences comprising a. The method of claim 5, wherein the generating of the corrected colloquial string comprises: Repeated phrase removal process for extracting and removing unnecessary phrases repeated in the colloquial string, Decomposition syllable combining process of extracting deconstructed syllables from the colloquial string and restoring correct syllables by combining consonants and vowels Learning and storing correction rules using syllables as basic correction units for a plurality of demonstration strings including colloquial errors and a plurality of correction strings for correcting errors of the plurality of demonstration strings, and based on the stored correction rules. String correction process to correct errors in spoken strings, Error correction method of a spoken sentence, characterized in that the step of performing at least one of the process. The method of claim 6, wherein the repeating phrase removing process comprises: When a predetermined phrase is repeated N times (N is a natural number of 2 or more) in the colloquial string, a colloquial phrase is repeated until the number of repetitions is reduced one by one until the predetermined phrase remains. How to correct errors in sentences. The method of claim 7, wherein the repeating phrase removal process, And constructing a repetitive phrase dictionary for the subject of correction exception, and performing repetitive phrase removal only on strings that are not in the repetitive phrase dictionary. The method of claim 6, wherein the disassembly syllable combination process, Checking whether an N-th reference character at an arbitrary position in the colloquial string corresponds to a Korean consonant; If the reference letter is a consonant, checking whether the N + 1 th next letter is a vowel; If the next letter is a vowel, combining the reference letter and the vowel letter to create one syllable, and checking again from the N + 2 th letter; Checking whether the next letter is the same consonant as the reference letter; And If the reference letter and the next letter is the same consonant, comprising the step of combining the reference letter and the next letter to a single consonant, The method for correcting errors in spoken sentences, characterized in that to sequentially perform a test for each letter from the leftmost to the rightmost side of the colloquial string. The method of claim 6, wherein the string correction process, Storing the plurality of demonstration strings and the plurality of correction strings in advance to construct a verification data dictionary that is a target of calibration rule learning; Learning calibration rules for the plurality of demonstration strings and the plurality of calibration strings, and storing the learned calibration rules to build a calibration rule dictionary; Searching for a calibration rule of a type corresponding to the colloquial string among the learned calibration rules; And Correcting the colloquial string by applying the found correction rule Error correction method of spoken sentences comprising a. The method of claim 10, wherein the step of constructing the correction rule dictionary, Comparing a demonstration string including an error with a calibration string in which an error corresponding to the demonstration string is corrected; Extracting different portions of the demonstration string and the calibration string according to a comparison result; The extracted different parts are regarded as target syllable strings and modified syllable strings respectively, and the left syllable context and right syllable context that are located at the left and right sides of the corrected syllable string based on the test string are optimized for preliminary correction rules. Generating; And Measuring a performance of the generated preliminary correction rule and adding a high-performance correction rule to a correction rule dictionary. The method of claim 11, wherein the extraction of the different portions, If the different parts of the demonstration string and the correction string is one or more, the longest part is considered to be a different part. The method of claim 10, wherein the correcting of the colloquial strings, Searching whether there is a target syllable sequence matching the colloquial string from the verification data dictionary, and returning information on the target syllable sequence if there is a target syllable sequence; Determining whether a correction rule is applicable by examining left and right contexts of the spoken character string based on the information on the target syllable string; And If the correction rule is applicable, extracting a correction rule of the target syllable string from the correction rule dictionary and generating the corrected spoken character string by applying the extracted correction rule to the spoken character string. Error correction method of spoken sentences comprising a. The method of claim 6, wherein the string correction process, A method of correcting a spoken sentence, comprising performing at least one of spelling correction, spacing correction, emoticon correction, and slang correction.

Images