KR20020023543A - Korean language stemmer for natural language searching system and stemming method thereof - Google Patents

Abstract

PURPOSE: A Korean language stemmer for a natural language retrieval system is provided to automatically collect word variants, which appear in a database or a query language, as stemming types while processing a Korean natural language, so as to remarkably improve a retrieval capacity of a natural language text database of an online/offline information retrieval system and a Web environment. CONSTITUTION: A stemmer automatically removes stop words by a stop-words removing routine, and automatically processes the rest natural language-type texts. Two processing steps are implemented. The first step is a basic processing step of matching a rule table 1 with a rule table 2, and iteratively progressing an application process, then applying a fixing stem length. The second step is the second processing step of iteratively progressing matching processes of a rule table 3, a rule table 4, a stem dictionary, and a word end file, and applying rules. According to an irregular conjugation condition, a rewrite rule is applied to remove various types of ends of words which appear in a natural language text.

Description

Korean language stemmer and naturalization method for natural language search system {KOREAN LANGUAGE STEMMER FOR NATURAL LANGUAGE SEARCHING SYSTEM AND STEMMING METHOD THEREOF}

The present invention relates to a Korean language stemmer implemented as a main module of an information retrieval system and to a method of stamming the same in order to improve retrieval performance of a natural language text database in an online / offline information retrieval system and a web environment. It is about.

One of the major problems encountered in indexing and retrieval in natural language retrieval systems is that words carrying the same meaning in text databases have word variants. In general, morphological variants of words in Korean are generated by the phonological and morphological conditions under which words are composed.

The development of this stammer takes account of the morphological conditions that bring about changes in the text. For example, as shown in FIG. 11 according to the condition of the tip of the stem and the form of the ending in Korean, it has various variants according to the rule or irregular utilization of verbs and the semi-flexion of nouns. This results in a drop in the reproducibility level due to a missing search for a suitable record if the query and the index of the document do not exactly match.

In addition, in an information retrieval system that provides a ranked output according to the relevance ranking of documents, if the system does not consider all the morphological variants appearing in the document record, it affects the output ranking and the accuracy of the retrieval. Significantly lowers. In the user interface, when a user with limited knowledge of a terminology searches for a natural language using a query, a difference is made in the search result according to the form of the natural language used as the query.

In order to solve this problem, in an online information retrieval system, a professional searcher has typically created a search word by using the right, left, and both sides of a word as a search strategy, and used it in the search. However, this strategy has a limitation in that the effectiveness of the search results is determined by the searcher's expertise in using the searcher's search and system usage, and the various search strategies used in writing the questionnaire.

As described above, in the conventional information retrieval system, the retrieval of words appearing with morphological variants is not solved only by the index word matching algorithm of the system.

The assumption based on the present invention is that the semantically useful information in a word is present in the stem or root, and the change of the ending is changed only for grammatical purposes.

According to the present invention, variants of the same language transmit the same meaning. When the Korean word with different surface types is indexed in a document in the information retrieval system, the system generates a natural language text field when creating a natural language type query for the search request. It is an object of the present invention to provide a Hangul stemmer and a method of stamming that can automatically execute an automatic process for a record and a user's query word.

The term ending in Hangul stemmer includes a case in which a suffix and a survey are used according to a song's use in a Hangul word, and a ending generated by use of a verb. Hangul stammer according to the present invention is implemented as a module that maintains the meaning of the stem using the various dictionaries, rules and algorithms implemented in the system and removes various endings, using the indexed words stammed in the system for natural language information retrieval Significantly improve search effectiveness.

1 is a data flow diagram of the Hangul stemmer according to the present invention,

2 is a diagram showing an example of the configuration of rule table 1;

3 is a diagram showing an example of the configuration of rule table 2;

4 is a diagram showing an example of the configuration of the rule table 3;

5 is a diagram showing an example of the configuration of rule table 4;

6 is a view showing a configuration example of a stem dictionary;

7 is a view showing a configuration example of a mother file;

8 is a view showing an example of a 'degree' section of the rule table 1 of FIG.

9 is a view showing an example of a word that is terminated after proceeding to the rewriting step of the extended stemming step 1;

10 is a view showing in detail the stemming algorithm of the Hangul stemmer according to the present invention;

11 is a diagram for explaining a general morphological variant.

In order to achieve the above object, the Hangul stemming method according to the present invention comprises the steps of designing a predetermined kind of context-dependent rule table for stamming a natural language Hangul word, and calling the table recursively in the system. Characterized in that it comprises a step of applying in.

In the Hangul stemming method, the method may further include preparing a stem dictionary and a mother file for processing various irregular applications, and applying one or more matching methods according to conditions.

The matching method is at least one of complete matching and partial matching.

Furthermore, the Hangul stemmer according to the present invention is characterized in that the processing means for processing the index word processing target of Hangul by applying the minimum length rule, and the index word processing target processed by the processing means according to the natural language syllable conditions to be processed currently It is characterized in that it is provided with a streaming means for processing by applying the rule of.

In addition, in the Hangul stemmer described above, the index word processing target of Hangul includes a message and a verb.

The minimum stem length rule applied by the processing means is a fixed stem length rule, and the rule applied by the stemming means is a context-dependent stem length rule.

(Example)

Prior to the description of the embodiments of the present invention, the terminology removing routine, which is not included in the algorithm claimed by the present invention, is executed by reference before the driving of the stammer according to the present invention.

Terminology elimination routines use the terminology file consisting of words and usage forms that are predicted by the system to be useful for retrieving suitable information in database searches before executing the algorithm of the present invention to drive the stemmer according to the present invention. It is a procedure to remove from query and database before executing. To do this, an algorithm for developing a stopword file, creating a stopword removal program, and calling a stopword removal procedure is implemented by an algorithm that compares the stopword file with the current word and deletes a word that has been completely matched.

The following sources were used to collect candidate stopwords that could be included in the stopwords dictionary. In other words,

. Word groups belonging to the subcomponents of Korean sentences

. Word groups included in the part-of-speech group listed in the Korean reverse order dictionary

. References to stopword files on existing systems

. Candidate stopwords extracted from experimental literature population

Except for duplicate words from candidate stop words collected in each file from the above information sources, all the refractive forms included in each term were composed for the words with songs and utilization endings to form a final stop word dictionary.

In addition, in order to select the last stopword from the candidate stopword list, the following criteria were used. In other words,

. Adverbs belonging to the components of the Hangul sentence, as adjectives, meaningless meanings: which, that, etc.,

. Investigative, noun, auxiliary, and pronoun words: one, none, no, it is, etc.

. Words in the part-of-speech area that belong to the main components of Hangul sentences or words that are not useful as the main words: elderly, young, low, young, etc.

On the other hand, the following words may be considered as candidate stopwords with low occurrence frequency and low meaning as a subject in the document group. However, the following words are not included in the stopword dictionary for general purpose stopwords. . In other words,

. Word types determined by subtle modulations or expressions of meaning: white, white

White, white, long, long, etc.

. Word types of various expressions according to the movement of objects or people:

I embrace you,

. As a result of the experiment of the Korean literature, it was shown that a large number of Chinese characters by Korean notation are used in the Korean sentence, and among them, a plurality of homonyms are included as shown in the following example. For example, enemy (noun), non content-bearing (suffix); Sex (noun), non content-bearing (suffix); When: time: noun; time: noun; Inner side: noun, knew: verb, etc. Words of this type are not included in the stopwords dictionary.

As a result, a final stopword dictionary consisting of a total of 2,469 words was constructed, which is used in routines to exclude stopword candidates from the group of words to be analyzed in the stemming algorithm.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

The Hangul stemmer according to the present invention is composed of an extended stemming step, which is a first processing step and a second processing step. According to the rules or irregularities of the morphological surface form of the input text, the syllable of the processed result can be used in the search engine after only the first processing process. After going through the subprocedures, the final string is generated and can be used by the search engine.

Detailed processing of the present invention is as follows.

First, when a natural language string is input to the system, the stopword removing routine uses the stopword file to delete the stopword to check whether the input word is a stopword. Then, the stop word processing routine is executed and the Hangul stammer processing step is performed on the remaining syllables.

In the first processing step, if the ending is matched in rule table 1 with a basic ending elimination routine, the rule matched in rule table 1 is applied only when the length condition between fixed words is satisfied. Each rule constituting the rule table 1 is coded with elements indicating the ending order of syllables, the syllable to be removed, the activation of the rule table 2, the iterative progression of the rule application, or the end of work. When the applying step of the rule table 1 is finished in the primary processing step, the processing proceeds to the applying step of the rule table 2 as the context-dependent processing step. Activation of Rule Table 2 corresponds to one of the conditions defined by the system after the removal of the mother in the initial processing stage, namely abbreviation, irregular use, between the stem and the mother, or between the words and verbs. By applying rule table 2, the processing is performed so as not to result in over stemming or under stemming.

According to the surface layer of the result syllable by the first processing step, the Hangul stammer proceeds to the extended stamming step. The extended stemming step is based on the checking of the condition implemented by the system algorithm for the natural language text which requires more complicated processing, the querying of various dictionaries built in the system, the rewriting rules, etc. Create In the extended stemming phase, the first phase checks the context-dependent minimum stem condition and proceeds to the routine corresponding to the current input syllable among the following three cases defined in the system.

If the input syllable is one syllable, matching is performed with the rule table 3, and then the matched rule is applied to find the result. On the other hand, if no rule is found, the system outputs the current input syllable as the final processing result and ends the operation.

If the syllables in the storage place are two syllables, the order of processing is to apply the rule that satisfies the conditions in the order of matching with the dictionary of the dictionary, complete matching of the mother file and the rule table 4, and matching with the rule table 5. do.

If the syllable after the first processing routine is 3 syllables or more, the entry corresponding to the number of syllables increased by one syllable in the current string is searched using a partial matching technique in the stem dictionary. If a partial match is found, the current matching entry is output as the final result except the last syllable. If the partial matching fails, the system outputs the current word as the final result.

Next, the above-described user natural language query processing will be described in detail with reference to FIG. 1.

The process of query processing, database text processing, and database search results for query terms until the search results are output in the system of which the presenter is implemented are as follows.

<Processing of question>

step result

<First Query> ‘About the development of multimedia or hypermedia, or high

Research on authoring tools for pertext authoring

Removing stopwords Hypertext in the development of multimedia or hypermedia

To the authoring tool

<Primary treatment>

R1 for multimedia and multimedia development

Hypertext authoring to authoring

R2 Multimedia Hypermedia Development Hypertext Authoring

Authoring tool

<Secondary processing>

Multimedia hypermedia development hypertext authoring tool

<Processing of Various Records in Database>

Before stemming After stemming

Multimedia in Multimedia

Multimedia

In multimedia

Multimedia

Hypermedia with Hypermedia

Hypermedia

Hypermedia

With hypermedia

Hypermedia is

Than hypermedia

Hypermedia

Development

Develop

Developed

Developed

Developed

Developed

Development

Developed

Development

Hypertext only in Hypertext

In hypertext

Between hypertexts

As hypertext

Copyrighted

Action

To author

Copyright

Authoring

Authoring Tools on Authoring Tools

To authoring tool

Authoring tool

Even the authoring tool

Authoring tools

When the database search engine is running, each text data record in the database to be accessed excludes stop words, and each syllable in natural language form is processed in sentences, endings, suffixes, and search parts used for song processing. Construct a compressed text database stored as a stem that conveys only meaning.

On the other hand, the matching process between the query and the database is performed as follows.

When searching the system, even if the user is not familiar with the search logic (e.g., Boolean logic) or search strategy supported by the search system, the user inputs a natural language search sentence that is used in everyday life. In case of database processing, the system generates the user's query sentence as a string consisting of the stem of each search word through the recursive processing by this algorithm.

The search procedure matches records from the database that match the query string, and outputs the searched result sets in the order that the system predicts that the user will be most satisfied using the search algorithm adopted by the system.

The above description is based on FIG. 1 for the query "substantially available for the development of multimedia or hypermedia or for authoring tools for hypertext authoring" which can be used to describe the operation of the stammer of the present invention in detail. Here is an example search.

Here, the example query does not include words that need to proceed to step 3.2 of the secondary processing. Therefore, after processing the word through the 'stem dictionary' query in step 3.1 of the secondary processing, this is the final form. Therefore, the process proceeds to a search procedure (<matching query and database>).

Although there is no <processing of various records in the database> in FIG. 1, this part includes that the algorithm of the present invention not only processes the text of the query, but also processes the text records of the database, and processes the processed query and the processed query. This is because the final search result is output from the system by matching with the text database.

Next, a configuration example of the rules tables 1 to 4 will be described with reference to FIGS. 2 to 5.

Rule Table 1:

In Rule Table 1, each rule is composed of the following four elements, an example of which is shown in FIG.

-The mother of syllables in the reverse order of syllables listed in the mother dictionary, which is added to the stem when the chant is used or when the word is used;

-Optionally adopted symbol '>>', added to the end of the 98 cases investigated for the present invention and proceeding to activation of rule table 2;

A number from 0 to 9 representing the number of syllables of the ending to be removed;

The symbol '>' indicating a cyclical application of the rule only if the condition is met;

Example 1) Pray 4

If the syllables entered from the system and read sequentially from the previous one match the syllable type 'Toraijika' in the rule found by searching the R1 file, the four syllables [8 bytes] are deleted from the beginning, and the R2 file Activate to search for rules that begin with a 'to' in R2 files.

Example 2) Doraitehan 5>

If the syllable of the entered word matches the rule 'Doritehan' found in R1, delete 5 syllables from the beginning, then activate R1 again and continue matching the first syllable of the remaining syllables with R1. .

Rule Table 2:

In Rule Table 2, each rule consists of the following five elements. If the rule is matched, the rule indicated by the rule is checked, and then the action defined by the rule is performed in the procedure. An example of the rule table 2 is as shown in FIG.

-Mother in the reverse order or abbreviation between mother and fish;

A number from 0 to 9 representing the number of syllables of the ending to be removed;

-The symbols ‘~’ and ‘!’ Representing the number of syllables immediately before the first syllable of the ending or abbreviated ending;

The symbols '>', '>>' and '.' Indicating the continuation or termination of the application of other rules in the rules table;

-One or more syllables to be replaced after removing the ending

Example 1) LA!

If the first syllable among the syllables of the word passed in the first process starts with 'la' and read the syllable before 'la' and satisfies the condition of 1 syllable (2 bytes) or more, 'la' corresponding to the first syllable of the currently read syllable Delete 'and activate R1 to continue matching.

Example 2) Hours ~ 0.

If the first syllable of the word syllable passed in the first step matches the poem and there are no syllables before the poem syllable, read the syllable as it is and store it in the temporary memory.

Rule Table 3:

This rule table is a form of a text file that examines the regularity and irregularities of words in which Korean words and verbs have single syllable stems, which are described in books and reports on Korean dictionary and morphology. This is designed to minimize the probability that the stemmer according to the present invention will have an over-source tampering, ie, an error ending with a syllable or spell to be deleted. This file consists of 716 syllable Korean monosyllable stems.

Ex) Hull 1 hull.

If the first syllable of the word syllable that enters the secondary processing stage (extended stemming) matches 'Hun', the read syllable 'Hun' is converted to 'Hull' and stored in temporary storage, and the current procedure is terminated. do it.

Rule Table 4:

Although the rules table is a word that has stems with different meanings (eg, pool solve; fu draw) in Korean, when the stem meets the ending or suffix, the rules have the same surface type (eg, pool). There will be cases in which forms will be solved according to will solve; full will draw) or ending (e.g., solve and parser draw and solve). Alternatively, the class of words is completely different (eg, verses, idioms), where a verb has a form of use, while a verse does not utilize that form (full will draw; pool a gum).

Example 1) Runu 1

If the first two syllables (4bytes) match 'Lunu' after reading the word phrases entered into the secondary processing step sequentially, convert only the string corresponding to the first two bytes into 'Hae' and store the whole syllables (4bytes) Save it and exit the processing routine.

Example 2) Russel 2 Luth

If the first two syllables (4bytes) match the 'Russle' after reading the word phrases entered into the secondary processing step sequentially, all the read strings (4bytes) are converted to 'Less', and the words (4bytes) are stored in the memory. Save and exit the processing routine.

Next, the structure of the stem dictionary and the mother file will be described with reference to FIGS. 7 and 8.

First, the stem dictionary is described.

This stemming dictionary is used to supplement the case where words with more than two syllable stems are over-stamped, i.e., over-deleted syllables that should not be deleted. It is designed to avoid errors that are handled as The stem dictionary does not have a separate format, but consists of a text file for complete matching. However, in the case of Korean compound words, in addition to including the compound word in an entry, when a compound word is composed of one or more stems, these words are regarded as each stem and these words are also included in the dictionary. A schematic example of the stem dictionary is shown in FIG. 7.

Since the mother file is not used alone in the system, but together with rule table 3, the rule matched in rule table 3 is applied in the procedure when both conditions of these two files are satisfied when searching for processing conditions. do. An example of a mother file is as shown in FIG.

The following describes in detail how these rule tables are created and how they are applied, and further describes the operation of these rule tables.

In the implementation of the Hangul stammer algorithm according to the present invention, all five rule tables may be used sequentially according to the morphological characteristics of the input words. After proceeding, if the current rule encounters an end-of-process symbol, the operation ends. If there are no more rules in the matched rule table that match the current word's morphological condition, or if the rule indicates the end of processing, processing terminates. Otherwise, the routine continues until the minimum stemming condition is met. After proceeding, the word stored in the current storage location is output as the final processing result.

For the application of rules in the system, each rule is defined as a C-type structure so that the computer can recognize the structure of the rule table and run it as commanded by the program. In the declaration section, we define the structure of each rule table, and check the condition of rule application in each subroutine of the program.

Each rule table (R1, R2, R3, R4), stem dictionary, and mother file, when called to apply to the processing steps required by the algorithm's corresponding routine, the rule table, or one or more rule tables, depending on the conditions of the current word. If the rule is searched for and the matching rule matches, the action defined in the rule is executed. The types of actions to be executed are syllable removal, maintenance of the current syllable, rewriting syllables, activation of a rule table called by the current rule after processing, and termination of processing.

The following description will be given with reference to FIG. 8, which is one of the rule tables used in processing a word whose morphological characteristics are not complicated.

Searching for the rule in the corresponding section ('-degree') that matches the last syllable of the input word and applying it as follows shows the main characteristics of the first stage of the algorithm. That is, for the input three words 'motor ado', 'temperature' and 'hearth degree', 'motor ado' is deleted by applying the ending '-do' among candidate endings '-rado' and rule table 2 is activated. Then, the condition is examined in the extended stemming step, and '-la' is applied to output the 'motor' to the stem. The input word 'temperature' is output as 'temperature' without changing the shape of the input word because the minimum length condition is not satisfied in the step of searching for a condition that a rule can be applied in the ending elimination routine. Candidate endings applicable to the word `` hearth degree '' are ‘-로도’ and ‘-도’. However, the application of the candidate ending ‘-로도’ does not meet the minimum stem length condition, so the ending ‘-도도’ is adopted. The application of the ending parent '-degree' in rule table 1 indicates activation (>>) of rule table 2, not the first step. Therefore, after checking whether the condition is satisfied in the extended stemming step, the word 'hearth degree' Delete the ending '-do' for the output of 'hearth' as stem.

The extended stamming and rewriting routines, which are the second stages of the algorithm, proceed to the application of the rule table 2, and the words to be analyzed are the input words in which some syllables of words are removed or maintained in the first stage. The detailed steps involved in the extended stemming and rewriting routines are as follows:

In rule table 2, we directly access the matched section by matching the last syllable of the word, and then compare the syllables of the analyzed word with the syllables of the rule sequentially in the section to search for candidate rules. Proceed until found.

When a rule is found, the rule checks the character after the current syllable.

-If the character of the next position is a number, the syllable of the same size as the number indicated by the rule is removed from the word to be analyzed, and if it is a symbol, the condition is satisfied by examining the condition immediately before the syllable of the matched current position. Only delete as many syllables as indicated in the rule.

-After deleting the ending, the rule checks whether there is a syllable to replace, and if there is a syllable to replace, inserts it at the current position.

-After replacing the syllables, read the position after the replacement syllable in the current rule to decide whether to finish or continue the work. In other words,

Return to the first step and proceed with the rules of the next step;

Proceed with extended stemming by continuing to search the other sections of rule table 2;

This is the case when the stemming work routine is terminated for the word currently being analyzed, and after the mother is removed and rewritten, the remaining syllables are output as stems.

An example of the terminated word after proceeding to the rewriting step of the extended stamming step 1 is shown in FIG. 9.

Next, the Hangul stemmer algorithm of FIG. 10 will be described in detail in numerical order.

1. By applying the condition of fixed word length (two syllables), the algorithm checks the current syllable number for the current word processed using the rules tables R1 and R2.

2. If the number of syllables to be processed currently is 1 (case a), the process proceeds to the next step (3 in FIG. 10), and when a matching rule is found in the corresponding rule table, the result of applying the rule is output and the matching is made in the corresponding rule table. If no rule is found, print the current word.

If the number of syllables to be processed currently is 2 (case b), the process proceeds to the next step (3 in FIG. 10).

If the number of syllables to be processed currently is 3 or more (case c), the process proceeds to the step (4 in FIG. 10).

3. Sequentially proceed as follows with the words accepted in the previous step.

If the matched word string and the stem dictionary match the entry in the stem dictionary, the process proceeds to the step (7 in FIG. 10) with the current word.

Otherwise, if there is no entry in the stem dictionary that matches the string of the accepted word, these two conditions are compared by comparing Rule Table 3 with the second spelling of the current syllable, and the first spelling of the mother file and the current syllable. If all of these are satisfied, the process proceeds to step 5 of FIG. 10.

If any of the above conditions are not satisfied, matching of the string of the current syllable and the rule table 4 is started. As a result of the matching, if the corresponding rule is found in the rule table 4, the process proceeds to the next step (5 in FIG. 10). If the rule is not found, the process proceeds to step 7 of FIG. 10.

4. If the number of syllables to be processed is 3 or more and accepted in the previous step, matching with the stem dictionary is started. If there is an entry that perfectly matches the syllable to be compared currently, it proceeds to the corresponding step (7 in FIG. 10). Otherwise, that is, if there is no entry, it proceeds to step 6 (6 in FIG. 10).

5. If the rule is searched in the syllable and the rules table currently being compared, and this is a rule that has a context-dependent condition as an action defined by the rule, check all the conditions of the current string and check whether the condition required by the rule is satisfied. (Eg, the number of syllables before the current spelling, syllable type, whether to delete the current syllable, or rewrite the current syllable). If the condition is satisfied, the rule is applied and the ending is deleted. Then, the flow proceeds to the corresponding step (7 in FIG. 10).

If the currently matched rule is a rule that does not define a context-dependent condition, the syllable being compared is deleted and the process proceeds to step 7 of FIG. 10.

6. If the current syllable is 3 or more (4 in Fig. 10) and this step is called by a routine, two types of matching, namely perfect matching or partial matching, are performed to search for the entry in the stem dictionary. Start searching by First, the stem dictionary is searched for an entry that completely matches the current syllable number (three syllables). If the entry is found, proceed to the next step (7 in FIG. 10). If not searched, the number of syllables plus one syllable is stored as the number of syllables in the current string, and then partial matching is performed to compare the character string only up to one syllable smaller than the number of syllables currently stored. If is satisfied, the matched entry is stored and output as a processed string.

7. Outputs the syllable processed as the final result and ends the routine.

In addition, in Korean, each syllable is combined to form one word, and according to various phonological and morphological conditions, the use of rules and irregularities, shortening of syllables, homonym between words and verbs, and irregular use of monosyllable words may occur. have. Therefore, in consideration of these characteristics, each rule in the rules table must be unconditionally matched to the syllable of the current syllable, the number of syllables of the current syllable, and even before the syllable of the word to be processed and the beginning syllable of each rule are completely matched. It is designed to check conditions such as rules designed to distinguish between a statement and a statement.

If such a condition is satisfied, the rule on the rule table is applied after checking the minimum stemming condition in the first processing step, and the procedure according to each condition is checked by checking the number of syllables of the word currently being processed in the second processing step. It is designed.

In the present invention, the fixed stem length is applied in the first processing step and the context-dependent stem length is applied in the second processing step corresponding to the extended stemming when the Hangul stemmer is implemented.

The application of fixed word length means that the algorithm executes by calling a recursive routine as long as the condition is satisfied for the word to be input and processed. In the process, if the number of syllables stored for the current processing is 2, it is designed to exit the cyclic routine and proceed to the secondary processing step. This design reflects the statistical characteristics of the number of syllables spoken in Korean so that the fixed word length is applied first in the first processing step.

In the subsequent secondary processing step, context-dependent stem length is applied, which proceeds to the corresponding program routine based on the number of syllables to be processed, even if the same syllable number is the syllable to be processed or the form of the syllable before the current syllable. The theoretical condition is examined first and the action defined in the rule is taken only when the condition and the condition defined in the rule are completely matched. The context-dependent condition of the currently processed word is defined in the rule table as described above. .

The Hangul stammer of the present invention, by implementing the Hangul version of the natural language search system (SMART) developed by Cornell University in the United States and tested as an index procedure of the system, statistically verified a significant increase in the standard search efficiency. In other words, as a result of statistical verification between the search using Hanger and the unused search, it was confirmed that the mean reproducibility and mean accuracy of search were significantly improved at the one-sided verification level of 0.05.

In addition, this invention is not limited to the Example mentioned above, A various deformation | transformation is possible, of course, in the range which does not deviate from the summary of this invention. For example, the Hangul stemmer according to the present invention may be mounted and used in a search engine which is generally used. Even in this case, as described above, the search efficiency can be significantly improved.

As described above, according to the present invention, it is possible to provide a Hangul stammer capable of remarkably improving the search performance of a natural language text database in an online / offline information retrieval system and a web environment.

Claims (6)

Designing a predetermined kind of context-dependent rule table for stamming natural Hangul words; And recursively calling the table in the system and applying the procedure in a procedure. The method of claim 1, further comprising preparing a stem dictionary and a mother file for processing for various irregular utilization, Hangul stemming method, characterized in that to apply one or more matching method according to the condition. 3. The method of claim 2, wherein the matching method comprises a perfect match and a partial match. Processing means for processing the index word processing target of Hangul by applying the minimum length rule; And a stemming means for processing the index word processing object processed by the processing means by applying a predetermined rule according to the natural language syllable condition to be processed at present. [5] The method of claim 4, wherein the Hangul indexer processing object includes a verb and a verb. 6. The Korean stemmer according to claim 4 or 5, wherein the minimum stem length rule applied by the processing means is a fixed stem length rule, and the rule applied by the stemming means is a context-dependent stem length rule.