KR100304654B1 - Method and apparatus for analyzing korean document - Google Patents

Abstract

PURPOSE: A method and apparatus for analyzing a Korean document is provided to perform a language coding of a document by analyzing all sorts of morphemes including function words by units, and analyzing sentence structures using a Korean analysis dictionary and a morpheme connection information table. CONSTITUTION: A Korean analysis dictionary(14) is equipped with information on morpheme category codes and so on. A pre-processing unit(10) separates a document inputted into paragraph units. A morpheme analysis unit(12) verifies a morpheme using a morpheme connection information table(16). A paragraph analysis unit(18) analyzes whether a paragraph is used as a paragraph element using the morpheme outputted from the morpheme analysis unit(12). A language code generating unit(20) generates the paragraph element outputted from the paragraph analysis unit(18) into a language code using a pronunciation dictionary(22) and a pause length table(24).

Description

Korean document interpretation method and apparatus

1 is a block diagram showing a conventional Korean document analyzing apparatus.

2 is a block diagram according to an embodiment of an apparatus for interpreting Korean documents according to the present invention.

3 is a diagram showing an example of preprocessing in the preprocessing section of FIG.

4 is a diagram showing an example of syntax analysis in the syntax analysis unit of FIG.

FIG. 5 is a diagram showing an example of reading a number in the language code generator of FIG.

FIG. 6 is a diagram showing an example of a rest length and its meaning in the rest table of FIG.

FIG. 7 is a diagram showing an example of a rhyme pattern in the language code generation section of FIG.

8 is a chart showing the performance evaluation of the Korean document interpretation apparatus according to the present invention.

* Explanation of symbols for main parts of the drawings

10: preprocessing unit 12: morphological analysis unit

14: Korean dictionary of interpretation 16: Connection information table

18: syntax analysis unit 20: language code generation unit

22: Pronunciation dictionary 24: rest table

The present invention relates to a method and apparatus for interpreting Korean documents, and in particular, to interpret not only functional words but also all morphemes by unit, by interpreting phrases using a Korean interpretation dictionary and a morpheme connection information table and classifying them by category, as well as rest periods and lengths. The present invention relates to a method and apparatus for interpreting Korean documents to achieve language encoding of documents by correcting.

In general, in a document speech converter such as an unlimited speech synthesizer or a text / speech system, when a human voice is spoken, artificially inserting information of phonological and rhyme that is naturally generated makes a good sentence meaningful and the sound quality is also clear. And nature. According to this need, the electric and mechanically generated voices need to adjust the rhythm and change the rhythm according to arbitrary laws.

Conventionally, as shown in FIG. 1, the phrase analyzer 1 analyzes a phrase or a clause based on information of a sentence extracted by using a Korean functional word, that is, a search and a ending, and reads a word or the like.

However, in such a case, it is difficult to characterize each phrase in a sentence only by Korean word analysis or functional word, and there is a problem of degrading the naturalness of the output sound.

Accordingly, an object of the present invention is to solve all the morphemes as well as functional words in order to solve the above problems by interpreting the phrases using the Korean interpretation dictionary and the morpheme access information table, and by category, not only the rhythm but also the rest period and length. It is to provide a method for achieving language encoding of documents by correcting the problem.

Another object of the present invention is to provide an apparatus most suitable for realizing the Korean document interpretation method.

In order to achieve the above object, the present invention provides a method for interpreting a Korean document, comprising: a preprocessing step of separating input documents into word units; A morpheme analysis step of identifying a morpheme, which is a unit of spacing, using a Korean interpretation dictionary and a connection information table of a word document formed in the preprocessing step; A syntactic analysis step of analyzing what syntax element, the smallest unit of a sentence component, is written using a morpheme formed in the morphological analysis step; And a language code generation process for generating a language code using a pronunciation dictionary and a rest period table for the syntax elements formed in the syntax interpretation step.

In order to achieve the above object, the Korean document interpretation apparatus according to the present invention includes a Korean interpretation dictionary, a morpheme access information table, information on morpheme headwords, exceptions and long vowel positions, morpheme category codes, etc., Korean, English, An apparatus for interpreting Korean documents having a phonetic dictionary, such as a special symbol, and a rest period table, the apparatus comprising: a preprocessing unit for separating input documents into word units; A morpheme analysis unit for identifying a morpheme, which is a unit of spacing, using the Korean interpretation dictionary and a connection information table with the Korean document output from the preprocessor unit; A syntax interpreter for analyzing what syntax element a word, which is the smallest unit of a sentence component, is written using a morpheme output from the morpheme interpreter; And a language code generator for generating a language code using the pronunciation dictionary and the rest length table of the syntax elements output from the syntax interpreter.

Hereinafter, the present invention will be described with reference to the accompanying drawings.

2 is a block diagram according to an embodiment of an apparatus for interpreting Korean documents according to the present invention.

The block diagram shown in FIG. 2 includes a Korean interpretation dictionary 14, which includes information on morpheme headwords, exception phonetics and long vowel positions, morpheme category codes, morpheme connection information table 16, Korean, The phonetic dictionary 22, the rest period table 24, the preprocessor 10 for separating the input document into word units, and the Korean document in the word unit output from the preprocessor 10 Using the Korean interpretation dictionary (14), the morpheme connection information table (16), the morpheme analysis unit (12) for identifying the morphemes as units of spacing, and the morphemes output from the morpheme analysis unit (12), Syntax parser 18 for analyzing what syntax element is used as the smallest word, and syntax element output from parser 18 using pronunciation dictionary 22 and rest table 24 Language code generation unit 20 for generating a language code by Achieved. Here, the input document is assumed to be a Korean document consisting of Hangul, Hanja codes, and ASCII codes.

FIG. 3 is a diagram showing an example of preprocessing in the preprocessor 10 of FIG. 2, and FIG. 4 is a diagram showing an example of syntax analysis in the syntax interpreter 18 of FIG. 2, and FIG. FIG. 6 is a diagram showing an example of reading a number in the language code generation unit 20. FIG. 6 is a diagram showing an example of a rest length and its meaning in the rest table 24 of FIG. 2. FIG. It is a table which shows an example of a rhyme pattern in the language code generation part 20. FIG.

8 is a chart showing the performance evaluation of the Korean document interpretation apparatus according to the present invention.

Next, the operation of the present invention will be described in detail with reference to FIGS. 2 to 8.

In FIG. 2, the Korean interpretation dictionary 14 necessary for the interpretation of the document has information such as the morpheme headword, the exception phoneme and the long vowel position, the morpheme category code, and the analysis dictionary information when the morpheme analysis unit 12 analyzes the morpheme. The connection information uses the longest matching method.

The preprocessing unit 10 is for separating Korean words into word units by input, where word word refers to a basic unit in which spacing is performed. Preprocessing includes processing such as Hangul / Hanja conversion, word separation, and sentence separation.

To convert Hanja code to Hangul code, we use the Hanja / Hangul conversion table that contains the first number of Hanja codes and the number of consecutive words. The word division refers to the division of sentences into word units, which are morphological interpretation units, and are classified into Korean (K), English (E), numbers (N), and special symbols (S). Sentence separation is the division of documents into sentence units, and the clues that separate sentences see periods (.), Question marks (?), And exclamation marks (!) As end of sentences. Blanks are also placed at the end of a sentence, as there is usually a space after the title or heading.

An example of pretreatment results for entering the National Education Charter as a Korean document is shown in Figure 3.

The morpheme analysis unit 14 analyzes a sentence and reconstructs it in a reading unit. In particular, the Korean morpheme analysis reveals morphemes that are components of a word that is a unit of spacing. In order to explain the morpheme analysis unit 14, the Korean morpheme classification, the dictionary structure of the analysis, the morpheme combination search and the longest match, which are the centers of the morpheme analysis, will be described using the access information table.

First, the morpheme classification can be defined as the smallest unit of words that combines a certain voice with a certain meaning, that is, a minimum significant unit. It was classified as Table 1 below.

TABLE 1

Next, the syntax analysis unit 18 will be described.

Humans can read long sentences properly because they can analyze the syntax of a sentence. In syntax analysis, the syntax structure is analyzed based on which syntax element, the smallest unit of sentence component, is the result of morphological analysis.

The syntax analysis method is classified in the order of syntax elements, single sentence, other interpretation, and independent word processing. An example of syntax analysis is shown in FIG.

1) detailed classification of syntax elements;

The word is the smallest unit of sentence components, and the seven words are classified into 19 according to their use in the sentence. This is shown in Table 2 below.

TABLE 2

2) single sentence

Korean sentences are divided into single and double sentences. A single sentence is a sentence in which the relationship between the subject and the predicate is once, and the double sentence is a sentence in which the single sentences are connected to each other or in multiple layers. A single sentence is to separate the double sentences in which the single sentences are connected to each other.

3) Case Study

In the case interpretation, the surface shock is determined for each word. The surface shock of each word is determined by the category of the last morpheme. For example, in the case of the word "we," the morpheme category of the last morpheme "은" is commonly used as a general survey.

4) standalone processing

Independent words are given rhymes without direct relation to the main or subcomponents and are components that affect the whole sentence. Interjections, sentence adverbs, words ending in a thorough investigation, and letters indicating the order are considered to be independent words.

The language code generation unit 20 generates phonological symbols for the syntax elements obtained in the syntax analysis. To create phonological symbols, phonological symbols are generated according to the rules for Korean, English, numbers and special symbols. This is described as follows.

1) Reading Korean

Hangeul reading is a problem of pronunciation due to the difference between the written and spoken symbols. That is, the phonemes that make up a word are affected by their interrelationships and are converted into sounds suitable for human speech. Phonological rules arising from the relationship between phonemes are enacted in Korean standard phoneme.

Phonological fluctuations are often handled regularly, but there is a possibility of random fluctuations of about 6-7%. In the case of the rule variation, the phonetic symbols are generated by processing the phonetic conversion table and the standard phonetic rules. In the case of irregularities, the phonetic symbols are searched in the Korean phonetic dictionary. In particular, heuristic aproach was used when the words of the same spelling were pronounced differently.

2) Reading English

In the case of English, the English words existing in the dictionary are read according to the registered pronunciation, otherwise each spelling is read one by one.

3) Reading numbers

Numbers appear in sentences like ‘number + unitary dependency noun’, ‘number, number’, ‘number, ..., number’, and other idiomatic expressions such as ‘8.15’ are used. An example of reading in each case is shown in FIG.

In the language code generation unit 20, the rhythm symbol generation generates the rest length and intonation pattern according to the connection between the syntax elements. This will be described sequentially.

1) Determine rest

In order to convey the meaning of a sentence well, read it properly.

This corresponds to the shim length determination. In the present invention, the break length was determined in six steps as shown in FIG.

The process of determining the rest length is as follows.

First, the rest length between syntax elements is determined using syntactic coupling between syntax elements. At this stage, if the rest is 5, 4, 3, the next break is a tubular word with more than 12 vowels, and the front and back units have more than 3 vowels, the rest is adjusted to 2. Also, if the read unit is a noun with 12 or more vowels and three or more vowels before and after it, adjust the pause length to 1.

2) Rhythm symbol generation

The rhyme pattern is defined according to the voiced number of the speech unit and its relationship. Based on this decision rule, a rhythm symbol generation table is prepared and processed in word units. FIG. 7 is based on the rhyme statistics of Korean voices. The number of voiced voices is up to four.

In order to evaluate the performance of the algorithm of the Korean document interpretation method according to the present invention, the accuracy of each step was measured using the 'National Education Charter' and thesis as input documents. This result is shown in FIG. The unit of measure is word, and the ratio of the number of failed words to the total number of words is recorded.

In FIG. 8, the success rates of morphological analysis and syntax analysis for 172 sentences and 2280 words were 97.7% and 95.0%, respectively. The main reason for the error is that one word is analyzed differently, ie ambiguity. In order to reduce such ambiguity, it is desirable to add an interpretation method using contextual information.

The success rate of phonological symbol generation is 99.8%. The reason for the failure is due to the reading of numbers in native or native language. To eliminate this failure, additional numeric reading and heuristic information must be added.

Rhythm symbols, that is, the success rate of rest period and rhyme pattern generation, were 93.1% and 94.0%, respectively. The criterion is to classify the sentence read by the person into the rest and rhyme pattern defined in the previous chapter, and then compare it with the result generated by the device. Inaccurate rests have a devastating effect on the naturalness of speech, so heuristics on human speech practice must be added to improve accuracy.

The Korean document interpretation method according to the present invention can be applied to an automatic translation system, an educational OA device, and the like, and can realize an interactive mode system operation.

As described above, the method and apparatus for interpreting Korean documents according to the present invention has an advantage in that phonology, rhyme, and rest processing that dominates the naturalness of machine sounds can be accurately performed.

In addition, it is possible to improve the interpretation speed of Korean documents by charting morphemes, morphemes, and grammatical interpretations of rhymes, rhymes, and breaks.

In addition, in the case of rest length, the unit from 1 second to 100 ms is classified according to sentence syntax elements, thereby improving the naturalness.

Claims (8)

A Korean document interpretation method for converting a Korean document into a voice, the method comprising: a preprocessing step of dividing an input Korean document into word units; A morpheme analysis step of analyzing a morpheme of a word, which is a spacing unit, using a Korean interpretation dictionary and a connection information table for each separated word; A syntax parsing step for analyzing what syntax element, the smallest unit of sentence component, is written using the analyzed morpheme; And a language code generation process of generating a phonological sign and a rhyme symbol by using a pronunciation dictionary and a rest period table, which are the criteria for synthesizing speech, for the syntax elements obtained by the syntax analysis. The language code generating step includes: Generating a phonological symbol for the syntax element using the phonetic dictionary for at least Korean, English, numbers, and special symbols, and determining a rest length having a plurality of levels using syntactic coupling between syntax elements And a rhythm symbol including a rest length and an accent pattern between the syntax elements using a length table. The method of claim 1, wherein the preprocessing step includes Hangul / Hanja conversion, word separation, and sentence separation processing. The method of claim 1, wherein the morpheme analysis step uses 60 morphemes based on the large-class parts of speech. The method of claim 1, wherein the syntactic analysis step comprises: a first step of classifying syntax elements according to usage of a word in a sentence; A second step of forming the sentence in a single sentence; A third step of deciding a surface shock by interpreting the word according to the category of the last morpheme; And a fourth step for treating interjective words, sentence adverbs, and ending words and words indicating the order as independent words. According to claim 1, wherein the rest length is determined by the length of the rest 5, 4, 3 according to the degree of syntactic coupling between the syntax elements, the reading unit is a tubular word having more than 12 vowels and three units before and after If there are more than one vowel, the rest length is determined to be 2, and the reading unit is a noun having 12 or more vowels, and if the front and back have three or more vowels, the rest length is determined to be 1. 6. The method of claim 5, wherein the sentence length is 5, the length of a single sentence is 4, the length of a rest is 3, the rest for breath is 2, the rest for meaning 1 is 1, and no rest is 0. Korean document interpretation method characterized in that the setting. Korean interpretation dictionary containing information on morphological headings, exceptions and long vowels, morpheme category codes, morpheme access information table, pronunciation dictionary of Korean, English, numbers, special symbols, and rest table And a Korean document interpreter for converting a Korean document into a voice, the apparatus comprising: a preprocessor for separating the input Korean document by word units; A morpheme analysis unit for analyzing a morpheme of a word, which is a spacing unit, using the Korean dictionary and access information table for each word; A syntax parser for analyzing which syntax element, the smallest unit of sentence component, is written using the analyzed morpheme; And a language code generation unit for generating a phonological sign and a rhyme symbol using a phonetic dictionary and a rest period table which are the criteria for synthesizing speech for the syntax elements obtained by the syntax analysis. The rest length table for generating phonological symbols for the syntax elements by using the pronunciation dictionary for English, numbers, and special symbols, and determining the rest length having a plurality of levels by using syntactic coupling between syntax elements. Korean document interpretation apparatus for generating a rhyme symbol including the rest length and intonation pattern between the syntax elements. 8. The method according to claim 7, wherein the morpheme analysis unit parses the word output from the preprocessing unit, inputs the analyzed morpheme into the Korean interpretation dictionary, obtains dictionary information, and inputs the morpheme pair into the connection information table. An apparatus for interpreting Korean documents, characterized in that it obtains status information and analyzes morphemes.