KR0146549B1 - Korean language text acoustic translation method - Google Patents

Abstract

The present invention relates to a Korean text / speech conversion method, which improves the naturalness of synthesized speech through a language processing module and a rhyme processing module using rules, and in the synthesized speech generating module, improves the clarity of the synthesized speech using a TD-PSOLA synthesizer. In order to provide a voice conversion method, a first step of classifying the synthesis unit by analyzing the phonological structure form and the constraints of phoneme connection of Korean; A second step of easily accessing the synthesis unit in phoneme units and creating a synthesis unit database in a structure for processing the duration of the phoneme and controlling the pitch in real time; A third step of importing data required for each segment of a syllable into phoneme and half-syllable morphemes in the synthesis unit database; And after performing preprocessing on text sentences and performing word analysis, parsing process, letter / phonic conversion, applying proper rhyme rules according to sentence structure, and using phonetic symbols and rhyme information Including the fourth step by searching the synthesized units, it is possible to increase the fluency of the naturalness of the synthesized sound, to easily implement, and to greatly improve the intelligibility of the synthesized sound.

Description

Korean text / to-speech method

1 is a block diagram of hardware to which the present invention is applied

2 is a block diagram of a synthesis unit database used in the present invention

3 is a combination flow chart for each type of CDU used in the present invention.

4 is a flow chart according to the present invention

5 is a flowchart of a parsing process according to the present invention.

6 is a flow chart of the rhyme processing process according to the present invention

7 is a flowchart of word, syllable and phoneme duration calculation according to the present invention.

* Explanation of symbols for main parts of the drawings

11: character input device 12: central processing unit

13 synthesis unit database 14 D / A converter

21: Structure of the dbdic file in the compound unit DB

22: Structure of the dbptch file in the compound unit DB

23: Structure of the dbsp file in the compound unit DB

The present invention improves the naturalness of the synthesized sound through a language processing module and a rhyme processing module using a rule, and in the generation of the synthesized sound, Korean text / speech conversion using the time domain pitch synchronous overlap and add (TD-PSOLA) synthesizer to increase the intelligibility of the synthesized sound. (text-to-speech conversion: hereinafter referred to as TTS).

The function of speech synthesis is that a computer provides various types of information to a human who is a user as a voice, and the user may output existing text data or text information provided from a conversation partner by voice using speech synthesis. Of course, in order to provide a high quality speech synthesis service to the user, it is necessary to be fluent in the intelligibility and naturalness of synthesized speech, to have a vocal speed and appropriate semantic emphasis, and to be easily implemented in terms of hardware and software.

Synthesis units currently in use include sentences, words, syllable combinations, or smaller units, such as syllables, phonemes, and phonemes. The selection of the synthesis unit, the creation of the synthesis unit DB, and the combination method between the synthesis units are important factors that directly affect the sound quality and tone of the synthesized sound.

Rhymes come in three forms: pitch height, intensity, and length. The change in pitch of the note contains accent, the intensity of the note means semantic stress, and the length of note contains distinctive duration difference and rhyme boundary information according to articulation point, articulation method, and articulation combination. Therefore, it is essential to formulate rhyme implementation rules by finding the types and meanings of distinguishable patterns of actual rhyme patterns, their meanings, and the relationship between rhyme patterns according to the form, syntax structure, and context of text sentences. Therefore, an analysis method of a text sentence, a rhyme implementation model, and a rhyme pattern implementation method are important source technologies for determining the intelligibility and naturalness of the synthesized sound.

As a synthesizer, analysis / synthesis such as Linear Predictive Coding (LPC), Line Spectral Pairs (LSP), and Formant, and time domain processing synthesizers such as TD-PSOLA are being studied. These synthesizers are closely related to the intelligibility and tone of the synthesized sound. The selection of the synthesizer is determined in consideration of the naturalness, fluency, and complexity.

However, there are many difficulties in producing natural synthesized sounds that are close to humans, and there are still few commercially available products. Especially, the rule synthesis method for converting unlimited vocabulary into voices that are close to humans is a method of selecting and combining synthetic units for clarity. However, there are still insufficient technologies such as a rhythm realization method for realizing nature and a synthesis method for synthesizing sound generation. Therefore, a new technology is required.

According to the present invention, a combination of phonemes, half-syllables, and syllables is used as a synthesis unit, and the composition unit DB is written with 1,200 CDUs (context dependent units), which includes a minimum number of Korean phonological environments. In order to implement the rhyme, first, the method of composing unit is composed according to the CDU writing principle. The Korean text / enhancing method improves the naturalness and fluency of the synthesized sound by developing the control method of the length and the emphasis process, and the TD-PSOLA synthesizer, which is one of the efficient waveform editing methods, is easy to implement and greatly improves the clarity of the synthesized sound. The purpose is to provide a voice conversion method.

The present invention to achieve the above object, Character input means for receiving a Korean character; Central control means for receiving each Korean character from the character input means and controlling each component according to the algorithm of the present invention; A synthesis unit database, which is stored in a storage device as a CDU synthesis unit DB used for a synthesis algorithm, and transmits necessary data to the central processing unit; And a digital / analog converting means for converting synthesized digital data into an analog signal and outputting the result to an external device, the method comprising: classifying a synthesized unit by analyzing a phonological structure form of a Korean phone and a constraint of a phoneme connection; First step; A second step of creating a synthesis unit database in a structure for easily accessing the synthesis unit in phoneme units, and processing the duration of the phoneme and the pitch control in real time; A third step of importing data required for each segment of the syllable into phoneme and half-syllable morphemes in the synthesis unit database; And the language processing module performs word processing after performing preprocessing on the input text sentence, performs a parsing process, and then performs a character / phonic conversion. The rhyme processing module receives the processing result of the language processing module to construct a sentence. Applying a suitable rhyme rule according to the above, and the synthesized sound generating module includes a fourth step of receiving the processing results of the language processing module and the rhyme processing module and retrieving the phonetic symbols and the rhyme information from the synthesis unit DB to synthesize the synthesis units. It features.

Hereinafter, with reference to the accompanying drawings will be described an embodiment according to the present invention;

1 is a block diagram of hardware to which the present invention is applied, where 11 is a character input device, 12 is a central processing unit, 13 is a synthesis unit database, and 14 is a D / A conversion device.

Looking at the operation, the character input device 11 is responsible for receiving the Korean characters that can be expressed in the KS5601 complete type and the two-byte combination type to pass to the central processing unit (12). The CPU 12 receives Korean characters from the character input device 11 and controls each component according to the algorithm of the present invention. The synthesis unit database 13 is stored in the storage device as the CDU synthesis unit DB used for the synthesis algorithm, and is responsible for transferring necessary data to the CPU 12. The D / A converter 14 is a device that converts synthesized digital data into an analog signal and outputs it to the outside.

Table 1 below is a classification table for each type of CDU used in the present invention. In order to synthesize speech, each syllable is classified with reference to the preceding syllable and the following syllable.

The synthesis unit for high quality Korean TTS system analyzes the phonological and rhyme environment of Korean and selects the synthesis unit necessary for synthesis, and determines the detailed unit to be used according to the environment within the syllables. Segmentation and notation are used to form a synthesis unit according to the change of phonetic environment. A total of 1204 synthesis units were prepared by analyzing the phonological structure form and the constraints of phoneme connection. The generated CDU is used to fetch data from the synthesis unit database and is used to combine the synthesis units.

2 is a block diagram of a composition unit database used in the present invention, which is a structure for easily accessing a synthesis unit in phoneme units and processing a change in duration of a phoneme and a pitch control in real time. It is.

The synthesis unit DB includes a synthesis unit address data file 21, a pitch mark file 22, and an audio data file 23. The voice data file 23 stores PCM (Pulse Code Modulation) data of the synthesis unit in order.

The pitch mark file 22 stores the positions and pitch values of pitch marks, which are voice signals stored in the voice data file 23, in the form of samples.

The synthesis unit address data file 21 is a synthesis unit number for each synthesis unit, the start / end point in the voice data file 23, the number of segments, the start point in the pitch mark file 22 of each segment, Pitch mark number information is stored.

Using these database structures, the process of recalling the speech signal of the pitch section with pitch value n from the start point of 'A' is as follows.

First, the information is read from the composition unit address data file 21 according to the DB access number nl corresponding to 'A'. Since the vowel 'k' is the second segment in the nl CDU, the starting point information n2 in the pitch mark file 22 corresponding to the second segment and the number of pitch marks n3 of the segment are read. The start point n4 and the sample number information n5 in the audio data file 23 of each pitch mark are read in order of finding the starting point using the starting point information n2 in the pitch mark file 22, and it is checked whether the number of samples and the pitch value n match. do. If there is a match, n5 samples are read from the starting point n4 in the audio data file 23.

3 is a combination flow chart for each type of CDU used in the present invention.

Each syllable in the sentence is divided into five units: initial C1, first vowel V1, second vowel V2, final C2, and C3 to bring the appropriate data from the synthesis unit DB into phonemes and half-syllable morphemes. Data selection in the synthesis unit DB is determined according to the conditions of all the Vp, the final Cp of the preceding syllable, the initial C1, the vowel V, the final C2 of the target syllable, the initial Cn of the subsequent syllable, and the vowel Vn. The type of coupling is as follows:

1. C1 type

When the first syllable of a word: C1 = CV

When Cp = C1 = 'ㄹ': C1 = e1LV

Others: C1 = eCV

2. V1 type

When there is a primitive: V1 = C1

When the first syllable of a word begins with a vowel

-Single syllable or double vowel without support: V1 = V

-When the finality is 'ㅇ' in double vowels: V1 = V

Others: V1 = -1 (determined in the next step)

When there is a final syllable in the preceding syllable: V1 = eCV

When the preceding syllable ends with a short vowel: V1 = VV

When the preceding syllable ends with the 'j' series collection: V1 = ejV

When the preceding syllable ends with the 'w' series collection: V1 = ewV

3. V2 type

When there is no bell

-At the end of a word: V2 = V1

When there is a consonant in subsequent syllables: V2 = VCe

-When subsequent syllables are short vowels: V2 = VV

-When the next syllable is 'j' collection: V2 = Vje

-When subsequent syllables are 'w' collections: V2 = Vwe

When there is a bell

-At end of word: V2 = VC

-When closed: V2 = VC

-When Cp = ㄹ, C1 = ㄹ: V2 = VL1e

-When subsequent syllables begin with a vowel: V2: VCe

When there is a meteor and a subsequent syllable: V2 = VCDa

4. C2 type

When there is finality: C2 = V2

5. C3 type

When there is a shooting star and a subsequent syllable: C3 = aCCw

6. Finally, when V1 = -1: V1 = V2

4 is a flow chart according to the present invention, the function of each module is as follows.

The language processing module first performs a preprocessing process of interpreting the input text sentence 41 into Korean for an abbreviation, a sentence symbol, and a specific term (42). In the next process, a word analysis is performed to estimate and assign grammatical functions to each word of an input sentence using a group of about 60 groups in a lexical dictionary, a survey of registered Korean words, a utilization-type ending, an adverb, and a conjunction (43). In operation 44, a parsing process of estimating a syntax structure of an input sentence using Korean grammar is performed. Subsequently, an exception pronunciation dictionary is searched for and registered words are processed according to the exception pronunciation dictionary, and words not registered in the exception pronunciation dictionary are changed as they are input sentences according to the word dictionary search and Korean pronunciation rules (45).

The rhyme processing module receives the processing result of the language processing module and applies the appropriate rhyme rules according to the sentence structure to generate information related to the naturalness and fluency of the speed, intonation, and reading of the synthesized sound (46).

In the synthesized sound generation module, the phonetic symbols and rhyme information obtained through the above processing are retrieved from the synthesis unit DB, and the synthesized units are adjusted, processed, and combined using the TD-PSOLA method (47). Finally, the synthesized sound is generated and output to the user as a voice (48).

5 is a flowchart of a parsing process according to the present invention.

The purpose of syntactic analysis using functional words is to separate the phrases sequentially based on spaces from the input sentences and to assign grammatical properties to each phrase using the morpheme dictionary (51). Attribute definitions are shown in Table 2 below. The estimated syntax structure information is used to generate control information for processing the rhyme of the synthesized sound. In the present invention, the linguistic analysis elements of sentences are set based on the grammatical information defined in the morpheme dictionary, and the rhyme characteristics of the speech are classified into sentence, clause, phrase, and word form (52, 53, 54, 55). .

-There are written sentence (positive / negative), question (with or without question, inversion) and admiration.

The classifications of clauses include position (front / middle / back of the statement), equality relations, and mathematical relations (part-of-speech dependence of the word being modified).

The phrases are classified into grammatical cases (housewives / predicates), types of phrases (noun phrases, verb phrases), positions within sentences, and relative positional relationships with specific phrases.

-Analysis elements of the word are defined as the combination of morphemes and functional words.

6 is a flowchart of a rhyme processing process according to the present invention, using the result of parsing generated in the parsing process, calculating the duration control of FIG. To adjust the speed and pitch of the synthesized sound.

The duration calculation is calculated in the order of the duration of the word, the duration of the syllable, and the duration of the phoneme (56, 57, 58). Pitch control rules are written according to the basic pattern of pitch according to the sentence pattern, the degree of change (slow / rapid) related to the syntax structure, the analysis of the starting point and influence of the change, functional classification and analysis (64, 65, 66).

The fundamental frequency is calculated by first generating basic patterns in the order of sentences, clauses, and phrases, and then performing local calculations considering the phonological environment in words.

-Create a basic retrograde pattern using syntax information, calculate the degree of deviation and elevation in the supersonic system, and use the part-of-speech information to obtain the relative peak size.

Phonological information is used to create lively accents of local peaks and valleys according to the articulation location and articulation method.

Table 2 is an attribute categorization table used to assign grammatical functions (attributes) to each word by using a survey, its usage forms, auxiliary words, adverbs, and conjunctions.

7 is a flowchart of calculating words, syllables, and phoneme durations according to the present invention, where syllables become longer at the boundary of phrases and phrases, and duration changes when the next phrase is shortened within a word or phrase. First, the word duration WDdur is calculated (71).

Where RFdur is the average duration of a syllable, a is a proportional constant, and j is a syllable within a word.

Next, the duration SYLdur of each syllable in the word is calculated (72).

Then, the duration stretching of the syllable before the boundary point of the sentence, clause, and phrase is adjusted (73). Then, the initial expansion rate PRCNTO of each syllable is obtained, which is then used as an initial value of the calculation of the expansion rate of each phoneme according to the phonetic environment. (74).

Here, INHdur _i represents the intrinsic duration of the phonemes constituting the syllable.

Thereafter, the expansion rate PRCNT calculation of the phoneme duration according to the phonological environment is sequentially applied to each phoneme, and is calculated using the duration change rate PRcnt _i assigned to each rule (75).

Finally, the phoneme duration PHONdur is calculated using the obtained rate of change of duration of the phoneme, the inherent duration of the phoneme INHdur, and the minimum duration MINdur (76).

Table 3 is an accent generation model classification table. There are four basic pitch contours: plain text, question, exclamation, and phrase / phrase. There are four types of ending pitch models: the end of the plain sentence, the end of the interrogation, the end of the exclamation, and the end of the phrase / phrase. In addition, there are two models of specific syllables: local pitch characteristics of specific syllables and local pitch characteristics of specific syllable endings.

Here, t denotes a time normalized in units of sentences, phrases, and words.

As described above, the present invention uses a combination of phonemes, syllables, and syllables as a synthesis unit, and the composition unit DB is written with 1,200 CDUs (context dependent units), which includes a minimum number of Korean phonological environments. Combination method between composition units is prepared according to the CDU writing principle, and in order to implement the rhyme, first, the linguistic analysis element (attribute) of Korean language is defined. By developing the control method of length and emphasis processing, it is possible to improve the naturalness and fluency of synthesized sound, and to use TD-PSOLA synthesizer, which is one of the efficient waveform editing methods, which is easy to implement and greatly improves the clarity of synthesized sound. have.

Claims (8)

Character input means (11) for receiving Korean characters; A central control means (12) for receiving each Korean character from the character input means (11) and controlling each component according to the algorithm of the present invention; A synthesis unit database (13) which is stored in a storage device as a CDU synthesis unit DB used for a synthesis algorithm, and transmits necessary data to the central processing means (12); And a digital / analog converting means (14) for converting the synthesized digital data into an analog signal and outputting the result to the outside. A first step of classifying; A second step of creating a synthesis unit database in a structure for easily accessing the synthesis unit in phoneme units, and processing the duration of the phoneme and the pitch control in real time; A third step of bringing the database required for each segment of the syllable into the phoneme and half-syllable morpheme in the synthesis unit database; And the language processing module performs word processing after performing preprocessing on the input text sentence, performs a parsing process, and then performs a character / phonic conversion. The rhyme processing module receives the processing result of the language processing module to construct a sentence. According to a suitable rhyme rule, the synthesized sound generating module receives the processing results of the language processing module and the rhythm processing module, and searches for a phonetic symbol and rhyme information in a synthesis unit DB to synthesize synthesis units (41 to 48). Korean text / voice conversion method comprising a). The method of claim 1, wherein the classification synthesis unit is 1204 in the first step. 2. The synthesis unit database (DB) of claim 1, further comprising: a voice data file (23) which sequentially stores Pulse Code Modulation (PCM) data of the synthesis unit; A pitch mark file (22) for storing positions of pitch marks, which are voice signals stored in the voice data file (23), and respective pitch values in the form of samples; And for each synthesis unit, a synthesis unit number, a start point / end point in the voice data file 23, a number of segments, a start point in the pitch mark file 22 of each segment, and a number of pitch marks. And a compound unit address data file (21). The method of claim 1, wherein the third step comprises separating each syllable into five units: initial C1, first vowel V1, second vowel V2, final C2, and C3 to separate data suitable for each phoneme from the synthesis unit DB. Korean text / to-speech method, characterized in that imported to the stem. The method of claim 4, wherein the data selection in the synthesis unit DB is based on the conditions of the vowel Vp of the preceding syllable, the final Cp, the initial C1 of the target syllable, the vowel V, the final C2, the initial Cn of the subsequent syllable, and the vowel Vn. Type C1 is C1 = CV for the first syllable of the word, C1 = e1LV when Cp = C1 = 'ㄹ', C1 = eCV for guitars, and V1 = C1 for initials with V1, and starts with vowels If the first syllable is the first syllable, the single syllable or unsupported double vowel is V1 = V, the final vowel for the double vowel is V1 = V, the other is V1 = -1 (determined in the next step), and the final syllable is the final syllable. V1 = eCV, V1 = VV when the preceding syllable ends with a short vowel, V1 = ejV when the preceding syllable ends with the 'j' series, and V1 = ewV when the preceding syllable ends with the 'w' series Type V2, there is no finality, and at the end of the word V2 = V1, there is no finality and there is no initialism in subsequent syllables V2 = VV when V2 = VCe and no final and subsequent syllables are short vowels V2 = VV when there is no longitudinal and subsequent syllables 'j' series Is V2 = Vwe, and there is a finality, V2 = VC at the end of the word, and V2 = VC when there is a finality and closure, and there is a finality, V2 = VL1e when Cp = ㄹ, C1 = d. And V2: VCe when the next syllable starts with a vowel, V2 = VCDa when there is a lasting star, and a consonant with the last syllable, V2 = VCDa. C3 = aCCw when there is a consonant of subsequent syllables, and finally V1 = V2 when V1 = -1. According to claim 1, The fourth step (41 to 48), the language processing module, after performing a pre-processing process for first interpreting the input text sentence in the abbreviation, sentence symbol, a specific term in Korean group in the lexical dictionary The sentence analysis is performed by estimating and assigning grammatical functions to each word of the input sentence by using the classification, registered Korean survey, conjugation endings, adverbs, and conjunctions, and then estimating the syntax structure of the input sentence using Korean grammar. Characters that perform the parsing process, search the exception pronunciation dictionary, process the registered words according to the exception pronunciation dictionary, and change the input sentence to phoneme according to the word dictionary search and Korean pronunciation rules not registered in the exception pronunciation dictionary. A fifth step 41 to 45 for performing a phonological conversion process; The rhyme processing module receives a processing result of the language processing module and generates information related to the naturalness and fluency, such as fastness, intonation, jumping, and the like, by applying an appropriate rhyme rule according to the sentence structure; And a synthesized sound generation module that retrieves the phonetic symbols and rhyme information obtained through the above process from the synthesis unit DB, adjusts, processes, and combines the synthesized units by a TD-PSOLA method to generate a synthesized sound and outputs it to the user as a voice. Korean text to speech conversion method comprising the steps (47,48). 7. The method of claim 6, wherein the sixth step (46) comprises: an eighth step (61 to 63) of calculating the duration in the order of the duration of the word, the duration of the syllable, and the duration of the phoneme; And a ninth step (64, 65) for creating a pitch control rule according to the degree of change (slow / abrupt) related to the basic pattern syntax structure of the pitch according to the sentence pattern, the analysis of the starting point and the effect of the change, and the functional classification and analysis. Korean text / voice conversion method comprising a. 8. The method of claim 7, wherein the eighth step (61 to 63) comprises: a tenth step (71, 72) of obtaining the duration SYLdur of each syllable in the word after obtaining the duration WDdur of the word; An eleventh step (73, 74) of obtaining an initial stretching ratio PRCNTO of each syllable after adjusting the duration stretching of the syllable before the sentence, clause, or phrase boundary point; And the corresponding rule is applied to each phoneme in sequence, and the expansion rate of phoneme duration according to phonetic environment is calculated using the duration change rate PRcnt _i assigned to each rule. And a twelfth step (75,76) of calculating the duration PHONdur of the phoneme using the intrinsic duration INHdur, the minimum duration MINdur.