KR0150366B1 - Graphic user interface - Google Patents

Abstract

The present invention relates to a method of generating intonation rules using a graphical user interface, and more particularly, to provide a method of generating intonation rules using a graphical user interface in order to generate a more natural voice in speech synthesis using an unlimited speech synthesizer. A first step (11 to 15) of speech synthesis of the test data read from the storage means (2) by the speech synthesis means (5) and calculating the average intonation, average left error, and left error of each syllable; A second step 16 of comparing the magnitude of the left-hand error of the syllable and the average left-hand error; And a third step (17 to 17) of correcting the intonation data and creating the intonation rule after storing the analyzed information of the sentence according to the comparison result of the second step 16, or immediately modifying the intonation data and creating the intonation rule. It is easy to improve and regularize the naturalness of the final synthesized sound, including 23), and can extract rhyme patterns in a range where the naturalness is not greatly impaired, and it is a rhythm rule experiment and writing tool for speech linguists without programming ability. There is an effect that can be useful.

Description

How to create intonation rule using graphical user interface (GUI)

1 is a block diagram of a system to which the present invention is applied.

2 is a flow chart in accordance with the present invention.

* Explanation of symbols for main parts of the drawings

1: central processing unit 2: storage unit

3: output device 4: input device

5: speech synthesizer 6: D / A converter

7: speaker

The present invention relates to a method of generating intonation rules using a graphical user interface, and more particularly, to a method of generating intonation rules using a graphical user interface in order to generate a more natural voice in speech synthesis using an unlimited speech synthesizer.

Speech synthesis is meaningful because computers provide various types of information in speech to human users. The user may output the existing text data or the text information provided from the conversation partner using the voice synthesizer. Synthesized sound is high in clarity and naturalness, and can be provided with high quality speech synthesis service only when the user is fluent in speech speed and proper semantic emphasis.

To this end, a rule is required to properly assign accents, which are the rhyme elements of language, to text data, and the creation of such intonation rules requires a lot of time to regularize the phonetic linguistic knowledge based on the speech data. In addition, the research developer's knowledge about the phonetic linguistic knowledge required for generating the intonation rule is insufficient, and there is a limit to generating accurate intonation rule.

Thus, until now, there has been a problem that it is difficult to generate intonation rules based on phonetic linguistic knowledge.

The present invention devised to solve the above problem rate is to create an intonation rule in which a linguist can easily create an accent rule by analyzing and outputting a rhyme phenomenon occurring in a language using a graphical user interface in a speech synthesizer using an unlimited speech synthesizer. The purpose is to provide a method.

The present invention to achieve the above object, the central processing means for controlling each component; Storage means for storing voice waveforms and text data and intonation data; Input / output means for input / output with an operator; Speech synthesizing means for synthesizing the output data of the central processing means into speech; D / A conversion means for converting the digital signal of the speech synthesis means into an analog signal; And an inflection rule generating method applied to a speech conversion system having a speaker for outputting speech data of said D / A conversion means, wherein said speech synthesis means synthesizes the test data read from said storage means in said speech synthesizing means. Calculating a mean intonation, an average left error, and a left error; A second step of comparing the magnitude of the left-hand error of the syllable and the average left-hand error; And a third step of modifying the intonation data and creating an intonation rule or storing the intonation data immediately after storing the analyzed information of the sentence according to the comparison result of the second step. do.

Hereinafter, with reference to the accompanying drawings will be described an embodiment according to the present invention;

1 is a configuration diagram of a system to which the present invention is applied, which includes a central processing unit 1 for controlling each component, a storage device 2 for storing voice waveforms, text data, and intonation data, and a screen and printer for the operator. An output device 3 for outputting through; an input device 4 for receiving input data from an operator; a voice synthesis device 5 for synthesizing output data of the central processing unit 1 into voice; A D / A converter 6 for converting the digital signal of (5) into an analog signal and a speaker 7 for outputting audio data of the D / A converter 6 are provided.

The present invention is carried out by being loaded into the central processing unit 1, and the storage unit 2 stores the intonation data formed by the research developer according to general rules, and the speech linguist centrally processes the intonation data of the storage unit 2. After processing by the apparatus 1, the speech synthesis apparatus 5 is synthesized into speech, and then digital / analog (D / A) is converted and output through the speaker 7. If the outputted voice requires correction, the correction is made through the input device 4. At this time, the speech and original speech data synthesized through the speech synthesizing apparatus 5 are output to the output apparatus 3. When this process is repeated and completed, the modified data is stored, and the accent rule is generated by recording the factors affecting the naturalness shown in the above process.

2 is an overall flowchart according to the present invention.

When the present invention is loaded, voice waveforms, text data, and intonation data are read from the storage device 2 and output to the screen (11). In this case, the voice waveform may show a change in the voice waveform with respect to the modified intonation part by showing the voice portion corresponding to the character. In addition, the text data outputs a Hangul code corresponding to a sentence and an international phonetic symbol (IPA), and the accent data and the text data are output on a line so that the desired part can be corrected when the intonation is corrected.

The speech synthesizing apparatus 5 processes the read data according to a rule, generates intonation data, stores it in the storage device 2, and outputs the intonation data of the synthesized sound to the output device 3 (12).

The average of the intonation data in the section corresponding to each syllable of the intonation data of the original sound stored in the storage device 2 and the intonation data output through the speech synthesis device 5 is calculated (13). The average value of each syllable intonation is reference data for comparing the difference between the original and the synthetic intonation data.

Where N is the number of pitches in the syllable, Denotes a pitch.

Subsequently, the difference between the accent data of the original sound and the accent data generated by the rule (voice synthesizer) is obtained as the mean square error (MSE: Mean Square Error) between the mean intonation syllable of the original sound and the mean intonation of the syllable of the synthesized sound. (14). The mean left error is the difference between the accent data of the original voice and the synthesized voice, and represents the average difference rate for the entire sentence.

Where M is the number of pitches in the sentence, Is the average pitch of the original sound, Indicates the average pitch of the synthesized sound.

Then, the left error between the average intonation syllable of the original syllable and the average intonation of the syllable by the syllable is calculated (15). Determined by comparison (16).

If the left-handed error for each syllable is larger than the average left-handed error, the sentence analysis result (information affecting naturalness) is stored in the storage device 2 for a large difference between the accent curve of the original sound and the intonation curve of the synthesized sound ( 17). At this time, the information stored in the storage device 2 includes a kind of sentence (a written sentence, an interrogative sentence, an exclamation sentence) and a kind of a clause (a position, an equality relation, a formula relation), and a kind of a phrase (noun phrase). , Verb phrases), the association of words with functional words, the position of words in clauses and phrases, the number of syllables that make up a word, and the location of syllables in words. These information are factors that affect the naturalness of the speech synthesis apparatus 5 and are analyzed and output by the speech synthesis apparatus 5.

After each syllable's left-handed error is smaller than the average left-handed error or after storing the sentence analysis result, the intonation data of the synthesized speech is corrected using the graphical user interface (GUI) (18). To modify, simply use the mouse to create the accent curve on the screen.

When the modification is completed, the speech synthesis apparatus 5 is driven (19), and the driven speech synthesis apparatus 5 synthesizes the speech using the modified intonation data and performs digital / analog conversion (20).

From the synthesized sound generated by the modified accent data, a storage message for determining whether to store the modified accent curve in the storage device 2 is received and judged (21) and when the storage message is corrected, the modified accent data is stored ( 22) A new intonation rule is created and terminated using the modified intonation data and sentence-analyzed information (elements affecting naturalness) (23). The creation of intonation rules uses training algorithms, such as the Hidden Markov Model, neural networks, or a table-look-up approach.

If the stored message is a command requesting modification again, the process is repeated from step 18 of modifying the intonation data.

The accent curve of the original sound is a reference data for generating an accent curve of a form that can be easily ordered, that is, the accent curve can be modeled by a formula, or a model that can be modeled with a minimum accent pattern.

The present invention as described above is easy to improve and regularize the naturalness of the final synthesized sound, it is possible to extract a rhyme pattern of a range that does not significantly impair the naturalness, the intonation data and sentence analysis information stored the intonation rule for speech synthesis Can be written by training and can be useful as a rhythm rule experiment and writing tool for speech linguists without programming skills.

Claims (3)

Central processing means (1) for controlling each component; Storage means (2) for storing speech waveforms and text data and intonation data; Input / output means (3,4) for inputting / outputting the operator; Speech synthesizing means (5) for synthesizing the output data of said central processing means (1) into speech; D / A conversion means (6) for converting the digital signal of the speech synthesis means (5) into an analog signal; And a loudspeaker 7 for outputting speech data of the D / A converting means 6, wherein the intonation rule generating method is applied to the speech converting system, wherein the test data read from the storing means 2 is read. A first step (11 to 15) for speech synthesis by the speech synthesis means 5 and calculating the syllable-based average intonation, the average left error, and the left error; A second step (16) of comparing a left error of the syllable and a mean left error; And a third step (17 to 17) of correcting the intonation data and creating an intonation rule or storing the intonation data immediately after storing the analyzed information of the sentence according to the comparison result of the second step (16). 23) A method of generating intonation rule, characterized in that it comprises a. 2. The method of claim 1, wherein the first steps (11 to 15) comprise: a fourth step (11) of reading voice waveforms, text data and intonation data from the storage means (2); The speech synthesizing means 5 processes the read data according to a rule to generate intonation data and stores the intonation data in the storage means 2 and then outputs the intonation data of the synthesized sound to the input / output means 3 and 4. A fifth step 12; A sixth step (13) of calculating an average of the intonation data for each syllable of the intonation data of the original sound stored in the storage means (2) and the intonation data output through the speech synthesis means (5); Step 7 (14), which calculates a mean square error (MSE: mean square error) between the average intonation syllable of the original syllable and the average intonation syllable of the syllable, and the average intonation syllable of the syllable of the original syllable And an eighth step (15) of calculating a left error with the accent. According to claim 1, wherein the third step (17 to 23), if the left error of each syllable is larger than the average left error, sentence analysis results for the portion where the difference between the accent curve of the original sound and the accent curve of the synthesized sound is large. A fourth step (17) of storing in the electric field means (2); After each syllable's left-handed error is smaller than the average left-handed error or after storing sentence analysis results, the intonation data of the synthesized speech is corrected using a graphical user interface (GUI), and the speech synthesis means 5 corrects the modified intonation data. A fifth step (18 to 20) of performing digital / analog conversion after synthesizing the voice using the synthesized voice; A sixth step 21 of determining the type of the stored message; And storing the modified intonation data according to the determination result of the sixth step 21 and newly creating the intonation rule using the modified intonation data and sentence-analyzed information, or repeatedly performing the fifth intonation (18 to 20). Accent rule generation method comprising the seventh step (22, 23).