JPH11259094A - Regular speech synthesis device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To make it possible for a general user to simply correct prosodic errors by providing this device with a function correcting prosodic information added to character strings selected by user according to his corrected contents. SOLUTION: This device controls (1001) a whole system by programs stored in a disk 10, displays a mail (1002), calculates parameters presenting features of a speech from a character string inputted from a character string input device 3, and transfers them to a memory in a main storage device 2 (1003). Moreover, this device selects and creates prosodic parameters from a word dictionary with prosodic information, and creates a prosodic pattern of a sentence (1004) by summing up those prosodic parameters and those of the phrase components obtained from the number of the phrases. Further, speech synthesis is executed from the parameters outputted from a synthesis parameter calculation part 2 and the prosodic parameter created by a prosodic parameter creating program 1004, and processing for transmitting (1005), etc., the speech waveform to the memory is executed.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a ruled speech synthesizer capable of modifying a prosody of a speech synthesized from a character string into a prosody that does not give a user a sense of incongruity or a preferred prosody.

[0002]

2. Description of the Related Art In recent years, multimedia products such as personal computers and portable information devices have become remarkably widespread, and the market is expanding to ordinary households. As is the case with audio media, speech synthesis / recognition technology has been incorporated into portable information devices and car navigation systems, not to mention use in personal computers. In particular, speech synthesis is used for reading out e-mails on personal computers and portable information devices, and is used for guidance on destinations in car navigation, utterance of place names, and the like.

[0003]

In a mail, a sentence similar to a spoken word is frequently used, and in a car navigation, a proper noun such as a personal name or a place name is frequently used. As you can see in the spoken words, "It's not a good idea" and "I ate it."
There are many sentences that can be uniquely interpreted only when they are spoken and perceive the prosody.

[0004] For example, in the case of "not a good idea", it can be interpreted as "exclamation" or "denial". In addition, proper nouns are not covered in Japanese language dictionaries or accent dictionaries, and there are a huge number of proper nouns. As for the prosody, for example, as seen in Sakata and Maeda,
The back kanji of the two kanji that make up the name is the same "Ta"
Even if the front kanji is a kanji with the same accent type, these prosody will be a type 1 accent in the case of "Sakata" and a type 0 accent in the case of "Maeda". There is an event.

[0005] If the prosody of such a sentence or proper noun is incorrect, it will hinder the understanding of the utterance content. Therefore, it is necessary for the speech synthesizer to accurately represent the above prosody. However, in the current speech synthesizing apparatus, when the above-mentioned sentence, proper noun, or the like is given as text, the synthesis processing unit can only read out the text according to the prosody defined as the default value, giving the user a sense of discomfort. further,
It has been difficult for a general user to modify the above-mentioned prosody and register it as a correct or favorite prosody.

SUMMARY OF THE INVENTION An object of the present invention is to allow a general user to easily correct an error in a prosody and to generate a prosody whose meaning cannot be determined when judging the meaning of a sentence by the prosody. Reduce prosody errors,
It is an object of the present invention to provide means capable of synthesizing a sound that is comfortable for a user to listen to.

[0007]

According to one aspect of the present invention, there is provided a character string input unit for inputting a character string, and a synthesizing unit for calculating a parameter representing a feature of a voice from the character string. A parameter calculating unit, a voice synthesizing unit for synthesizing voice from the parameter, a voice output unit for outputting voice synthesized by the voice synthesizing unit, a voice input unit for inputting voice by a user, and an A / D In a rule speech synthesizer comprising a speech recognition processing unit for converting and further extracting speech parameters such as pitches by speech analysis, and an information storage device for storing the character strings, prosodic parameters, synthesized speech, etc. A ruled speech synthesizer having a function of correcting the prosody information given to a given character string in accordance with the content of a user's correction is provided.

[0008] Also, there is provided a rule speech synthesis apparatus having a function of generating a prosody different from the above all prosody pattern candidates for a plurality of prosody pattern candidates output from the speech synthesis unit. When the user selects the prosody pattern candidate, the speech synthesis unit synthesizes a synthesized speech using the selected prosody pattern candidate, and has a function of outputting a synthesized speech from the speech output unit. An apparatus is provided.

[0009] Further, a function of analyzing the utterance content inputted by the user to the voice input unit in the voice recognition processing unit and performing voice synthesis in the voice synthesis unit using pitch information obtained as a result of the analysis. Is provided.

The pitch of each extracted frame is defined as
A ruled speech synthesizer having a function of shifting from a user's utterance band to a synthesized sound source band by performing conversion by a certain function is provided.

[0011]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment in which the present invention is applied to a system will be described below with reference to the drawings. This system is a system that handles regular speech synthesis, and is a system that synthesizes a given text as speech. An e-mail reading system will be described as a target example for describing the present invention.

FIG. 1 is a block diagram showing the configuration of the present invention. 1 is an information processing device that performs processing in accordance with the activated program processing, 2 is a main storage device that stores data and the like, 3 is a character string represented by a keyboard, which is input and transferred to a memory in the main storage device 2 Character string input device for
Is a position input device for inputting position information on a screen, typified by a mouse, 5 is a voice input device, typified by a microphone, for a user to input voice, 6 is a display, typified by a display. Information display device for presentation, 7 is a screen control device for controlling character information, image information, etc., which are information displayed on the screen of 6, 6 is an audio output control device for controlling the volume of output audio and the like, 9 Reference numeral denotes a sound output device for outputting sound or the like, represented by a speaker, and reference numeral 10 denotes a disk device.

The disk 10 stores, in addition to an execution program for performing the processing of the present invention, a program for performing basic control of a computer system, data to be processed, a dictionary referred to during processing, a sound source database, and the like. You. That is, 1001 is a system program for controlling the whole system, 1002 is a mail display program, 1003 is a parameter that calculates the characteristics of the voice from the character string input from the character string input device 3 and is transferred to the memory in the main storage device 2 A synthesis parameter calculation program 1004 selects and generates a prosody parameter from a word dictionary with prosody information described later, and further sums the generated prosody parameter and the prosody parameter of a phrase component obtained from the number of phrases to obtain a sentence. 1005 is a prosody pattern generation program for generating a prosody pattern of the above, performs voice synthesis based on the parameters output from the synthesis parameter calculation unit 2 and the prosody parameters generated by the prosody pattern generation program 1004, and stores the voice waveform in the memory. Speech synthesis program to be transferred, 10 Reference numeral 06 denotes a prosody modification program for constructing a function and a user interface for modifying the prosody of the voice output from the audio output device 9, and 1007 outputs the voice waveform transferred to the memory by the voice synthesis program 1005 to the voice output device. Sound output program, 1
008 represents the voice input from the microphone 5 as A
A voice input program 1009 for / D conversion extracts a pitch parameter representing the pitch of the sound from the voice converted into digital voice by the voice input program 1008, and stores a voice recognition program 101 in the memory.
Reference numeral 0 denotes a word dictionary with prosody information in which word information and prosody which may be given as prosody to be added to the word are described.
Is a sound source database storing sound source parameters which are one of the voice synthesis parameters used in the voice synthesis program 1005. Here, FIG. 1 shows that programs and data are stored on a single disk 10, but they may be stored on a plurality of disk devices (not shown).

The execution program and the data to be accessed are read into the main storage device 2 as needed, and the information processing device 1 performs data processing based on the present invention.

First, a system program 1001 is started. Next, the user activates the mail display program 1002. As a result, the mail is displayed on the screen of the information display device. The screen is, for example, a screen as shown in FIG. Reference numeral 200 denotes a mail menu display button for selecting a mail to be read out, 201 denotes a place where the mail is displayed, 202 denotes a button for starting reading out, and 203 denotes a button for pressing the prosody.

Here, the sentence "Mr. Sakata ate it" is explained. The character string displayed on the screen is stored in a memory on the main storage device 2. Next, a synthesis parameter calculation program is started, a character string is read from the memory, and the character string is divided into phonemes. A method of dividing a character string into phonemes is described in, for example, Miyazaki et al.'S method (“Language processing method for Japanese sentence speech output” Transactions of Information Processing Society of Japan, Vol. 27, No. 11, pp. 1053-1).
061, 1986). Of course, the above calculation method is an example, and a method of dividing another phoneme may be used.
In this way, the character string “It seems that Mr. Sakata ate it” is “s / a / k / a / t / a / s”.
/ A / n / g / a, s / o / r / e / w / o /, t / a
/ B / e / t / a / s / o / u / d / a. And the phoneme divided data is stored in the memory. Here, “s”, “a”, etc. are symbols indicating phonemes.

Of course, the above phoneme symbol data is an example,
Other phoneme symbol representations may be used. The unit to be obtained is not limited to a phoneme unit, but may be a unit obtained by dividing a phoneme into two or a syllable. Next, the phoneme divided data divided into phonemes is read out from the memory, the duration of each phoneme is calculated, and the duration data is transferred to the memory. The method of calculating the duration of each phoneme is described, for example, by the method of Sakasaka et al. (“Phonological time length control for speech synthesis by rules”, IEICE Transactions, Vo.
l.J67-A, No.7, pp.629-636, 1984)
Use Of course, the above calculation method is an example, and another calculation method of the phoneme duration may be used.

In this manner, the phoneme division data "s / a / k / a / t / a / s / a / n / g / a, s / a /
o / r / e / w / o /, t / a / b / e / t / a / s /
o / u / d / a. , For example, duration data in milliseconds is obtained as a calculation result and stored in the memory. Of course, the above-described duration is an example, and duration data in seconds or the like may be used.

Further, the character string is morphologically analyzed and stored in a memory. Here, "Sakata / san / ga /, /
Ate / yes /. ". Next, the character string and the phoneme division data are read out from the memory, and the morphological analysis results stored in the memory and the word dictionary with prosody information 10 are read.
Check with 10.

The structure of the word dictionary with prosody information is shown in FIG.
become that way. Reference numeral 300 denotes a cell in which a word type indicating the type of each morpheme is stored. Reference numeral 301 denotes a cell that stores a word string, 302 denotes a cell that stores a word number, 303 denotes a cell that stores contents of a word string displayed in kana, 304 denotes a priority number cell that indicates a priority when prosody is selected, and 305. Is a prosody number cell for identifying the prosody, and 306 is a cell in which a parameter indicating a pitch shape for each prosody number is stored.

The result of the morphological analysis is compared with the word dictionary, and assignment is performed using the extracted prosodic parameters. As for the morpheme of the proper noun "Sakata", here, the prosody number having the priority 1 is selected. That is, prosody number 1
-4 will be selected. The generated prosody data is transferred to the memory. In addition, as for “eat” and “yes.”, The combination of 40-1 and 30-1 (40-1-3) is determined from the number in parentheses () in the prosody number of “eat”.
0-1), a combination of 40-2 and 30-2 (40-2
-30-2) is generated.

As described above, the character string "It seems that Mr. Sakata ate it."
saNga, sorewo, ta'betasouda. ", prosodic data B" sakatas
aNga, sorewo, tabetaso'uda. "and stored in memory.

Here, syllables with “′” are syllables with accent nuclei, “,” is a phrase delimiter, and “.” Is a symbol representing the end of a sentence. Of course, the above symbol is an example, and other symbols may be used. These two types of prosody express "desire" and "hearing", respectively.

Next, a prosody which is not interpreted as "desire" or "hearing" or interpreted as either is generated.
For example, the pitch values of the phonemes of the two types of prosody data A and B are added to calculate an intermediate value. The calculated data of each phoneme is stored in the memory as prosody data C. Next, the phoneme division data and the prosody data are read from the memory, the fundamental frequency is calculated, and the fundamental frequency data is transferred to the memory. The calculation method of the fundamental frequency is described in, for example, the method of Fujisaki et al.
445-453, 1971). Of course, the above calculation method is an example, and another calculation method of the fundamental frequency may be used.

In this manner, the character string "It seems that Mr. Sakata ate it." Is converted into the fundamental frequency data of Formula 1 and stored in the memory.

[0026]

## EQU1 ## (F0 [1], F0 [2],..., F0
[I],..., F0 [p]) Here, F0 [i] is a numerical value representing the fundamental frequency, for example, a value obtained in units of 10 milliseconds. Also, p
Is the number of values representing the fundamental frequency.

Of course, the above method of representing the fundamental frequency is merely an example, and if it is a method capable of finding the value of the fundamental frequency, the method of finding the fundamental frequency in time units other than a fixed time interval, and the method of finding the fundamental frequency It may be a method or the like represented by a set of model parameters. The synthesis parameters calculated in this way are transferred to the memory.

Subsequently, the voice synthesizing unit 4 reads out the synthesis parameters from the memory, performs voice synthesis by driving the synthesis filter, and generates voice data. As a speech synthesis method, for example, a method described in the document “Furui:“ Digital Speech Processing ”, p.22, Tokai University Press, 1985” is used. Of course, the above speech synthesis method is an example, and another speech synthesis method may be used. The generated audio data is stored in the memory. Next, audio data stored in the memory is output through the audio output device 5. According to the above procedure, a desired voice is synthesized.

As another example, in the case of "yellow tiger hut", it is not possible to determine whether the tiger is yellow or the hut is yellow from the character string alone. However, any interpretation can be derived from the expansion and contraction of the pose length between “yellow” and “tiger” and the fluctuation of the pitch of “tiger”. For example, when the pause length is set to 50 msec, 501-4-1 in FIG.
Can be determined as "yellow tiger", the pause length is 50 msec, and the rising pitch of "o" of "tora" is higher than the pitch of "i" which is the last phoneme of "yellow". By lowering the frequency by 70 Hz, here, by employing 501-4-2 of the prosody number 305 of "tiger" in the word dictionary, it is possible to guide the interpretation that "hut" is "yellow". Also, by making the pause length between “yellow” and “tiger” 100 msec or more,
Can be interpreted as "yellow".

Next, if the user wants to modify the prosody of "Sakata" in the example sentence "Mr. Sakata ate it.", The location to be modified, in this case, the location of "Sakata" is entered into the position input device. Select with mouse etc. which is one of
When the prosody modification button 203 displayed on the screen is pressed, a prosody modification window 205 is displayed as shown in FIG. Further, synthesized speech data is generated from the character string "Sakata" by the above-described prosody pattern candidate generation method and speech synthesis method, and the generated speech data is stored in the memory. Here, it is assumed that six types of prosody pattern candidates are generated and six types of audio data are generated.

Prosody pattern candidate A, prosody pattern candidate B, prosody pattern candidate C, prosody pattern candidate D, prosody pattern candidate E, and prosody pattern candidate F displayed in the above window
When one is selected, one of the audio data stored in the memory is output through the audio output device 5. If there is a prosody pattern candidate that the user has determined to be correct or a favorite prosody, the user selects one of the prosody pattern candidates A to D, and is displayed in the “prosody modification window” 205. When the prosody determination button is pressed, the prosody pattern candidate selected by the user is registered in the word dictionary with prosody information 1010, and the prosody priority number 3
04 is registered as “1”.

For example, when the prosody pattern candidate C is selected, the prosody priority number of the prosody number 1-3 becomes "1". As a result, next, when the character string appears in the mail, speech synthesis is performed by the above-described method using the registered prosody data.

It is also possible for the user to modify the prosody by inputting a voice. In that case, first,
A prosody modification desired portion is specified on the screen, and further, the voice input button 204 is pressed, and the user utters “Sakata” using the voice input device 5. When the utterance ends, press the above voice input button. For example, audio has a sampling frequency of 1
6 kHz, 16-bit quantization, monaural sound. Next, the voice input device 5 transfers the voice data to the memory.

Next, the speech parameters are analyzed. Here, the speech recognition program 1009 is started, the speech data is read from the memory, the fundamental frequency is analyzed, and the fundamental frequency data (F'0

[0], F'0 [1], ..., F '
0 [i],..., F′0 [k]). Further, the fundamental frequency data is set to the same region as the sound source frequency region stored in the sound source database. Assuming that the fundamental frequency center value in the sound source database is M_P,
Thereby, fundamental frequency data suitable for the sound source is calculated.

[0035]

F′0 [p] = ０F′0 [p] / p−M_P +
F'0 [p] The basic frequency data F'0 [p] is transferred to the memory. Here, F′0 [i] is the value of the fundamental frequency, and k ′
Is the number of values of the fundamental frequency data. As a method of analyzing the fundamental frequency, for example, a method of obtaining the fundamental frequency from the autocorrelation function of the residual signal of the LPC analysis is used. Of course, the above calculation method is an example, and another fundamental frequency analysis method may be used. In this way, the fundamental frequency data of the voice “Sakata” is stored in the memory. The above data is registered in the word dictionary with prosody information. In the case of registration, the prosody priority number is registered as "1", and the prosodic symbols are stored in a time series of the fundamental frequencies.

[0036]

As described above, according to the present invention, the prosody information given to the character string selected by the user can be corrected in accordance with the contents of the correction by the user. Further, for a plurality of prosody pattern candidates output from the speech synthesis unit, a prosody different from the all prosody pattern candidates can be generated. In addition, when the user selects the prosody pattern candidate, the synthesized voice using the selected prosody pattern candidate is synthesized by the voice synthesis unit, and the synthesized voice can be output from the voice output unit.

The speech recognition processing unit analyzes the utterance content input by the user to the speech input unit, and performs speech synthesis in the speech synthesis unit using pitch information obtained as a result of the analysis. Can be. Further, by converting the extracted pitch of each frame by a certain function, it is possible to shift from the utterance band of the user to the sound source band of the synthesized sound.

[Brief description of the drawings]

FIG. 1 is a system configuration diagram of an embodiment of a rule speech synthesizer to which the present invention is applied.

FIG. 2 is an explanatory diagram (1) showing an example of a screen display of the rule speech synthesizer shown in FIG. 1;

FIG. 3 is an explanatory diagram showing an example of the structure of a word dictionary with prosody information.

FIG. 4 is an explanatory view (2) showing an example of a screen display of the rule speech synthesizer shown in FIG. 1;

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... Information processing device, 2 ... Main storage device, 3 ... Character string input device, 4 ... Position information input device, 5 ... Voice input device, 6 ... Information display device, 7 ... Display control device, 8 ... Sound output control device , 9 ... sound output device, 10 ... disk, 1001 ... system program, 1002 ... mail display program,
1003 ... Synthesis parameter calculation program, 1004 ...
Prosody parameter generation program, 1005 ... speech synthesis program, 1006 ... prosody modification program, 1007 ...
Voice output program, 1008 ... voice input program,
1009: Voice recognition program, 1010: Word dictionary with prosody information, 1011: Sound source database.

Claims (5)

[Claims] A character string input unit for inputting a character string; a synthesis parameter calculation unit for calculating a parameter representing a characteristic of a voice from the character string; a voice synthesis unit for synthesizing a voice from the parameter; A voice output unit for outputting a voice synthesized by the unit, a voice input unit for performing a voice input by a user, a voice recognition processing unit for A / D converting the voice, and further extracting voice parameters such as a pitch by voice analysis. In a rule speech synthesizer comprising an information storage device for storing the above-mentioned character strings, prosodic parameters, synthesized speech, etc., the prosody information given to the character string selected by the user is corrected according to the user's correction request. Rule speech synthesizer with functions. 2. The apparatus according to claim 1, further comprising a function of generating, for a plurality of prosody pattern candidates output from the speech synthesis unit, a prosody pattern different from all of the prosody pattern candidates. apparatus. 3. The apparatus according to claim 1, wherein when a user selects the prosody pattern candidate, a synthesized speech using the selected prosody pattern candidate is synthesized by the speech synthesis unit, and the synthesized speech is output from the speech output unit. A rule speech synthesizer with a function to output synthesized speech. 4. The apparatus according to claim 1, wherein the utterance content input by the user to the voice input unit is analyzed by the voice recognition processing unit, a time series of pitch is extracted, and the pitch information is extracted. A rule speech synthesizer having a function of performing speech synthesis in the speech synthesis unit using the same. 5. The apparatus according to claim 4, further comprising a function of converting the time series of the extracted pitch by a predetermined function to shift from the utterance band of the user to the sound source band of the synthesized sound. Rule speech synthesizer.