JPH03273280A - Voice synthesizing system for vocal exercise - Google Patents

Abstract

PURPOSE:To provide teacher's voice appropriate for a learner by converting the rhythm information of an input voice into the rhythm information of a standard voice, synthesizing both the information and outputting the synthesized voice as the teacher's voice. CONSTITUTION:A pitch extracting/analyzing part 70 extracts the pitch of an input voice by using an input voice file analyzed by an input voice analyzing part 40 and receiving the sound element boundary of the input voice by a data collating part 50, finds out a pitch dividing position, analyzes the input voice file synchronously with the pitch, and writes the analyzed result in the 2nd synthesized wave forming file similarly to the pitch synchronizing analysis of the reference voice file. A synthesizing part 80 synthesizes a voice by using at least one of the rhythm information of reference voices and the residual characteristics and voice path characteristics of the input voice based upon the synthesized wave forming file to form a teacher's voice and the rhythm information of the teacher's voice is converted into that of the standard voice. A synthesized wave storing part 130 stoers the teacher's voice and outputs it when necessary. Thus, correct training can be attained.

Description

[Detailed Description of the Invention] (Industrial Application Field) The present invention analyzes the voice of a practitioner and extracts its features.
The present invention relates to a speech synthesis method for practicing vocalization in which synthesized speech obtained by converting the characteristics of the voice of the practitioner into the characteristics of the voice of a standard speaker is used as the teacher's voice.

(Prior Art) Conventional vocal training devices analyze the pisochi frequency and formant frequency of the voice uttered by a practitioner and display the analysis results as a screen on a monitor. The screen above! ! A training device that performs vocal training while listening to the voice is already known. For details of this vocal training device, please refer to the paper by Mr. Umezaki et al. titled “Development of vocalization/speech training device for deaf and dumb people” (March 1988, Proceedings of the Acoustical Society of Japan, 1-4-11, p. 297- 2
98) (Reference 1).

In addition to this, based on knowledge of phonetics, the voice uttered by the practitioner is analyzed for various parameters using various sensors, and the training method is determined based on the parameters.
2. Description of the Related Art A training device is known that performs vocal training by comparing each parameter of the voice uttered by a practitioner with the parameters of a teacher's voice.

For details of this device, please refer to the fS paper titled “Development of a vocal training device for people with language disabilities (6th report)” by Hiro Yamada et al.
January 1988 IEICE Technical Research Report ET
87-8 p, 25-30) (Reference 2).

Recently, vocal training devices have been developed that allow you to practice rhythm intonation, f# sound pronunciation, and word comprehension while listening to a teacher's voice, and that allow you to listen and compare the voice uttered by the practitioner with the teacher's voice. . For details, see the paper titled “Development of an English Speech Practice System 2” by Mr. Takada et al. (March 1987, IEICE Technical Report ET86-12, p. 49-5.
2) (Reference 3).

(Lesson H that the invention attempts to solve) However, the utterance lesson a! of Document 1! Since the i-device displays the analyzed input voice, that is, the intonation of only the voice uttered by the practitioner, on the display, it is not possible to check how the input voice deviates from the standard utterance, and it is difficult for the practitioner to had the disadvantage that it was difficult to tell the difference between the standard voice and one's own voice.

Furthermore, the vocal training device of Document 2 requires the practitioner to wear various sensors during vocal training, which is extremely troublesome, and the addition of the sensors distorts the way the practitioner speaks.

In sentences v and 3, the vocal training coloring is called practice of rhythm and intonation, and when the practitioner pronounces according to the pre-registered teacher's voice, the rhythm and intonation of the teacher's voice and the voice uttered by the practitioner are adjusted. Although it is displayed on the display, it is difficult to know what to correct just by looking at the display screen, and it simply indicates that the practitioner's utterance is different from the teacher's voice, which is the standard voice. The drawback was that it was difficult to connect to learning vocalizations. In addition, in this training device, the practitioner uses a W quasi-voice uttered by another speaker as a teacher's voice to compare with his/her own voice, but the teacher's voice is phonologically different from the practitioner's voice. , since not only the prosodic information but also the voice quality etc. are different, practitioners tend to pay attention to the differences in voice quality etc., and the disadvantage is that it is difficult to train the phonological and prosodic information so that it approaches the teacher's voice. Ta. Furthermore, since the speaking rate of a standard speaker is generally constant, it may not match the speaking rate of a practitioner, and the practitioner has had to change the speaking rate during language pronunciation training.

Therefore, an object of the present invention is to provide a teacher's voice that has been left to each practitioner by generating a teacher's voice with a voice quality and speaking rate close to that of the individual practitioner, and furthermore, to provide a teacher's voice that has been left to each practitioner, and to further reproduce the voice of the practitioner on a phoneme-by-phoneme basis. By comparing and displaying a comparison with the teacher's voice, it is possible to clearly identify which parts have problems, and to understand the distance between the pronunciation's voice and the teacher's voice, and to show the degree of learning achievement. To provide a speech synthesis method for vocal practice which displays the warmness in an easy-to-understand manner.

(Means for Solving the Problems) The first voice synthesis method for vocal practice according to the present invention inputs the voice uttered by a practitioner as input voice, analyzes the input voice, and analyzes temporal changes in the fundamental frequency. means for extracting prosodic information such as phoneme duration and phoneme duration; means for recording prosodic information of a standard speech that is the speech of a standard speaker that has been analyzed in advance; A correspondence relationship is determined from each piece of prosodic information, the prosodic information of the input speech is converted into prosodic information of the standard speech that has been analyzed in advance, the input speech and the standard speech are synthesized, and the synthesized speech is used by the teacher. means for outputting the teacher's voice and the prosodic information extracted from the input voice on a screen, and in the output, phoneme names and phoneme boundaries of the input voice are placed in corresponding positions of the standard voice, respectively; It is characterized by having a means for displaying.

The second voice synthesis method for vocal practice according to the present invention inputs the voice uttered by a practitioner as input voice, analyzes the input voice, and analyzes the input voice to obtain prosodic information such as temporal changes in fundamental frequency and phoneme duration length. It is a means to extract and the voice of a standard speaker that has been analyzed in advance! means for recording prosodic information of a quasi-speech; and a prosody of the standard speech in which the correspondence between phonemes of the input speech and the standard speech is determined from the prosodic information of each, and the prosodic information of the input speech is analyzed in advance. Means for converting into information, synthesizing the input voice and the standard voice, and outputting the synthesized voice as a teacher voice; and a sound obtained from the analysis result of the input voice! j! Sweet face II obtained from the analysis results of the duration and the standard voice! means for calculating the difference by comparing the continuous duration lengths, and outputting a standard speech whose phoneme duration length is expanded or contracted by the difference as the standard speech to be used in the teacher speech output means, or between the input speech and the standard speech. For each, find the relationship between the duration of the stressed vowel and the duration of other vowels,
The duration length of the stressed vowel in the standard speech is converted into the duration length of the vowel in the input speech, and for other vowels, the duration length between each vowel in the standard speech is converted to the duration length of each vowel in the input speech. and outputting the 111i quasi-speech obtained by converting the duration length between vowels as a standard speech used in the teacher speech output means.

The third utterance I according to the present invention [the learning speech synthesis method inputs the voice uttered by the practitioner as the input voice, analyzes the input voice, and analyzes the temporal changes in the fundamental frequency and the prosody such as phoneme duration length. means for extracting information; means for recording aS information of a standard speech that is the speech of a standard speaker analyzed in advance; and prosodic information on the correspondence between phonemes of the input speech and the tIA quasi-speech. The mark obtained by analyzing the prosodic information of the input speech in advance! #, a means for converting the input speech into prosodic information and synthesizing the input speech and the quasi-speech, and outputting the synthesized speech as a teacher speech, and recording a plurality of speeches having different durations in advance; , Compare the duration lengths of the input voice and the standard voice, and if the duration lengths are different, select the voice whose duration length is closest to the input voice from the plurality of voices and select the voice. The present invention is characterized by comprising: means for outputting a standard voice to the teacher voice output means.

The fourth voice synthesis method for vocal training according to the present invention inputs the voice uttered by a practitioner as input voice, analyzes the input voice, and analyzes the input voice to obtain prosodic information such as temporal changes in fundamental frequency and phoneme duration length. means for recording prosodic information of a standard voice that is the voice of a standard speaker analyzed in advance; and determining correspondence between phonemes of the input voice and the standard fvM from the respective prosodic information. , means for converting prosodic information of the input speech into prosodic information of the standard speech that has been analyzed in advance, synthesizing the input speech and the standard speech, and outputting the synthesized speech as a teacher speech; The present invention is characterized by comprising means for calculating the distance between the prosodic information of the voice and the teacher voice, converting the distance into a learning achievement level, and outputting it on a screen.

(Function) According to the present invention, a practitioner can accurately learn prosodic information, especially rhythm and intonation, of a standard speaker, and can pronounce the same. Use both visual and auditory IfI information for practice. By displaying the correspondence between phonemes between the teacher's voice and the voice uttered by the practitioner on a screen that displays visual information, it is possible to confirm where and how different the voice is. In addition, the teacher's voice uses the voice of the practitioner and a voice that has the quality of the voice of the practitioner to reduce the difference between the voice of the practitioner and the teacher's voice and emphasize the difference in prosodic information between the two voices. , it is possible to practice while grasping the image of when a practitioner Shirami accurately imitates the prosodic information of a standard voice.

For example, with regard to intonation, in order to clearly see where the intonation of a practitioner differs from that of a standard speaker, the intonation of the practitioner is superimposed on the intonation of a standard speaker on the screen, and Although the parts are different, it is visually informed to the practitioner. For practice, it displays what phoneme it is, shows the problem part more accurately to the practitioner, evaluates the sound for each phoneme, and shows the parts that are different from the teacher's sound. As the teacher's voice, a standard voice is used for the prosodic information to be practiced, and a voice synthesized using the practitioner's voice is used for other vocal tract characteristics and sound source characteristics.

The synthesized speech used as the e-teacher's voice belongs to the practitioner, except for the prosodic information that they wish to practice, so when the practitioner listens to and compares his or her own vocalization with the teacher's voice, there is less difference in voice quality, and it is easier to hear the difference between the two voices. It becomes easier to understand the difference in the prosodic information that you want to practice, making it easier to learn the target prosody.

Furthermore, in cases where the speaking speed of the trainee and the standard speaker are significantly different, the speaking speed of the standard speaker is expanded or contracted. Since the speech synthesis is performed after getting the speech rate close to that of the teacher's voice, the problem of the trainee being unable to match the speech rate of the teacher's voice and being unable to perform proper training is solved.

(Example) Next, the present invention will be explained in detail with reference to the drawings.

FIG. 1 is a block diagram showing an embodiment of a first speech synthesis method for practicing speaking according to the present invention. The standard speech pitch extraction unit 30 performs linear predictive analysis on the standard speech input through the input terminal 1 to separate it into residual characteristics and vocal tract characteristics, and further performs pitch extraction on the residual characteristics using an autocorrelation method. to extract pitch division positions and phoneme boundaries, and make all of these into a standard audio file. This method includes Iwatahiro's “speech pitch extraction device” (Patent Application No. 1983-210690).
The method of (Reference 4) is used. The data storage section 10 stores the standard speech data extracted by the standard speech pitch extraction section 30, that is, the standard speech file. When using existing data as standard audio, this data storage section 10 is extracted.

The input voice detection unit 40 performs a linear predictive analysis on the input voice, which is the voice of a practitioner inputted through the input terminal 6, and separates it into residual characteristics and vocal tract characteristics, and further performs automatic voice detection.
Find the starting and ending positions of the input audio. For own voice detection,
“An Algorithm for Determining the Endp” by Rabiner et al.
oints of 15olatedUtterr
ances (Bell 5ystTech, J,
, Vol. 54. No, 2pp297-315
．． The method of February 1975) (Reference 5) is used.

The data matching unit 50 uses DP matching to determine the distance between the analysis results of the input voice and the standard voice analyzed by the input voice detection unit 40 and the standard voice pitch extraction unit 30, respectively, and calculates the distance between the analysis results on the time axis of both voices. Take action. For this DP matching, we used the method described by Sakoe et al.'s "Similarity Evaluation of Voice Bangs Using Dynamic Programming" (August 1970, Proceedings of the 1970 National Conference of the Institute of Electronics and Communication Engineers, p. 136) (Reference 6). use Additionally, by using DP mapping, the positions of phoneme boundaries in the input voice are given to the input voice file using a standard voice file that contains phoneme names and phoneme boundary positions in advance.

The analysis section 60 converts the standard speech file into which the pitch division positions in the standard speech have been inserted by the standard speech pitch extraction section 30 into the standard speech file.
Analyze with pitch synchronization 0 Separate the analyzed standard voice file into phoneme names, phoneme boundary positions, pitch division positions, duration length, residual characteristics, vocal tract characteristics, etc. and create the first synthetic waveform creation file. Write.

The pitch extraction/analysis section 70 performs pitch extraction on the input speech using the input speech file that has been analyzed by the input speech analysis section 40 and given the phoneme boundaries in the input speech by the data matching section 50, and performs pitch division. Find the position and analyze the input audio file with pitch synchronization based on the result. This analysis result is written into the second synthetic waveform creation file in the same way as the pitch synchronization analysis of the standard audio file.

The synthesis unit 80 uses the first and second synthesis waveform creation files to generate at least one piece of prosodic information of the standard speech;
Speech synthesis is performed using the residual characteristics and vocal tract characteristics of the input speech to generate teacher speech. Furthermore, the synthesis unit 80 uses the input speech file and the standard speech file, which have information such as pitch division positions, phoneme boundaries, residual characteristics, and vocal tract characteristics, to convert the prosodic information of the target teacher's speech into the standard speech. Convert to something.

At this time, pitch control is used to input voice or mark l! The zero synthesis unit 80, which converts the time length of audio, not only generates teacher audio, but also plays back standard audio and input audio.

Each of these synthesized voices is output to the output terminal 5.
The synthesized waveform storage unit 130 stores the teacher's voice and outputs the teacher's voice when necessary.

The screen display unit 150 displays each of the prosodic information corresponding to the phoneme core and phoneme boundary positions of the input voice obtained by the data matching unit 50 and the phoneme core and phoneme boundary positions of the teacher voice obtained by the synthesis unit 80, based on the phoneme core and phoneme boundary positions of the input voice obtained by the data matching unit 50. Add the phoneme core and phoneme boundary information to the location and output it to the screen. for the information displayed on these screens.

When a signal specifying the phoneme of the part to be reproduced is inputted to the input terminal 3 from a keyboard, mouse, etc., the synthesizer 80 performs speech synthesis on the part of the sound corresponding to the signal, and the teacher's sound is output from the output terminal 5.

FIG. 2 is a block diagram showing an embodiment of the second voice synthesis method for practicing speaking according to the present invention. In FIG. 2, $a precept elements having the same numbers as those in the embodiment shown in FIG. 1 operate in the same manner as those in the embodiment shown in FIG. Compare the overall phonological duration length of the standard speech and input speech from Calculate the difference. What about this difference? #Expansion/contraction of the audio time length is performed on the residual signal. #This compression uses a pitch control method.

This method is described in Mr. Iwata's paper “Residual flFJ fR
"Study of a speech synthesis system" (October 1988)
Acoustical Society of Japan Proceedings 3-2-7 p, 183-1
84 (described in detail in text y'y>).

Another method for converting the speaking speed is to use the speaking speed converter 90.
From the input speech file and the standard speech file, which are the analysis results of the input speech and standard speech, calculate the ratio of the duration of the most strongly uttered stressed vowel to the duration of other vowels. Find each voice. Furthermore, the m-sequence interval length of the most strongly uttered stressed vowel in the standard speech is converted into the duration length of the corresponding vowel part in the input speech. In this way, the duration lengths of other vowels are expanded and contracted, and in this way, the duration lengths of the standard speech are recalculated.

FIG. 3 is a block diagram showing an embodiment of the third voice synthesis method for practicing speaking according to the present invention. In FIG. 3, components that are numbered the same as in the embodiment shown in FIGS. 1 and 2 operate in the same manner as in the embodiment shown in FIGS. The speech rate comparison unit 100 compares the time length of the input voice and the standard voice. If the time lengths of the input voice and the standard voice are significantly different, a standard voice close to the time length of the input voice is retrieved from the data storage unit 20. Call, change the standard voice to the same duration as the input voice.

FIG. 4 is a block diagram showing an embodiment of the fourth voice synthesis method for practicing speaking according to the present invention. In FIG. 4, components having the same numbers as in the embodiment shown in FIGS. 1, 2 and 3 are the same as in the embodiment shown in FIGS. 1, 2 and 3. take action. The distance calculation unit 120
The distance between the prosodic parameters of the teacher's voice and the input voice is determined, and the result is compared with a predetermined evaluation value correspondence table and converted into an evaluation value (for example, 0 to 100 points).

The screen output unit 110 displays the evaluation value obtained by the distance calculation unit 120 and the temporal change in the prosodic parameters of the teacher voice and the input voice. When displaying evaluation values, display them in a visually easy-to-understand manner using numerical values, graphs, diagrams, etc. In addition, we have provided a column to give advice on how to pronounce the voice, for example, if you are practicing the duration length, it will display messages such as ``Please pronounce the a sound shorter.'' ``Please make the 1 sound a little longer.'' However, in the case of pitch, the common parts in the sentences for displaying 0, such as "Please pronounce the sound a high, please pronounce the sound u sound low," are displayed in advance on the screen output section. 110, insert the necessary phoneme core into the phoneme part, and display it on the screen.Therefore, by correlating the past evaluation value with the current evaluation value, it is possible for the practitioner to grasp the progress level. can.

The evaluation value storage unit 140 stores past evaluation values,
Displaying past practice results on the screen 1
These evaluation results are displayed on the screen via the screen output terminal 20.

<Effects of the Invention> As explained in detail above, according to the present invention, the quality of the teacher's voice used for vocal practice is close to that of the practitioner, and the speaking speed is similar to that of the practitioner. Since they are almost the same, it is possible to provide a voice synthesis method for vocal practice that generates a teacher's voice that is easy for a beginner to understand and suitable for each individual practitioner. In addition, in the screen display of the speech synthesis method for vocal practice of the present invention, by displaying phoneme boundaries in both the teacher's voice and the input voice, it becomes easier to understand which sounds have problems, and when practicing that sound. Since the method is also displayed, it has a great effect on the practitioner's progress.

Ru.

1... Input terminal, 3... Input terminal, 5... Output terminal, 6... Input terminal, 1o... Data storage section, 2o
... Screen output terminal, 30... Standard speech pitch extraction section, 4o... Input speech detection section, 50... Data collation section, 6o... Analysis section, 70... Pitch extraction/analysis section ,
8o... Occlusion area.

90... Speech rate conversion section, 100... Speech rate comparison section, 110... Screen output section, 120... Distance calculation section, 130... Synthetic waveform storage section, 140... Evaluation value storage Section, 150... Screen display section.

[Brief explanation of drawings]

Claims (4)

[Claims] (1) Input the voice uttered by the practitioner as input voice,
means for analyzing the input speech to extract prosodic information such as temporal changes in fundamental frequency and phoneme duration length; and means for recording prosodic information of a standard speech that is the speech of a standard speaker that has been analyzed in advance. , find the correspondence between phonemes of the input speech and the standard speech from their respective prosodic information, convert the prosodic information of the input speech into prosodic information of the standard speech that has been analyzed in advance, and compare the input speech and the standard speech. means for synthesizing the synthesized speech with a standard speech and outputting the synthesized speech as a teacher speech; outputting the prosodic information extracted from the teacher speech and the input speech on a screen; and means for displaying phoneme boundaries at corresponding positions of the standard speech. (2) Input the voice uttered by the practitioner as input voice,
means for analyzing the input speech to extract prosodic information such as temporal changes in fundamental frequency and phoneme duration length; and means for recording prosodic information of a standard speech that is the speech of a standard speaker that has been analyzed in advance. , find the correspondence between phonemes of the input speech and the standard speech from their respective prosodic information, convert the prosodic information of the input speech into prosodic information of the standard speech that has been analyzed in advance, and compare the input speech and the standard speech. means for synthesizing a standard speech and outputting the synthesized speech as a teacher speech, and comparing a phoneme duration length obtained from an analysis result of the input speech and a phoneme duration length obtained from an analysis result of the standard speech. means for calculating the difference and outputting a standard speech whose phoneme duration length is expanded or contracted by the difference as a standard speech to be used in the teacher speech output means; The relationship between the duration length and the duration length of other vowels is determined, and the duration length of the stressed vowel in the standard speech is converted to the duration length of the vowel in the input speech. Converting the duration length between each vowel in the standard speech into the duration length between each vowel in the input speech, and converting the duration length between the vowels into the standard speech used in the teacher speech output means. A speech synthesis method for practicing vocalization, comprising: means for outputting. (3) Input the voice uttered by the practitioner as input voice,
means for analyzing the input speech to extract prosodic information such as temporal changes in fundamental frequency and phoneme duration length; and means for recording prosodic information of a standard speech that is the speech of a standard speaker that has been analyzed in advance. , find the correspondence between phonemes of the input speech and the standard speech from their respective prosodic information, convert the prosodic information of the input speech into prosodic information of the standard speech that has been analyzed in advance, and compare the input speech and the standard speech. a means for synthesizing a standard voice and outputting the synthesized voice as a teacher voice; recording a plurality of voices having different durations in advance; and comparing the durations of the input voice and the standard voice; When the duration lengths are different, means for selecting a voice whose duration length is closest to the input voice from among the plurality of voices and outputting the voice to the teacher voice output means as a standard voice; A speech synthesis method for vocal practice, which is characterized by: (4) Input the voice uttered by the practitioner as input voice,
means for analyzing the input speech to extract prosodic information such as temporal changes in fundamental frequency and phoneme duration length; and means for recording prosodic information of a standard speech that is the speech of a standard speaker that has been analyzed in advance. , find the correspondence between the phonemes of the input speech and the standard speech from their respective prosodic information, convert the prosodic information of the input speech into prosodic information of the standard speech that has been analyzed in advance, and compare the input speech and the standard speech. a means for synthesizing the synthesized speech with a voice and outputting the synthesized voice as a teacher's voice; calculating a distance between prosodic information between the input voice and the teacher's voice, converting the distance into a learning achievement level and outputting it on a screen; 1. A speech synthesis method for vocal practice, comprising: