JP2006276660A - Method of indicating features of intonation variation by modification of tone and computer program thereof - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a method capable of measuring intonation variations underlying a speech under natural conditions. <P>SOLUTION: The method of measuring intonation variations includes a step 80 of preparing a prescribed set of citation values 56 of fundamental frequency (F0) targets for each of lexical tones obtained from isolated syllables 50 of a loudspeaker. The set of citation values of F0 targets characterizes a corresponding lexical tone. The method further includes a step 82 of extracting F0 target values for each syllable in sample speech data 52 of the loudspeaker and a step of calculating a prescribed first parameter 58 that measures the change from a citation value of a lexical tone of a syllable to an F0 target value, for each of the F0 target values of each syllable in the sample speech data 52. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to spoken language processing, and more particularly, to measuring a change in intonation in spoken language to synthesize a desired intonation voice.

  Chinese basic frequency (F0) outline (intonation in a general sense) is the actual tone (excluding the tone of a word) that represents multiple tones of a word and contrasts between plain and questioned sentences Is to clarify. Traditionally called the first, second, third and fourth voices (tones 1 to 4), each of which has its own distinctive characteristics that distinguish it from the others, and this There is a neutral tone (tone 0) without such a remarkable feature.

  The type of tone is an element that directly composes Chinese syllables. For example, “ma” has the following five different meanings depending on the type of tone.

このために重要な問題が生じる。テキスト−トゥ−スピーチ（ｔｅｘｔ−ｔｏ−ｓｐｅｅｃｈ：ＴＴＳ）合成においてイントネーションを合成する際に、語の声調と実際のイントネーションとの相互作用をどのように明らかにするか、ということである。これはＴＴＳを会話システムに適用する際に非常に重要である。会話システムでは例えば、疑問、メッセージの確認、及び感情が、人間によって、通常は音節のイントネーション（すなわち語の声調）と区別され、さらに通常の平叙文とも区別されるイントネーションのパターンで実現される［非特許文献１参照］。

This creates an important problem. It is how to clarify the interaction between the tone of the word and the actual intonation when synthesizing intonation in text-to-speech (TTS) synthesis. This is very important when applying TTS to conversational systems. In a conversation system, for example, questions, message confirmations, and emotions are realized by humans in an intonation pattern that is usually distinguished from syllable intonations (ie, tone of words) and also from normal plain text [ Non-patent document 1].

A possible solution to this is probably a Fujisaki model that decomposes the F0 contour into accent and phrase components [see Non-Patent Document 2]. Intonation changes may be distributed to both accent and phrase components, but the number of parameters in the model is limited. In order to deal with the effects of actual intonation on the tone of words, linguists generally focus on changes in pitch range at the level of syllables [3] or phrases [4].
G. Cochance Key and C.I. See, “Prosodic Modeling Using Soft Template” Speech Communication, Vol. 39, pp. 311-352, 2003 (G. Kochanski and C. Shih, "Prosody modeling with soft templates," Speech Communication, Vol. 39, pp. 3l1-352, 2003.) H. Fujisaki and K. Hirose, “Analysis of Speech Basic Frequency Contours in Japanese Declarations”, Journal of the Acoustical Society of Japan, Vol. 233-242, 1984 (H. Fujisaki and K. Hirose, "Analysis of voice fundamental frequency contours for declarative sentences of Japanese," J. Acoust. Soc. Japan, Vol.5, No.4, pp. 233-242 , 1984.) J. et al. Shen, “Pitch range of tone and intonation in Beijing dialect”, research report on experimental phonetics, T. Phosphorus and L. J. et al. One, Peking University Press, pp. 73-130, 1985 (Chinese) (J. Shen, "Pitch range of tone and intonation in Beijing dialect," in Working papers in experimental phonetics, ed. By T. Lin and LJ Wang, Beijing Univ. Press, pp. 73-130, 1985. (in Chinese)) Z. Wu, “New method of intonation analysis for Mandarin Chinese: Frequency transposition processing of phrase contours in sentences”, analysis, perception and processing of spoken language, Fan et al., Pp. 255-268, 1996 (Z. Wu, "A new method of intonation analysis for standard Chinese: frequency transposition processing of phrasal contours in a sentence," Analysis, perception and processing of spoken language, ed. By G. Fant, et al, pp. 255-268, 1996.) Y. R. Chao, Chinese spoken grammar. Berkeley, University of California Press, 1968 (YR Chao, A grammar of spoken Chinese. Berkeley, University of California Press, 1968.) P. Kratochiville, Mandarin intonation, intonation system, in 20 languages, Hurst and A.M. D. Christo, Cambridge University Press, 417-431, 1998 (P. Kratochvil, Intonation in Beijing Chinese, in Intonation systems, a survey of twenty languages, ed. By D. Hirst and AD Cristo, Cambridge Uni. Press, 417 -431, 1998.) J. et al. D. and K.K. Hirose, “Experimental Evaluation of Functional Modeling of Basic Frequency Contours in Mandarin Chinese” ISCSLP2000, Beijing, pp. 319-322, 2000 (J. Ni and K. Hirose, "Experimental evaluation of a functional modeling of fundamental frequency contours of standard Chinese sentences," ISCSLP2000, Beijing, pp. 319-322, 2000.) J. et al. D. and H.H. Kawai, “Pitch Range Fixes Chinese Tone and Intonation Patterns” Phonetic Prosody 2004, Nara pp. 95-98, 2004 (J. Ni and H. Kawai, "Pitch targets anchor Chinese tone and intonation patterns," Speech Prosody 2004, Nara, pp. 95-98, 2004.) J. et al. D. and H.H. Kawai, “Extraction of Tone Features through Analysis-Based Pattern Matching by Parametric Modeling and Synthesis,” ICASSP 2003, pp. 72-75, 2003 (J. Ni and H. Kawai, "Tone feature extraction through parametric modeling and analysis-by-synthesis-based pattern matching," ICASSP2003, pp. 72-75, 2003) J. et al. D. and H.H. Kawai, “skeletonization of Chinese basic frequency contour by function model and its evaluation”, TAL 2004, pp. 151-154, Beijing, 2004 (J. Ni and H. Kawai, "Skeletonising Chinese fundamental frequency contours with a functional model and its evaluation," TAL2004, pp. 151-154, Beijing, 2004.) J. et al. To Heart, R.D. Coriner and C.I. Cohen, perceptual study of intonation: an experimental, phonetic approach to phonetic melodies, Cambridge University Press, 1990. phonetic approach to speech melody, Cambridge University Press, 1990.)

  The limitation of such an approach is that the measured pitch range includes more or less word tone effects. Furthermore, if the tone of a word being uttered is all tone 1, the pitch range cannot be calculated. This is because tone 1 has the characteristics of the high sound level and there is no low sound characteristic that can be used as a reference for estimating the pitch range.

  The present invention addresses the problem of measuring this intonation change from a different direction, including dependence on the tone type, including tone changes within the reference values from the separated individual syllables, and the F0 contour. Try to avoid the difficulties that arise when breaking down the undulations.

  Accordingly, one object of the present invention is to provide a method capable of measuring changes in intonation underlying speech under natural conditions.

  Another object of the present invention is to provide a method capable of measuring a change in intonation based on speech without being affected by the tone of a word.

  According to the first aspect of the present invention, a method of characterizing intonation types by tone transformation is a method of characterizing a reference value for a fundamental frequency (F0) target for each tone of a word obtained from an individual syllable of a speaker. Preparing a predetermined set, the reference set of reference values for the F0 target characterizing the tone of the corresponding word, and extracting the F0 target value for each syllable in the sample speech data of the speaker; For each of the F0 target values of each syllable in the sample voice data, calculating a predetermined first parameter representing a degree of change from the reference value related to the tone of the word of the syllable to the F0 target value is further included. .

  Preferably, the preparing step includes recording a plurality of individual syllables by the speaker for each of the tone of the word, and extracting an F0 target value of each recorded syllable according to the tone of each word. And, for each word tone, averaging each F0 target value of each F0 target characterizing the tone of the word.

  More preferably, in this method, the predetermined first parameter is changed to the predetermined second parameter so that the distribution of the predetermined second parameter is balanced on both sides of the predetermined reference value of the predetermined second parameter. The method further includes normalizing.

  A second aspect of the present invention relates to a computer program that, when executed on a computer, causes the computer to perform any of the steps described above.

A. Method Overview A. 1 Deformation Deformation constructed with a functional model, which is dealt with in Non-Patent Document 7, makes it possible to map F0 contours in various vocal ranges into a normalized space called λ space-time. Here, f0 represents F0 in Hertz display, and λ represents F0 at λ (normalized frequency). The deformation between f0 and λ is expressed by the following equation.

ここでＡ（λ，ζ）は単純な共振システム内での振幅−周波数応答を表す。

Where A (λ, ζ) represents the amplitude-frequency response in a simple resonant system.

ζは共振システムの減衰比を表す。物理的には、減衰比は共振システム中の粘性抵抗の等価物を表す。他のモデルパラメータは以下を示す。

ζ represents the damping ratio of the resonant system. Physically, the damping ratio represents the equivalent of viscous resistance in a resonant system. Other model parameters are as follows:

[f0 _b, f0 _t]: the highest and lowest frequencies of the vocal range
[λ _b , λ _t ]: The highest frequency and the lowest frequency of the voice range expressed by λ The voice range [f 0 _b , f 0 _t ] depends on the speaker. Actually, it can be measured as the frequency range of the speech of the target speaker. In most cases, λ _t and λ _b can be fixed at 1 and 2, respectively.

Given λ and ζ, f0 can be directly calculated by the above transformation. For convenience, T _f0 () represents the deformation from λ to f0 in ζ.

f0 = _Tf0 (λ, ζ) (3)
On the other hand, λ (or ζ) can also be determined by iterative processing, given f0 and ζ (or λ). _Let T _λ () represent the deformation from f0 to λ at ζ. As f0 increases, the value represented by λ decreases.

λ = T _λ (f0, ζ) (4)
Further, T _ζ () represents ζ for deformation from λ to f0.

ζ = T _ζ (λ, f0) (5)
A. 2 Tone Deformation This transformation provides a method for measuring the change from f0 ₁ to f0 ₂ in [f0 _b , f0 _t ], as shown by ζ below.

ζ = T _ζ (T _λ (f0 ₁ , ζ ₀ ), f0 ₂ ) (6)
Here, ζ ₀ is a reference value of ζ when both f0 ₁ and f0 ₂ are mapped to λ values. Preferably, ζ ₀ is fixed at 0.156.

In order to guarantee a one-to-one mapping between f0 ₁ and f0 ₂ , ζ must belong to the set of (0, 0.7], so that f0 ₁ = T _f0 (λ _i , ζ _As can be seen in FIG. 1 under the condition of ₀ ), a constraint between f0 ₁ and f0 ₂ is derived for each ζ.

λ ₂ = T _λ (T _f0 (λ ₁ , ζ ₀ ), ζ) (7)
As ζ moves away from the reference ζ ₀ (= 0.156), λ ₁ changes non-linearly and monotonously to λ ₂ , and the range sharply narrows at both ends of the region [1, 2].

In order to balance ζ on both sides of ζ ₀ , the normalized damping ratio ζ _n is defined as ζ _n ∈ [−1, 1] as follows.

この方法を拡張して、語の声調及びピッチアクセント等の、２個のＦ０ターゲットのシーケンス間の変化を測定することが可能である。ある声調の中でのすべてのＦ０ターゲットは、同じζ_０におけるλによる相対量として表される。この方法を２個の声調間の変化を測定するために用いる利点は、声調内の内部変化が見え、このため、実際の声調の変化を測定可能となることである。

This method can be extended to measure changes between sequences of two F0 targets, such as word tone and pitch accent. All F0 targets in a tone are expressed as relative quantities with λ at the same ζ ₀ . The advantage of using this method to measure the change between two tones is that internal changes in the tone can be seen, thus making it possible to measure the actual tone change.

2 to 4 show an example in which this tone modification is applied to a Mandarin tone. FIG. 2 (a) shows a four word tone (tones 1 to 4 superimposed on the same time axis as shown in box 30) repeated 6 times, and FIG. 2 (b) ζ _n = 0, which represents the tone of the reference word with no target tone change. As shown in FIG. 3B, when ζ _n changes linearly from 0 to −1 in 2 seconds, the tone sequence in FIG. 2A changes to that shown in FIG. ζ _n corresponds to the thick line in FIG. 4B, and the tone sequence in FIG. 2A changes to the thick line shown in FIG. Certainly, a phenomenon in which the pitch range becomes narrow in a very high / low range of the voice range is often seen in actual speech.

A. 3. Measurement of changes in intonation Stonal intonation is called tone. The time-F0 contour that matches the syllable is known as the tone pattern. Chao's tone theory [see Non-Patent Document 5]. The four word tones are represented as four tone patterns, and this is represented by a number of selected F0 targets as shown in FIG. Each tone is characterized by a main target [see Non-Patent Document 6]. ]. In FIG. 5, the main target is indicated by a black circle.

  The change in tone clearly indicated by the F0 contour is a change in the tone of the underlying word in a specific manner [see Non-Patent Document 6]. The F0 contour can be reliably represented by the sequence of F0 targets, and the number and type of F0 targets can be determined from the tone of the underlying word according to the tone pattern [see Non-Patent Document 8]. Therefore, the algorithm for measuring the tone change from the F0 contour using tone transformation basically includes the following steps.

  Initialization: Determine the reference value (reference value) of the F0 target for the four tone patterns according to the average F0 target measured from individual syllables by the speaker.

  Step 1: F0 target (observed value) is extracted from the F0 contour according to the tone pattern of FIG. An algorithm for estimating the F0 target from the F0 contour can be used as described in Non-Patent Documents 9 and 10, whereby a tone feature is first extracted and then converted to an F0 target.

Step 2: Create a pair (f0 _i , ^ f0 _i ) for the tone pattern. Here, f0i represents the observed value of the i-th F0 target, and ^ f0 _i (the “^” symbol in front of “f” is supposed to be written above the original f) represents the reference value. For tone 0, the reference value for this F0 target is simply the reference value for the last F0 target in the preceding tone.

Step 3: Calculate ζ _i = T _ζ (T _λ (^ f 0 _i , ζ ₀ ), f 0 _i ), and ζ _n . However, i = 1,... N (number of F0 targets). This represents the characteristics of intonation changes.

FIG. 6 shows (a) the F0 target pair used for estimating the intonation pattern characterized by ζ _n (circle), and (b) the F0 target pair for the F0 contour obtained from the corresponding speech data. The reference value (triangle) and the observed value (circle) are plotted. Line P0P4 indicates ζ _n = −1.045t + 0.686, and line P5P7 indicates ζ _n = −0.809t + 1.198.

B. DESCRIPTION OF EXAMPLE 1 Structure B. 1.1 Functional Block FIG. 7 is a block diagram showing a speech synthesis system 40 according to one embodiment of the present invention. Referring to FIG. 7, the speech synthesis system 40 includes a storage device 50 for a reference utterance of a predetermined speaker, a storage device 52 for storing a sample utterance of the speaker, and a tone of the reference utterance. An intonation extraction module 54 for extracting a reference F0 target and for each sample utterance stored in the storage device 52 for extracting a sequence of normalized damping ratios ζ _n indicative of intonation changes.

The speech synthesis system 40 further includes a storage device 56 for storing the reference F0 target of the reference utterance and a storage device 58 for storing the sequence of ζ _n . The sequence of the damping ratio ζ _n represents the characteristics of the intonation change of the sample utterance. Therefore, the user can specify a desired intonation using the sequence of ζ _n stored in the storage device 58.

  The speech synthesis system 40 further receives intonation information 60 associated with the input text 62 to be synthesized, an F0 synthesizer 64 for synthesizing F0 for each syllable in the input text 62, and the input text 62 and F0. An audio synthesizer 66 for synthesizing an audio signal according to F0 output from the synthesizer 64 is included.

The intonation extraction module 54 extracts a F0 target from each syllable of the reference utterance in the storage device 50, and stores the extracted f0 target in the storage device 56. The first target extraction module 80 stores the extracted f0 target in the storage device 56. Calculating a damping ratio ζ _n for each of the F0 targets output from the second target extraction module 82 and the second target extraction module 82 for extracting the F0 target from each of the syllables of the sample utterances, and a ζ _n calculation module 84 for outputting the sequence of ζ _n to the storage device 58.

The F0 synthesizer 64 calculates f0 _i of each syllable of the input text 62 according to the following formula using a ζ calculation module 90 that calculates ζ from the sequence of ζ _n in the intonation information, and the calculated f0 _i is voiced. And an F0 calculation module 90 for outputting to the synthesizer 66.

f0 _i = T _f0 (T _λ (f0 _i , ζ ₀ ), ζ) (9)
B. 1.2 Implementation by Computer The module shown in FIG. 7 is implemented by computer software in this embodiment. FIG. 8 shows a control structure of a computer program that realizes the first target extraction module 80. Referring to FIG. 8, the program begins at step 100 and steps 102-120 are repeated for each of tone 1 to tone 4 found in the reference utterance.

  In step 102, the variable SUM is initialized to zero.

  At step 110, steps 112-116 are repeated for all of the tone data of interest within the reference utterance. In step 114, the F0 target is extracted from the syllable speech data. The extracted F0 is added to the SUM at step 116.

  After steps 112 to 116 are repeated for all syllables of the tone of interest, step 118 determines the SUM average. At step 118, this average is stored in memory in association with the subject's tone.

  At the end of this process, the average F0 of tone 1 to tone 4 is stored in the memory.

FIG. 9 shows a control structure of a computer program for realizing the second target extraction module 82 and the ζ _n calculation module 84 shown in FIG. Referring to FIG. 9, in step 140, F0 contours are calculated for all of the sample utterances stored in storage device 52. In step 142, steps 144 to 152 are repeated for all syllables of the input text 62 (see FIG. 7).

In this repetition, first, the F0 target of the tone of the syllable being processed is extracted. Let the extracted i-th F0 target be f0 _i , 1 ≦ i ≦ N (the number of targets in speech).

In step 146, f0 _i extracted in step 144 is paired with ^ f0 _i of the syllable tone pattern. Here, f0 _i represents a reference value of f0 _i . For tone 0, the reference value for that F0 target simply takes the reference value for the last F0 target of the preceding tone.

In step 148, ζ _i is calculated according to the following equation:

ζ _i = T _ζ (T _λ (^ f 0 _i , ζ ₀ ), f 0 _i ) (10)
At step 150, normalized ζ _ni (1 ≦ i ≦ N) is calculated according to the following equation:

ステップ１５２で、結果ζ_ｎｉが記憶装置５８に記憶される（図７を参照）。

In step 152, the result ζ _ni is stored in the storage device 58 (see FIG. 7).

After repeating the above process for all syllables of the sample utterances stored in the storage device 52, the user can describe any intonation using the normalized ζ _n . Therefore, the intonation information 60 can be prepared in the form of a sequence of ζ _n .

  In this embodiment, the F0 synthesizer 64 shown in FIG. 7 is also realized by computer software. The control structure of this computer program is shown in FIG.

Referring to FIG. 10, when F0 synthesizer 64 is activated, intonation data ζ _ni in intonation information 60 is first read. Next, in step 172, steps 174 to 178 are repeated for all syllables of the input text 62. Here, ζ _ni (1 ≦ i ≦ N) is a sequence of normalized attenuation rates of intonation information 60.

In step 174, ζ _i is calculated from ζ _ni according to the inverse function of equation (11).

At step 176, the F0 target f0 _i of the i th syllable (tone) is calculated according to the following equation:

f0 _i = T _f0 (T _λ (^ f0 _i , ζ ₀ ), ζ _i ) (12)
Here, ^ f0 _i represents a reference value (F0 target) extracted from the reference utterance, and ζ ₀ represents a constant (preferably ζ ₀ is 0.156).

In step 178, f0 _i calculated in this way is stored in the memory.

Steps 174 to 178 are repeated for all syllables of the input text 62, and then the sequence of f0 _i is used as the F0 target of the tone sequence in the input text 62 in which the intonation pattern is specified by the intonation information 60 in step 180. Is output.

B. 1.3 Computer Hardware FIG. 11 shows the appearance of a computer system 330 of this embodiment that executes the above-described computer program, and FIG. 12 shows this system 330 in a block diagram.

  Referring to FIG. 11, this computer system 330 includes a computer 340 having an FD (flexible disk) drive 352 and a CD-ROM (compact disk read only memory) drive 350, a keyboard 346, a mouse 348, and a monitor 342. And a pair of speakers 372 and a microphone 370.

  Referring to FIG. 12, computer 340 further stores a CPU (central processing unit) 356, a bus 366 connected to CPU 356, FD drive 352, and CD-ROM drive 350, a hard disk 354, a boot-up program, and the like. A read only memory (ROM) 358, and a random access memory (RAM) 360 connected to the CPU 356 and storing application program instructions, system programs, data, and the like.

  Although not shown here, the computer 340 may further include a network adapter board that provides a connection to a local area network (LAN).

  A computer program for causing the computer system 330 to realize the above-described speech synthesis system is stored in the CD-ROM 362 or FD 364 inserted into the CD-ROM drive 350 or FD drive 352 and further transferred to the hard disk 354. Alternatively, the program may be transmitted to the computer 340 through a network (not shown) and stored in the hard disk 354. The program is loaded into the RAM 360 when executed. The program may be loaded directly into the RAM 360 from the CD-ROM 362, from the FD 364, or via a network.

  The program described with reference to FIGS. 8 to 10 includes a plurality of instructions for causing the computer 340 to realize the functional blocks of the speech synthesis system 40 of this embodiment. Some of the basic functions necessary to perform this method are provided by an operating system (OS) running on computer 340 or a third party program installed on computer 340. Therefore, this program does not necessarily include all functions necessary for realizing the system and method of this embodiment. This program may include only instructions that perform the above-described processing by calling an appropriate function or “tool” in a controlled manner so as to obtain a desired result. The operation of computer system 330 is well known and will not be repeated here.

B. 2 Operation The above-described speech synthesis system 40 (see FIG. 7) of this embodiment operates as follows. The operation of the speech synthesis system 40 is in three stages. That is, extraction of the F0 target from the reference utterance, calculation of ζ _n from the reference utterance, and F0 target and speech synthesis. The operation of the speech synthesis system 40 at these stages will be described below.

B. 2.1 Extraction of F0 Target from Reference Speech Referring to FIG. 7, voice data of a predetermined speaker is recorded for all of the tone 1 to tone 4 and stored in the storage device 50 as a reference utterance. For each of the tone 1 to tone 4, the first target extraction module 80 extracts the F0 target from the reference utterance. The average F0 target is stored in the storage device 56 for each of the tone 1 to tone 4.

B. 2.2 Calculation of ζ _n from reference utterance A sample utterance of the same speaker as the reference utterance is recorded and stored in the storage device 52. For each syllable of each sample utterance, the second target extraction module 82 extracts the F0 target. Then, for each of the F0 target output from the module 82, to calculate the _n zeta _n calculation module 84 is zeta, for each sample utterances to generate a sequence of zeta _n.

B. 2.3 F0 Target and Speech Synthesis The user prepares the input text 62 and related intonation information 60 that identifies the intonation that the user wants to synthesize the input text. The user can prepare intonation information by examining the sequence of ζ _n stored in the storage device 58.

When the intonation information 60 and the input text 62 are prepared, the ζ calculation module 90 calculates ζ for each syllable of the input text 62 and outputs this to the F0 calculation module 92. For example, for the i-th syllable, the ζ calculation module 90 calculates ζ _i of this syllable from ζ _ni according to the inverse function of equation (11).

The F0 calculation module 92 applies the following function to each of the syllables, ζ _i calculated in this way, ^ f0 _i stored in the storage device 56, and the constant ζ ₀ = 0.156. F0 target f0 _i is calculated.

f0 _i = T _f0 (T _λ (^ f0 _i , ζ ₀ ), ζ _i ) (13)
As a result, for the syllable in the input text 62, the F0 calculation module 92 outputs the sequence of f0 _i . This sequence is provided to the voice synthesizer 66.

Given the sequence of f0 _i from the F0 calculation module 92, the speech synthesizer 66 can synthesize the speech signal 68 of the input text 62 with the intonation specified by the intonation information 60.

C. Experimental Results Two experimental results are reported to show that the proposed method is able to account for higher levels of intonation changes in the measured F0 contour than the tone of the word. Audio samples were selected from a Chinese speech corpus and read by a specialized narrator. The voice range [f0 _b , f0 _t ] of the narrator is consistent with [100 Hz, 500 Hz], and the reference value of the voice of the word by the narrator is as shown in Table 1. Bold indicates major targets. The tone patterns corresponding to these reference values can be seen in FIG.

図１３〜図１６に示される結果は、４つの慣用の挨拶を含むイントネーション変化の分析から得られた。４つの挨拶の実際のイントネーションは音韻論的には同じであるが、語の声調のためにＦ０輪郭は大きく起伏する。計算の例として、表２は、図１３（ａ）に示されたサンプルからの観測値ｆ０_ｉ，ｉ＝１，…５、対応の参考値＾ｆ０_ｉ、及び結果として得られるパラメータζ_ｉ及びζ_ｎｉを列挙している。これらの結果は図１３（ｂ）に示される。

The results shown in FIGS. 13-16 were obtained from analysis of intonation changes including four conventional greetings. The actual intonation of the four greetings is phonologically the same, but the F0 contour is greatly undulated due to the tone of the word. As an example of the calculation, Table 2 shows the observed values f0 _i , i = 1,... 5 from the sample shown in FIG. 13 (a), the corresponding reference values ^ f0 _i , and the resulting parameters ζ _i and ζ _ni is listed. These results are shown in FIG. 13 (b).

In this example, the accent of the sentence is indicated by a decrease in ζ _n from 0.024, which is the main target of tone 2 (surface tone of first tone 3), to −0.423 of second tone 3. The sentence accents of other sentences also appear to fall consistently regardless of the type of underlying tone. The basic features shown in these four greetings are: (1) the accent of the sentence is located at the end of the utterance and the other syllable, (2) the last tone (tones 1-4) is a reference Maintaining the tone pattern (ie, ζ _n does not change). Tone 0 is considered to be a succession of tones that are the last non-tone 0. This result is consistent with the above assumptions. The phenomenon of intonation change is very similar to the so-called “hat pattern” that is usually used to describe intonation in non-tone language, as exemplified in Non-Patent Document 11, for example.

FIG. 17 shows an example of synthesizing a tone and intonation. FIG. 17A shows a reference value of the tone of the word that is the basis. FIG. 17B plots the intonation pattern according to ζ _n (t). FIG. 17 (c) shows these F0 targets (circles) and the contours (continuous lines) provided by the model with these targets. The “+” sequence indicates the measured F0 contour of the sample utterance.

  As is apparent from FIG. 17, the F0 contour provided by the model is very close to the original F0 contour.

FIG. 18 shows further results obtained by having the same speaker read several numbers. The number sequence read is neutral because it has no linguistic meaning. In order to obtain a clear result, only the ζ _n values of the main tone targets are plotted in the figure. In addition, there is no pause in these utterances. Two shapes appear in the intonation change. One is a line going down from the beginning to the end (left side). The other is a line composed of a descending portion followed by a flat portion. This descent occurs between the first two syllables. The revealed intonation changes are systematic at a high level beyond the tone of words.

  Approximately 200 Chinese samples from three speakers were analyzed. Although the actual intonation varies slightly in these samples, the results of the analysis show that this method can clearly reveal changes in intonation as indicated above.

D. CONCLUSION Embodiments of the present invention relate to a method of measuring intonation changes by excluding word tone from measured F0 contours. Intonation changes are sampled using the selected F0 targets that make up the tone pattern of the word and are characterized by a single point parameter on the time axis. The experimental results show that the proposed method is very promising for analyzing the actual Mandarin intonation buried in the F0 contour and mixed with the tone of the word. The actual intonation revealed was similar to the intonation reported in non-tonal languages. The proposed method attempts to automatically analyze the F0 contour with the tone of the underlying word, which is critical in speech synthesis, recognition and even understanding.

  The embodiment disclosed herein is merely an example, and the present invention is not limited to the above-described embodiment. The scope of the present invention is indicated by each claim in the claims after taking into account the description of the detailed description of the invention, and all modifications within the meaning and scope equivalent to the wording described therein are intended. Including.

It is a figure which shows the conditions between (lambda) ₁ , (lambda) _2, and (zeta). It is a figure which shows the example which applied tone deformation to the tone of Mandarin. It is a figure which shows another example which applied tone deformation to the tone of Mandarin. It is a figure which shows another example which applied tone deformation to the tone of Mandarin. It is a figure showing the tone of Mandarin language based on F0 target. It is the figure which plotted the reference value (triangle) and the observed value (circle) for the F0 target pair used for estimation of the intonation change in ζ _n (circle), and the original F0 contour. 1 is a block diagram of a speech synthesis system 40 according to an embodiment of the present invention. It is a figure which shows the control structure of the computer program which implement | achieves the 1st F0 target extraction module 80 with a flowchart. It is a figure which shows the control structure of the computer program which implement | achieves the 2nd target extraction module 82 and the ζ _n calculation module 84 with a flowchart. It is a figure which shows the control structure of the computer program which implement | achieves F0 synthesizer 64 with a flowchart. It is a perspective view of the computer system 330 which executes the computer program which concerns on one embodiment. 2 is a block diagram of system 330. FIG. Is a diagram showing the F0 contour of the greeting of the customary "ni3hao3" (Hello). It is a figure which shows the F0 outline of the usual greeting "zen3me0yang4a0?" It is a figure which shows F0 outline of the usual greeting "ni3mang2ma0?" It is a figure which shows the F0 outline of the usual greeting "ni3shen1ti3hao3ma0?" It is a figure which shows the example which synthesize | combines the feature of the prosody by a word, and the feature of the prosody which does not depend on a word. It is a figure which shows the change of neutral intonation in the read-out number sequence.

Explanation of symbols

40 speech synthesis system 50, 52, 56, 58 storage device 54 intonation extraction module 60 intonation information 62 input text 64 F0 synthesizer 66 speech synthesizer 68 speech signal with intonation 80 first F0 target extraction module 82 second F0 target extraction Module 84 ζ _n calculation module 90 ζ calculation module 92 F0 calculation module

Claims (4)

A method of expressing the characteristics of intonation changes by transformation of tone,
Providing a predetermined set of reference values for a fundamental frequency (F0) target for each of the speaker's tones of words obtained from individual syllables, wherein the set of reference values for the F0 target is a corresponding word And characterize the tone of
Extracting an F0 target value for each syllable in the speaker's sample voice data;
For each of the F0 target values of each syllable in the sample voice data, calculating a predetermined first parameter representing a degree of change from the reference value related to the tone of the word of the syllable to the F0 target value; A method for expressing the characteristics of changes in intonation by tone modification. The step of preparing comprises
Recording a plurality of individual syllables by the speaker for each tone of the word;
Extracting the F0 target value of each recorded syllable according to the tone of each word;
2. The method of claim 1, comprising, for each word tone, averaging each F0 target value of each F0 target characterizing the tone of the word. Normalizing the predetermined first parameter to the predetermined second parameter so that the distribution of the predetermined second parameter is balanced on both sides of the predetermined reference value of the predetermined second parameter; The method of claim 1, further comprising: A computer program that, when executed on a computer, causes the computer to perform all the steps according to any one of claims 1 to 3.

Images