JP6542823B2 - Acoustic model learning device, speech synthesizer, method thereof and program - Google Patents

Description

The present invention provides a speech synthesis apparatus that synthesizes speech using spectral envelope information and fundamental frequency (hereinafter also referred to as “F ₀ ”) information, an acoustic model learning apparatus that learns an acoustic model used in speech synthesis, and It relates to a method and program.

  There is a technology based on DNN (deep neural network) as a method of learning a speech synthesis model from speech data and generating synthesized speech (see Non-Patent Document 1). FIG. 1 is a functional block diagram of an acoustic model learning device 80 according to the prior art, and FIG. 2 is a functional block diagram of a speech synthesizer 90 according to the prior art.

Spectral envelope · F ₀ vector data creation unit 82, F ₀ data _{{f 1, f 2, ...} , f N} and the spectral envelope data _{{s 1, s 2, ...} , s N} from the spectral envelope · F Create ₀ data {x ₁ , x ₂ ,..., X _N }. However, it is assumed that the total number of learning voice data is N, and n = 1, 2,. In the figure, {f ₁ , f ₂ ,..., F _N } etc. are expressed as f _n etc. Language feature vector data generating unit 81, context data _{{t 1, t 2, ...} , t N} from the language feature vector data _{{l 1, l 2, ...} , l N} to create. Spectrum envelope generation model conversion parameter learning unit 84, the spectral envelope-F ₀ data _{{x 1, x 2, ...} , x N}, language feature vector data _{{l 1, l 2, ...} , l N} from The spectral envelope · F ₀ generation DNN is learned.

In the speech synthesizer 90, the text analysis unit 91 analyzes the text tex _o synthesized to obtain the context t _o . Language feature vector extraction section 92 extracts the language feature vector l _o from the context t _o. The spectral envelope generation unit 94 generates spectral envelope information s _o and F ₀ information f _o from the language feature quantity vector l _o using the spectral envelope and F ₀ generation DNN. Speech waveform generation unit 95, resulting spectral envelope information s _o, from F ₀ information f _o, the speech waveform generation, obtain a synthesized speech waveform z _o.

Zen et al., "Statistical parametric speech synthesis using deep neural networks", Acoustics, Speech and Signal Processing (ICASSP), 2013 IEEE International Conference on IEEE. 2013, pp. 7962-7966.

In speech uttered by humans, it is known that there is a dependency between spectral envelope information and F ₀ information. By reproducing this dependency, it is possible to improve the quality of synthetic speech.

However, in the prior art, an input language feature value vector obtained from the context, using DNN for outputting spectral envelope information and the F ₀ information, the dependency of the spectral envelope information and the F ₀ information is modeled in light Absent. Therefore, it is considered that there is room for improvement in voice quality.

The present invention explicitly models the dependency between spectrum envelope information and F ₀ information, and synthesizes speech with higher quality speech than before, an acoustic model learning apparatus for learning an acoustic model therefor, and a method therefor The purpose is to provide a program.

In order to solve the above problems, according to one aspect of the present invention, the acoustic model learning device sets the total number of learning speech data to N and N is any integer of 1 or more, n = 1, 2 , ..., N, N pieces of fundamental frequency information f _{L, n} respectively indicating the fundamental frequencies of the N pieces of learning speech data, and N pieces of contexts of the N pieces of learning speech data respectively represented by numerical vectors A fundamental frequency generation model learning unit that learns a fundamental frequency generation model using a linguistic feature vector as an input and a corresponding fundamental frequency information as an output using a linguistic feature vector l _{L, n,} and N fundamental frequencies Using fundamental information f _{L, n} , N language feature vectors l _{L, n,} and spectral envelope information s _{L, n} respectively indicating the spectral envelopes of N training speech data, fundamental frequency information and language A feature quantity vector is an input, and spectral envelope information is an output And a spectral envelope generation model learning unit for learning a spectral envelope generation model.

In order to solve the above problems, according to another aspect of the present invention, in the acoustic model learning method, the total number of learning speech data is N, and N is any integer of 1 or more, n = 1, N pieces of fundamental frequency information f _{L, n} indicating the fundamental frequencies of N pieces of learning voice data, and N pieces of contexts of N pieces of learning voice data represented by numerical vectors Basic frequency generation model learning step of learning a fundamental frequency generation model using a language feature vector as an input and a corresponding fundamental frequency information as an output using the language feature vector l _{L, n of} N, and N basics By using frequency information f _{L, n} , N language feature vectors l _{L, n,} and spectral envelope information s _{L, n} respectively indicating the spectral envelope of N learning speech data, basic frequency information and Inputs linguistic feature vectors and outputs spectral envelope information And a spectral envelope generation model learning step of learning a spectral envelope generation model.

  According to the present invention, it is possible to synthesize speech of higher quality than before.

The functional block diagram of the acoustic model learning apparatus which concerns on a prior art. The functional block diagram of the speech synthesizer concerning a prior art. 1 is a functional block diagram of an acoustic model learning device according to a first embodiment. The figure which shows the example of the processing flow of the acoustic model learning apparatus which concerns on 1st embodiment. FIG. 1 is a functional block diagram of a speech synthesizer according to a first embodiment. The figure which shows the example of the processing flow of the speech synthesizer concerning 1st embodiment. The functional block diagram of the acoustic model learning apparatus which concerns on 2nd embodiment. The figure which shows the example of the processing flow of the acoustic model learning apparatus which concerns on 2nd embodiment. FIG. 7 is a functional block diagram of a speech synthesis device according to a second embodiment. The figure which shows the example of the processing flow of the speech synthesizer concerning 2nd embodiment. The functional block diagram of the acoustic model learning apparatus which concerns on 3rd embodiment. The figure which shows the example of the processing flow of the acoustic model learning apparatus which concerns on 3rd embodiment. The functional block diagram of the speech synthesizer concerning a third embodiment. The figure which shows the example of the processing flow of the speech synthesizer concerning 3rd embodiment. The functional block diagram of the acoustic model learning apparatus which concerns on 4th embodiment. The figure which shows the example of the processing flow of the acoustic model learning apparatus which concerns on 4th embodiment. The functional block diagram of the speech synthesizer concerning a fourth embodiment. The figure which shows the example of the processing flow of the speech synthesizer concerning 4th embodiment.

  Hereinafter, embodiments of the present invention will be described. In the drawings used in the following description, the same reference numerals are given to constituent parts having the same functions and steps for performing the same processing, and redundant description will be omitted. In the following description, the processing performed for each element of a vector or matrix is applied to all elements of that vector or matrix unless otherwise noted.

<Point of the first embodiment>
In the present embodiment, the F ₀ information is utilized for the input of the DNN that generates the spectral envelope information. The DNN is configured to utilize the F ₀ information as input to the speech synthesizer, in addition to the conventional contexts such as reading and accent, and to output spectral envelope information reflecting the corresponding F ₀ information. With such a configuration, it is possible to generate spectrum envelope information that reflects the dependency with the F ₀ information. The naturalness of the synthesized speech is improved by satisfying the dependency between the generated spectral envelope information and the F ₀ information.

<Overall configuration>
The present embodiment includes an acoustic model learning device 110 and a speech synthesis device 120. FIGS. 3, 4, 5 and 6 are functional block diagrams of the acoustic model learning device 110, a process flow thereof, a functional block diagram of the speech synthesizer 120, and a process flow thereof.

In the acoustic model learning device 110, F ₀ data {f ₁ , f ₂ ,..., F _N }, spectrum envelope data {s ₁ , s ₂ ,..., S _N }, context data {t ₁ , t ₂ ,. Using t _N }, the F ₀ generation DNN (also described as DNN _{f in the} figure) and the spectral envelope generation DNN (also described as DNN _{s in the} figure) are learned.

In the speech synthesis device 120, the input text tex _o text analysis and speech feature quantity linguistic feature obtained from vector extraction amount vector l _o, and the F ₀ generation DNN, generates an F ₀ information f _o. Next, spectral envelope information s _o is generated from the language feature vector l _o , the generated F ₀ information f _o , and the spectral envelope generation DNN.

<Terminology, Description of Data to be Used>
・ F ₀ data, spectrum envelope data
F ₀ data, the spectral envelope data, respectively, the audio data (hereinafter, also referred to as training speech data) to be used in the acoustic model training obtained as a result of performing the signal processing on the audio signal, F ₀ information of each utterance a (pitch) f _n, spectral envelope information (cepstrum, Mel cepstrum, etc.) n total amount held data of each training speech data s _n. Using utterances number N in the context data, the F ₀ data _{{f 1, f 2, ...} , f N}, {s 1, s 2, ..., s N} the spectral envelope data is expressed by.

For example, when the time length of the n-th learning speech data of the N pieces of learning speech data is T _n frames, the F ₀ information f _n is data holding pitch information of each frame time. There is a 1 × T _n- dimensional real vector. Alternatively, it may be a 2 × T _n- dimensional real vector including voiced / unvoiced information.
For example, the spectral envelope information s _n is data holding phonological information of each frame time of the n-th training speech data, and even if only low dimensions of the extracted cepstrum and mel cepstrum are extracted and used Good. When the time length of the utterance n is T _n frames, for example, an M-dimensional mel cepstrum is used to form an M × T _n- dimensional real vector or the like.

Context Data Context data is data in which the context (speech information) of learning speech data is held for the total number N of learning speech data. For example, context data is represented by {t ₁ , t ₂ ,..., T _N }.
For example, the context t _n is information such as pronunciation given to the n-th learning speech data. The context needs to include phoneme information (pronunciation information) and accent information (accent type, accent phrase length). As a context, part-of-speech information may be included in addition to this. In addition, information on the start time and end time of each phoneme (phoneme segmentation information) may be stored.

Language Feature Amount Vector The language feature amount vector l _n is the context t _n represented by a numerical vector. For example, as in Non-Patent Document 1, the phoneme information and the accent information are each represented by 1-of-K, and further, they are combined with numerical information such as a sentence length to obtain a numerical vector. If the time length of the utterance is T _n frames, for example, a K-dimensional vector around a frame is used, and a K × T _n- dimensional real vector is used as the language feature vector l _n .
The linguistic feature quantity vector data holds corresponding linguistic feature quantity vectors l _n for each utterance included in the context data {t ₁ , t ₂ ,..., T _N }. Expressing as {l ₁ , l ₂ ,..., L _N } using the number of utterances N in the context data.

Language Feature Value F ₀ Vector The language feature value F ₀ vector is a vector holding information of both the language feature value vector l _n and the F ₀ information f _n . For example, the language feature vector l _n and the F ₀ information f _n are connected to create x _n = [l _n ^T , f _n ^T ] ^T.
The language feature amount · F ₀ vector data is obtained by extracting the language feature amount · F ₀ vector x _n from each of the learning speech data included in the N pieces of learning speech data, and holding it as data. It is represented by {x ₁ , x ₂ ,..., X _N } using the number of utterances N in the context data.

<Acoustic Model Learning Device 110 According to First Embodiment>
Perform acoustic model learning from the F ₀ data, spectral envelope data, and context data, and output a DNN acoustic model. Is different from the algorithm of the conventional method points, (1) the point of creating a language feature value · F ₀ vector data, a point to learn the F ₀ generation DNN to generate only (2) F _0, the generation of (3) the spectral envelope In order to utilize not only the language feature but also the F ₀ information, the language feature / F ₀ vector data is used as an input of the spectral envelope generation DNN.

FIG. 3 shows a functional block diagram of the acoustic model learning device 110 according to the first embodiment, and FIG. 4 shows its processing flow.
For example, the acoustic model learning device 110 is configured of a computer including a CPU, a RAM, and a ROM storing a program for executing the following processing, and is functionally configured as follows: . The acoustic model learning device 110 includes a language feature vector data generation unit 111, a language feature / F ₀ vector data generation unit 112, an F ₀ generation model learning unit 113, and a spectrum envelope generation model learning unit 114. The processing content of each part will be described below.

<Language feature vector data creation unit 111>
The language feature vector data creation unit 111 receives context data {t ₁ , t ₂ ,..., T _N } and creates a language feature vector l _n for the context t _{n of} each utterance (S 111). The result is held as language feature vector data {l ₁ , l ₂ ,..., L _N }.

<Language Feature Value / F ₀ Vector Data Creation Unit 112>
Linguistic feature quantity · F ₀ vector data creating unit 112, F ₀ data _{{f 1, f 2, ...} , f N} language feature vector data _{{l 1, l 2, ...} , l N} as input and , Lin the language feature vector l _n corresponding to the n-th training speech data and the F ₀ information f _n, and create as the language feature quantity · F ₀ vector x _n = [l _n ^T , f _n ^T ] ^T Then, the same processing is performed for N utterances, and the language feature amount / F ₀ vector data {x ₁ , x ₂ ,..., X _N } are created (S 112) and held.

<F ₀ Generation Model Learning Unit 113>
F ₀ generation model learning unit 113, the language feature vector data _{{l 1, l 2, ...} , l N} and F ₀ data _{{f 1, f 2, ...} , f N} as input and these data Learn a DNN (hereinafter also referred to as an F ₀ generation model, also described as DN N _{f in} the figure) that takes a language feature amount vector as an input and an output corresponding F ₀ information using (S 113), and holds . Any existing technique may be used as a learning method of the F ₀ generation model. For example, except for vectors used for input and output, the learning method, model configuration, and the like are the same as in Non-Patent Document 1.

<Spectrum envelope generation model learning unit 114>
Spectral envelope generator model learning unit 114, the language characteristic quantity · F ₀ vector data _{{x 1, x 2, ...} , x N} and spectral envelope data _{{s 1, s 2, ...} , s N} as input and, These data are used to learn a spectral envelope generation DNN (hereinafter also referred to as a spectral envelope generation model, also described as DNN _{s in} the figure), which receives as input a linguistic feature and an F ₀ vector, and outputs spectral envelope information. (S114). Any existing technique may be used as a learning method of the spectral envelope generation model. For example, except for vectors used for input and output, the learning method, model configuration, and the like are the same as in Non-Patent Document 1.

<Speech synthesizer 120 according to the first embodiment>
The speech synthesizer 120 generates synthesized speech z _o from the text tex _o to be synthesized. Conventional approaches algorithm differs from F ₀ generation DNN, generates only F ₀ information f _n, in generating the spectral envelope information s _o from the spectrum envelope generating DNN, together with linguistic feature quantity l _n F ₀ information f _n is used.

  FIG. 5 shows a functional block diagram of the speech synthesizer 120 according to the first embodiment, and FIG. 6 shows its processing flow.

For example, the voice synthesizer 120 is configured by a computer including a CPU, a RAM, and a ROM storing a program for executing the following processing, and is functionally configured as follows. The speech synthesizer 120 includes a text analysis unit 121, a language feature vector extraction unit 122, an F ₀ generation unit 123, a language feature / F ₀ vector generation unit 124B, a spectrum envelope generation unit 124, and an audio waveform generation. And a section 125. The processing content of each part will be described below.

<Text Analysis Unit 121>
The text analysis unit 121 takes as input the text tex _o to be subjected to speech synthesis, analyzes the text tex _o as text (S121), and obtains the context t _o .

<Language feature vector extraction unit 122>
The language feature vector extraction unit 122 receives the context t _o as an input, extracts the language feature vector l _o corresponding to the context t _o (S 122), and outputs it.

<F ₀ generation unit 123>
The F ₀ generation unit 123 receives the F ₀ generation model DDN _f in advance prior to speech synthesis. At the time of speech synthesis, the F ₀ generation unit 123 receives the language feature quantity vector _lo as input, performs forward propagation of the F ₀ generation model DDN _f , and outputs an output vector as F ₀ information f _o (S 123). Incidentally, F ₀ information f _o is information showing the basic frequency of the voice waveform corresponding to the text tex _o.

<Language Feature Value / F ₀ Vector Creation Unit 124 B>
Linguistic feature quantity-F ₀ vector creation section 124B receives as input the language feature vector l _o and F ₀ information f _o, connects the language feature vector l _o and F ₀ information f _o, linguistic feature quantity and It is created as an F ₀ vector x _o = [l _o ^T , f _o ^T ] ^T (S 124 B) and output.

<Spectrum envelope generation unit 124>
The spectrum envelope generation unit 124 receives the spectrum envelope generation model DDN _s in advance prior to speech synthesis. Spectrum envelope generating unit 124 inputs the linguistic feature quantity · F ₀ vector x _o, performs forward propagation of the spectral envelope generating model DDN _s, the output vector, as the spectral envelope information s _o, and outputs (S124). Incidentally, the spectral envelope information s _o is information indicating the spectral envelope information of a speech waveform corresponding to the text tex _o.

<Voice waveform generation unit 125>
The speech waveform generation unit 125 receives the F ₀ information f _o and the spectral envelope information s _o and generates a speech waveform (synthesized speech z _o ) corresponding to the text tex _o using these values (S 125), Output. Prior to speech waveform generation, for example, a speech parameter sequence smoothed in the time direction may be obtained using a maximum likelihood generation (MLPG) algorithm (see reference 1). Further, for example, (Reference 2) may be used for speech waveform generation.
[Reference 1] Masuko et al., "HMM-based speech synthesis using dynamic features," Theory of philosophy, vol. J79-D-II, no. 12, pp. 2184-2190, Dec. 1996.
[Reference 2] Imai et al., "Mel log spectral approximation (MLSA) filter for speech synthesis", Transactions of the Institute of Electronics, Information and Communication Engineers A Vol. J66-A No. 2 pp. 122-129, Feb. 1983.

<Effect>
With the above configuration, as the input speech synthesizer, read, in addition to the conventional context, such as accents, constituting the spectrum envelope generation DNN to output the spectral envelope information reflecting the F ₀ information. This makes it possible to generate spectral envelope information that satisfies the dependency relationship with the F ₀ information. This improves the quality of the synthesized speech.

Second Embodiment
Description will be made focusing on parts different from the first embodiment.

In the first embodiment, extremely high F ₀ and extremely low F ₀ may be used as the input of the spectral envelope generation DNN. In that case, the spectral envelope information generated from the spectral envelope generation DNN may become unstable and the quality of the synthesized speech may be degraded.

In order to solve this problem, in this embodiment, an output value when F ₀ information is input to a bounded function F (x) is used as an input of the spectral envelope generation DNN. The fact that the input of the spectral envelope generation DNN is bounded stabilizes the spectral envelope information generated from the spectral envelope generation DNN and improves the quality of the synthesized speech.

This embodiment, compared to the first embodiment, there are F ₀ conversion unit in front of the linguistic feature quantity · F ₀ vector (data) creating unit, that converts the F ₀ information is different.

<Terminology, Description of Data to be Used>
· Linguistic feature quantity and converting F ₀ vector linguistic feature quantity and converting F ₀ vector in the present embodiment is a vector which holds both the information of the language feature vector l _n and F ₀ information f _n. The point which uses the output which carried out F ₀ value conversion differs from a 1st embodiment. For example, two vectors of the language feature vector l _n and the converted F ₀ information f _n ⁽¹⁾ are connected to create x _n = [l _n ^T , f _n ^{(1) T} ] ^T.

<Sound Model Learning Device 210 According to Second Embodiment>
FIG. 7 shows a functional block diagram of the acoustic model learning device 210 according to the second embodiment, and FIG. 8 shows its processing flow.
The acoustic model learning device 210 includes a language feature vector data generation unit 111, a language feature / F ₀ vector data generation unit 112, an F ₀ generation model learning unit 113, a spectrum envelope generation model learning unit 114, and F _0. And a conversion unit 215.

を使用する。 <F ₀ conversion unit 215>
The F ₀ conversion unit 215 receives F ₀ data {t ₁ , t ₂ ,..., T _N } as input, and uses bounded vector function F (f _n ) to obtain F ₀ data {f ₁ , f ₂ , , F _N } is converted (S 215), and the converted F ₀ data {f ₁ ⁽¹⁾ , f ₂ ⁽¹⁾ ,..., F _N ⁽¹⁾ } are output. For example, _{assuming that} the frame length of the F ₀ information f _n is T _n frame and f _n = [f _n1 , f _n2 ,..., F _{nT_n} ] ^T , F (f _n ) = [G (f _n1 ), G (f _n2 ),..., G (f _n ^{T —} _n )] ^T. However, the subscript T_n means T _n . Here, G is a bounded scalar function. For example, as G, sigmoid function

Use

Note that the language feature value / F ₀ vector data creation unit 112 substitutes the converted F ₀ data {f ₁ ⁽¹⁾ , f ₂ ^{(in place} of the F ₀ data {f ₁ , f ₂ ,..., F _N } ⁾ . ¹⁾ , ..., f _N ⁽¹⁾ } is used. The other configuration is the same as that of the first embodiment.

<Speech synthesizer 220 according to the second embodiment>
FIG. 9 shows a functional block diagram of the speech synthesizer 220 according to the second embodiment, and FIG. 10 shows its processing flow.
The speech synthesis unit 220 includes a text analysis unit 121, a language feature vector extraction unit 122, an F ₀ generation unit 123, a language feature / F ₀ vector generation unit 124B, a spectrum envelope generation unit 124, and an audio waveform generation. Unit 125 and an F ₀ conversion unit 224A.

<F ₀ conversion unit 224A>
The F ₀ conversion unit 224 A receives the F ₀ information f _o and uses the bounded vector function F (f _n ) to convert the F ₀ information f _o (S 224 A), and the converted F ₀ information f _o Output ⁽¹⁾ . As a conversion method, a method corresponding to the F ₀ conversion unit 215 may be used.
Incidentally, the language characteristic quantity · F ₀ vector generating unit 124B has, F ₀ instead of the information f _o, using F ₀ information f _o ⁽¹⁾ after the conversion. The other configuration is the same as that of the first embodiment.

<Effect>
With such a configuration, the same effect as that of the first embodiment can be obtained. Furthermore, the spectral envelope information generated from the spectral envelope generation DNN is stabilized, and the quality of synthesized speech is improved. In the second embodiment, the F ₀ data and the F ₀ information to be used are limited, and it can be said that the first embodiment is limited.

Third Embodiment
Description will be made focusing on parts different from the second embodiment.

In the second embodiment, one fixed function is used as the vector function F (x). On the other hand, from the learning data, if it is possible to estimate an appropriate vector function F (x) as an input spectrum envelope generating DNN, considered more appropriate dependencies of the spectral envelope information and the F ₀ information is modeled.

To solve this problem, in the present embodiment, a function F (x; θ ⁽⁰⁾ ) having an F ₀ value conversion parameter θ ⁽⁰ ) is used instead of one fixed function F (x), The training data is used to estimate the F ₀ value conversion parameter θ ⁽⁰⁾ . By learning an appropriate vector function, the dependency between the spectral envelope information and the F ₀ information can be more flexibly modeled, and the synthetic speech quality can be improved.

<Terminology, Description of Data to be Used>
・ F ₀ value conversion parameter
The F ₀ value conversion parameter is a parameter used in parametric F ₀ value conversion, and is expressed by θ ⁽⁰⁾ .

Parametric Transform F ₀ Information The parametric transform F ₀ information is a real value output by parametric F ₀ value transformation. Expressed by f _n ⁽²⁾ .

Language Feature Value Parametric Conversion F ₀ Vector The language feature value parametric conversion F ₀ vector is a vector obtained from the language feature value vector l _n and parametric conversion F ₀ information f _n ⁽²⁾ . Expressed by x _n . For example, the language feature vector l _n and the parametric conversion F ₀ information f _n ⁽²⁾ are connected to create x _n = [l _n ^T , f _n ^{(2) T} ] ^T.

を使用する。または、ベクトル関数F(x;θ⁽⁰⁾)を、xを入力ベクトル、θ⁽⁰⁾をパラメータとするニューラルネットワークとしてもよい。 ・ Parametric F ₀ value conversion
Parametric conversion F ₀ information f _n ⁽²⁾ is output using F ₀ information f _n , F ₀ value conversion parameter θ ⁽⁰⁾ , and F ₀ value conversion function F (x; θ ⁽⁰⁾ ). At this time, a vector whose value range is bounded is used as F (x; θ ⁽⁰⁾ ). Also, in order to enable learning of θ ⁽⁰⁾ by error back propagation of DNN, a function is used in which the output value of F (x; θ ⁽⁰⁾ ) is differentiable with respect to θ ⁽⁰⁾ . For example, the frame length F ₀ information f _n and T _{n frame, f n = [f n1,} f n2, ..., f nT_n] T, F (f n; θ (0)) = [G (f n1; Let θ ⁽⁰⁾ , G (f _n2 ; θ ⁽⁰⁾ ), ..., G (f _{nT_n} ; θ ⁽⁰⁾ )] ^T , then the parametric sigmoid function

Use Alternatively, the vector function F (x; θ ⁽⁰⁾ ) may be a neural network using x as an input vector and θ ⁽⁰⁾ as a parameter.

<Sound Model Learning Device 310 According to Third Embodiment>
FIG. 11 shows a functional block diagram of the acoustic model learning device 310 according to the third embodiment, and FIG. 12 shows its process flow.

The acoustic model learning device 310 performs spectrum envelope generation DNN learning / F ₀ value conversion parameter estimation from spectrum envelope data, F ₀ data, and context data, and outputs a spectrum envelope generation DNN and F ₀ value conversion parameter. It differs from the two embodiments. The spectral envelope generating DNN learning · F ₀ value conversion parameter estimation, spectral envelope data, F ₀ data, from the language feature vector data, to estimate the F ₀ value conversion parameters of the spectral envelope DNN.

The acoustic model learning device 310 includes a language feature vector data generation unit 111, a language feature / F ₀ vector data generation unit 112, an F ₀ generation model learning unit 113, a spectrum envelope generation model / conversion parameter learning unit 314, and the like. , F ₀ conversion unit 315.

<F ₀ conversion unit 315>
Prior to learning, the F ₀ conversion unit 315 initializes the F ₀ value conversion parameter θ ⁽⁰⁾ in advance. For example, the F ₀ value conversion parameter θ ⁽⁰⁾ is initialized by a random number. For example, initialization of the F ₀ value conversion parameter θ ⁽⁰⁾ is sampled from a standard normal distribution when using a parametric sigmoid function as a bounded vector function F (x; θ ⁽⁰⁾ ).

F ₀ conversion unit 315, F ₀ data _{{f 1, f 2, ...} , f N} at the time of learning as input, F ₀ data _{{f 1, f 2, ...} , f N} and F ₀ value conversion parameters Perform parametric F ₀ value conversion using θ ⁽⁰⁾ and (f _n ⁽²⁾ = [G (f _n1 ; θ ⁽⁰⁾ ), G (f _n2 ; θ ⁽⁰⁾ ), ..., G ( f _{n T — n} ; θ ⁽⁰⁾ ]], S 315) Parametric transformation F ₀ data {f ₁ ⁽²⁾ , f ₂ ⁽²⁾ ,..., f _N ⁽²⁾ } are obtained and output.
Note that the language feature value / F ₀ vector data creation unit 112 replaces the F ₀ data {f ₁ ⁽¹⁾ , f ₂ ⁽¹⁾ ,..., F _N ⁽¹⁾ } with converted F ₀ data { Use f ₁ ⁽²⁾ , f ₂ ⁽²⁾ ,..., f _N ⁽²⁾ }.

<Spectrum envelope generation model / conversion parameter learning unit 314>
The spectral envelope generation model / conversion parameter learning unit 314 includes the F ₀ value conversion parameter θ ⁽⁰⁾ (initial value) and the language feature amount / F ₀ vector data {x ₁ , x ₂ , ..., x _N } (wherein With x _n = [f _n ^{(2) T} , l _n ^T ] ^T ) and spectral envelope data {s ₁ , s ₂ ,..., s _N } as input, using these data, the language feature amount F ₀ vector as an input, learns the spectrum envelope generation DNN and F ₀ value conversion parameter theta ⁽⁰⁾ to output the spectrum envelope information, outputs the spectral envelope generating DNN and F ₀ value after learning transformation parameter theta ⁽¹⁾ Do. For example, it learns as follows.

(1) Linguistic Feature Parameter-Parametric Transform F ₀ Vector x _n is an input vector of DNN, and DNN is forward propagated.
(2) Measure the error between the output vector z _n (spectral envelope information obtained from the n-th speech data for learning) and the spectral envelope information s _n , back propagate the error, and use the parameters W and F ₀ value conversion parameters of DNN Calculate the error gradient of θ ⁽⁰⁾ . However, the parameter W of DNN is initialized in advance by random numbers prior to learning. For example, random number initialization of W uses the same method as in Non-Patent Document 1. Also, as the error function, for example, the least square error of z _n and s _n is used.
(3) Update parameter W and F ₀ value conversion parameter θ ⁽⁰⁾ according to the error gradient.

Processing in the F ₀ conversion unit 315 (S315), processing in the linguistic feature quantity · F ₀ vector data creation unit 112 (S112), and the above-mentioned (1) to repeat ~ the process (3) until the convergence criterion.
The obtained parameter W ⁽ when determined to have converged) and the F ₀ value conversion parameter θ ⁽⁰⁾ are output as learned spectral envelope generation DNN and F ₀ value conversion parameter θ ⁽¹⁾ , respectively. For example, as the convergence determination, use is made of whether the number of iterations has reached a threshold, the change in error function between iterations has become smaller than the threshold, or both.

<Speech synthesizer 320 according to the third embodiment>
FIG. 13 shows a functional block diagram of the speech synthesizer 320 according to the third embodiment, and FIG. 14 shows its processing flow.
The voice synthesizer 320 differs from that of the second embodiment in that the F ₀ information f _o is converted using the F ₀ value conversion parameter θ ⁽¹⁾ obtained by the acoustic model learning device 310.
The speech synthesizer 320 includes a text analysis unit 121, a language feature vector extraction unit 122, an F ₀ generation unit 123, a language feature / F ₀ vector generation unit 124B, a spectrum envelope generation unit 124, and an audio waveform generation. Unit 125 and an F ₀ conversion unit 324A.

<F ₀ conversion unit 324A>
The F ₀ conversion unit 324 A receives the F ₀ value conversion parameter θ ⁽¹⁾ and the F ₀ information f _o as input, and uses the F ₀ information f _o and the F ₀ value conversion parameter θ ⁽¹⁾ to obtain a parametric F ₀ performs value conversion ^{_{(f o (2) = [}} G (f o1; θ (1)), G (f o2; θ (1)), ..., G (f oT_o; θ (1))], S324A) , Parametric transformation F ₀ data f _o ⁽²⁾ is output. At this time, the function F (x; θ ⁽¹⁾ ) used is the same as the parametric F ₀ value conversion used in the F ₀ conversion unit 315.
The language feature amount / F ₀ vector creation unit 124 B uses parametric conversion F ₀ information f _o ⁽²⁾ instead of the conversion F ₀ information f _o ⁽¹⁾ .

<Effect>
With such a configuration, the same effect as that of the second embodiment can be obtained. Further, dependency of the spectral envelope information and the F ₀ information is more flexible model, synthesized speech quality is improved.

Fourth Embodiment
Description will be made focusing on parts different from the third embodiment.

In the spectral envelope generation DNN learning / F ₀ value conversion parameter estimation of the third embodiment, in order to enable synthesis of high quality speech, it is sufficient to learn a spectral envelope generation DNN having a small parameter generation error. Here, in the case of using an algorithm dependent on an initial value such as the gradient method, there is a problem that an appropriate initial value needs to be set in order to sufficiently reduce the parameter error of the spectral envelope generation DNN.

In this embodiment, as an initial value of F ₀ value conversion parameters, using the spectrum envelope generation DNN learning · F ₀ value conversion parameters estimated F ₀ value in the estimation conversion parameters in the third embodiment theta ^(1). Since the F ₀ value conversion parameter θ ⁽¹⁾ estimated in the third embodiment is determined by a criterion for minimizing the parameter generation error of a certain spectral envelope generation DNN, it is referred to as the F ₀ value conversion parameter θ It is expected that, by setting ⁽¹⁾ and performing spectrum envelope generation DNN learning / F ₀ value conversion parameter estimation again, it is possible to learn spectrum envelope generation DNN with a smaller parameter generation error. This further improves the quality of synthesized speech.

<Terminology, Description of Data to be Used>
And re-estimating Parametric F ₀ value conversion parameter re-estimation parametric F ₀ value conversion parameters are obtained by an acoustic model learning unit 410, a parameter for parametric F ₀ value conversion, it is referred to as theta ^(2). The point which is re-estimated using parametric F ₀ value conversion parameter theta ⁽¹⁾ which is a learning result of a third embodiment as an initial value differs from a third embodiment.

<Acoustic Model Learning Device 410 According to Fourth Embodiment>
FIG. 15 shows a functional block diagram of an acoustic model learning device 410 according to the third embodiment, and FIG. 16 shows its processing flow.
The acoustic model learning device 410 performs spectral envelope generation DNN learning / F ₀ value conversion parameter re-estimation using the F ₀ value conversion parameter θ ⁽¹⁾ obtained in the third embodiment as an initial value, and generates a spectrum envelope. The point of outputting DNN and re-estimated F ₀ value conversion parameter θ ⁽²⁾ is different from the third embodiment.

The acoustic model learning device 410 includes an acoustic model learning device 310, a language feature vector data generation unit 111, a language feature / F ₀ vector data generation unit 112, an F ₀ generation model learning unit 113, and a spectrum envelope generation model. A conversion parameter learning unit 314 and an F ₀ conversion unit 415 are included.
The acoustic model learning device 310 executes the processing described in the third embodiment prior to the processing of the acoustic model learning device 410, obtains the F ₀ value conversion parameter θ ⁽¹⁾ (S310), and outputs it.

<F ₀ conversion unit 415 and spectrum envelope generation model / conversion parameter learning unit 414>
The F ₀ conversion unit 415 and the spectral envelope generation model / conversion parameter learning unit 414 replace the F ₀ value conversion parameter θ ⁽⁰⁾ in advance prior to learning, and convert the F ₀ value which is the output value of the acoustic model learning device 310 The parameter θ ⁽¹⁾ is set as an initial value. The processing contents of the F ₀ conversion unit 415 and the spectrum envelope generation model / conversion parameter learning unit 414 are the same as those of the F ₀ conversion unit 315 and the spectrum envelope generation model / conversion parameter learning unit 314 (S 415, S 414). The spectrum envelope generation model / conversion parameter learning unit 414 outputs the re-estimated F ₀ value conversion parameter θ ⁽²⁾ in place of the F ₀ value conversion parameter θ ⁽¹⁾ .

<Speech synthesizer 420 according to the fourth embodiment>
FIG. 17 shows a functional block diagram of the speech synthesizer 420 according to the third embodiment, and FIG. 18 shows its processing flow.
The speech synthesizer 420 differs from the third embodiment in that the re-estimated F ₀ value conversion parameter θ ⁽²⁾ is used instead of the F ₀ value conversion parameter θ ⁽¹⁾ at the time of spectrum envelope generation.
The speech synthesis unit 220 includes a text analysis unit 121, a language feature vector extraction unit 122, an F ₀ generation unit 123, a language feature / F ₀ vector generation unit 124B, a spectrum envelope generation unit 124, and an audio waveform generation. Unit 125 and an F ₀ conversion unit 424 A.

<F ₀ conversion unit 424A>
The F ₀ conversion unit 424 A receives the re-estimated F ₀ value conversion parameter θ ⁽²⁾ and the F ₀ information f _o as a parametric input using the F ₀ information f _o and the F ₀ value conversion parameter θ ^(1). perform F ₀ value conversion ^{_{(f o (2) = [}} G (f o1; θ (2)), G (f o2; θ (2)), ..., G (f oT_o; θ (2))], S424A), and outputs parametric conversion F ₀ data f _o ⁽²⁾ . At this time, the function G (x; θ ⁽²⁾ ) used is the same as the parametric F ₀ value conversion used in the F ₀ conversion unit 415.
Note that the language feature value / F ₀ vector creation unit 124 B uses parametric transformation F ₀ data f _o ⁽²⁾ instead of f _o ⁽¹⁾ .

<Effect>
With such a configuration, the same effect as that of the third embodiment can be obtained. Furthermore, it is expected that the spectral envelope generation DNN with a small parameter generation error can be learned.

<Other Modifications>
The present invention is not limited to the above embodiments and modifications. For example, the various processes described above may be performed not only in chronological order according to the description, but also in parallel or individually depending on the processing capability of the apparatus that executes the process or the necessity. In addition, changes can be made as appropriate without departing from the spirit of the present invention.

<Program and Recording Medium>
In addition, various processing functions in each device described in the above-described embodiment and modification may be realized by a computer. In that case, the processing content of the function that each device should have is described by a program. By executing this program on a computer, various processing functions in each of the above-described devices are realized on the computer.

  The program describing the processing content can be recorded in a computer readable recording medium. As the computer readable recording medium, any medium such as a magnetic recording device, an optical disc, a magneto-optical recording medium, a semiconductor memory, etc. may be used.

  Further, this program is distributed, for example, by selling, transferring, lending, etc. a portable recording medium such as a DVD, a CD-ROM or the like in which the program is recorded. Furthermore, the program may be stored in a storage device of a server computer, and the program may be distributed by transferring the program from the server computer to another computer via a network.

  For example, a computer that executes such a program first temporarily stores a program recorded on a portable recording medium or a program transferred from a server computer in its own storage unit. Then, at the time of execution of the process, the computer reads the program stored in its storage unit and executes the process according to the read program. In another embodiment of the program, the computer may read the program directly from the portable recording medium and execute processing in accordance with the program. Furthermore, each time a program is transferred from this server computer to this computer, processing according to the received program may be executed sequentially. In addition, a configuration in which the above-described processing is executed by a so-called ASP (Application Service Provider) type service that realizes processing functions only by executing instructions and acquiring results from the server computer without transferring the program to the computer It may be Note that the program includes information provided for processing by a computer that conforms to the program (such as data that is not a direct command to the computer but has a property that defines the processing of the computer).

  In addition, although each device is configured by executing a predetermined program on a computer, at least a part of the processing content may be realized as hardware.

Claims (8)

Let N be the total number of learning voice data, N be any integer greater than or equal to 1, n = 1, 2,..., N and N fundamental frequencies indicating the fundamental frequencies of N training voice data A language feature vector is input using information f _{L, n} and N language feature vectors l _{L, n} representing the contexts of the N pieces of speech data for learning as numerical vectors, respectively A fundamental frequency generation model learning unit that learns a fundamental frequency generation model that outputs fundamental frequency information;
N pieces of the fundamental frequency information f _{L, n} , N pieces of the language feature vector l _{L, n,} and spectrum envelope information s _{L, n} respectively indicating spectrum envelopes of the N pieces of learning speech data And a spectral envelope generation model learning unit for learning a spectral envelope generation model using fundamental frequency information and a linguistic feature vector as inputs and using spectral envelope information as outputs.
Acoustic model learning device. The acoustic model learning device according to claim 1, wherein
Including a fundamental frequency conversion unit that converts each of the N pieces of the fundamental frequency information f _{L, n} using a bounded scalar function g,
The N pieces of fundamental frequency information f _{L, n} used in the spectrum envelope generation model learning unit are values converted by the fundamental frequency conversion unit,
Acoustic model learning device. The acoustic model learning device according to claim 2, wherein
The fundamental frequency transformation unit transforms the N pieces of fundamental frequency information f _{L, n} using the scalar function g and its parameter θ,
In the spectrum envelope generation model learning unit, spectrum envelopes of N pieces of converted fundamental frequency information f _{L, n} , N pieces of the language feature vector l _{L, n} , and N pieces of the speech data for learning are generated. Using spectral envelope information s _{L, n} respectively shown, the converted fundamental frequency information and language feature vector are input, and a spectral envelope generation model as output spectral envelope information and parameter θ are learned.
Acoustic model learning device. The acoustic model learning device according to claim 3, wherein
In the spectrum envelope generation model learning unit, the parameter after learning is θ ⁽¹⁾ , and the fundamental frequency conversion unit uses the scalar function g and its parameter θ ⁽¹⁾ to obtain N pieces of fundamental frequency information f Convert _{L and n} respectively,
In the spectrum envelope generation model learning unit, spectrum envelopes of N pieces of converted fundamental frequency information f _{L, n} , N pieces of the language feature vector l _{L, n} , and N pieces of the speech data for learning are generated. A spectral envelope generation model and scalar function parameter θ ⁽¹⁾ which takes as input the fundamental frequency information after conversion and the linguistic feature vector using the spectral envelope information s _{L, n} respectively shown and outputs the spectral envelope information To learn
Acoustic model learning device. A speech synthesis apparatus which performs speech synthesis using the fundamental frequency generation model learned by the acoustic model learning device according to any one of claims 1 to 4 and the spectrum envelope generation model,
The fundamental frequency generation model is used to generate fundamental frequency information f _O indicating the fundamental frequency of the speech waveform corresponding to the target text from the language feature vector l _O corresponding to the context obtained by analyzing the target text by text analysis A fundamental frequency generation unit,
Spectrum envelope generation unit that generates spectrum envelope information s _O indicating the spectrum envelope of the speech waveform corresponding to the target text from the language feature vector l _O and the fundamental frequency information f _O using the spectrum envelope generation model When,
And a speech waveform generation unit that generates a speech waveform corresponding to the target text using the fundamental frequency information f _O and the spectrum envelope information s _O.
Speech synthesizer. Let N be the total number of learning voice data, N be any integer greater than or equal to 1, n = 1, 2,..., N and N fundamental frequencies indicating the fundamental frequencies of N training voice data A language feature vector is input using information f _{L, n} and N language feature vectors l _{L, n} representing the contexts of the N pieces of speech data for learning as numerical vectors, respectively A fundamental frequency generation model learning step of learning a fundamental frequency generation model that outputs fundamental frequency information;
N pieces of the fundamental frequency information f _{L, n} , N pieces of the language feature vector l _{L, n,} and spectrum envelope information s _{L, n} respectively indicating spectrum envelopes of the N pieces of learning speech data And a spectral envelope generation model learning step of learning a spectral envelope generation model using the fundamental frequency information and the linguistic feature vector as inputs and the spectral envelope information as output.
Acoustic model learning method. A speech synthesis method for speech synthesis using the fundamental frequency generation model learned by the acoustic model learning method according to claim 6 and the spectrum envelope generation model,
The fundamental frequency generation model is used to generate fundamental frequency information f _O indicating the fundamental frequency of the speech waveform corresponding to the target text from the language feature vector l _O corresponding to the context obtained by analyzing the target text by text analysis A fundamental frequency generation step,
A spectrum envelope generation step of generating spectrum envelope information s _O indicating a spectrum envelope of a speech waveform corresponding to the target text from the language feature vector l _O and the fundamental frequency information f _O using the spectrum envelope generation model When,
An audio waveform generation step of generating an audio waveform corresponding to a target text using the fundamental frequency information f _O and the spectrum envelope information s _O ;
Speech synthesis method.   A program for causing a computer to function as the acoustic model learning device according to any one of claims 1 to 4 or the speech synthesis device according to claim 5.

Images