JP2021056467A - Learning device, voice synthesis device and program - Google Patents

Abstract

To stably obtain a high-quality voice signal when synthesizing an arbitrary text.SOLUTION: A language analysis unit 11 of a learning device 1 performs language analysis processing on a text of a voice corpus, and generates a context-dependent label. A relative valuation unit 12 converts time information related to rhythm of the context-dependent label into a relative value, and generates a context-dependent relative label. A context question group processing unit 13 performs context question group application processing on the context-dependent relative label, and generates a language feature matrix. A voice analysis unit 14 performs voice analysis processing on a voice signal corresponding to each text of the voice corpus, and obtains an acoustic feature amount. An associating unit 15 obtains the duration length of each phoneme by associating using a phoneme alignment technique, and obtains a language feature amount corresponding to the acoustic feature amount. A learning unit 16 learns a time length model using the language feature matrix and the duration length for each phoneme, and learns an acoustic model using the language feature amount and the acoustic feature amount for each frame.SELECTED DRAWING: Figure 1

Description

The present invention relates to a learning device that learns a statistical model using a text and a voice signal, a voice synthesizer that synthesizes a voice signal from a text using a statistical model, and a program thereof.

Conventionally, as a method of learning a statistical model using a text and a corresponding voice signal and synthesizing a voice signal for an arbitrary text, deep learning (DL: Deep) using a deep neural network (DNN) is used. A technique based on Learning) is known (see, for example, Non-Patent Document 1).

FIG. 18 is an explanatory diagram showing a flow of the conventional pre-learning process described in Non-Patent Document 1. A conventional learning device that performs pre-learning learns a time-length model and an acoustic model using a text prepared in advance and a corresponding voice signal.

Specifically, the learning device reads the text from the voice corpus and obtains a context-sensitive label by linguistic analysis processing (step S1801), and applies a group of questions about the context prepared in advance to obtain a language feature matrix (step S1801). Step S1802). Further, the learning device reads out the voice signal from the voice corpus and obtains the acoustic feature amount by the voice analysis process (step S1803).

The learning device obtains the duration time length for each phoneme by time-associating the language feature matrix and the acoustic feature quantity (step S1804), and the language feature quantity is obtained from the duration time length and the language feature matrix for each phoneme. (Step S1805). Then, the learning device learns the time length model using the language feature matrix and the duration time length for each phoneme (step S1806). Further, the learning device learns the acoustic model using the language features and the acoustic features (step S1807).

FIG. 19 is an explanatory diagram showing a flow of the conventional speech synthesis process described in Non-Patent Document 1. A conventional speech synthesizer that performs speech synthesis inputs arbitrary text, obtains a context-sensitive label from the text by linguistic analysis processing (step S1901), and applies a group of questions about the context prepared in advance to obtain a language feature matrix. Obtain (step S1902).

The speech synthesizer estimates the duration length of each phoneme using the language feature matrix and the time length model learned in advance (step S1903), and obtains the language feature quantity from the duration time length of each phoneme and the language feature matrix. (Step S1904).

The speech synthesizer estimates the acoustic features using the language features and the pre-learned acoustic model (step S1905). Then, the voice synthesizer obtains a voice signal waveform synthesized by voice generation processing from the acoustic features to obtain a synthesized voice signal for an arbitrary text (step S1906).

In order to obtain the linguistic feature quantity in these series of processes, the learning device and the speech synthesizer first perform linguistic analysis processing such as morphological analysis and parsing on the text, and based on the information obtained by this. To find the context-sensitive label. The context-sensitive label is described in a predetermined context-sensitive label format, is obtained for each utterance, and is composed of phoneme units. The format of the context-sensitive label differs depending on the language, but in Japanese, a predetermined format can be used (see, for example, Non-Patent Document 2).

Next, the learning device and the speech synthesizer prepare a question group regarding the context in advance (see, for example, Non-Patent Document 3), apply the question group to the phoneme information of each line in the context-dependent label, and apply the question group to the language. Find the feature matrix. The learning device then obtains the duration length for each phoneme by time-associating the language feature matrix with the acoustic features. On the other hand, the speech synthesizer estimates the duration length of each phoneme by using the language feature matrix and the time length model learned in advance.

The learning device and the speech synthesizer correspond to the duration length of the phoneme in the language feature matrix of several frames (processing unit of the acoustic feature amount) according to the duration length of the phoneme currently being focused on in the speech. By adding the number of frames and the position information in the frame, the language feature amount corresponding to the acoustic feature amount is obtained.

Zhizheng Wu, Oliver Watts, Simon King, “Merlin: An Open Source Neural Network Speech Synthesis System”, in Proc. 9th ISCA Speech Synthesis Workshop (SSW9), September 2016, Sunnyvale, CA, USA. “An example of context-dependent label format for HMM-based speech synthesis in Japanese”, [online], HTS Working Group, Dec. 25, 2015, [Search on September 7, 1991], Internet <URL: http //hts.sp.nitech.ac.jp/> ＞ “Added japanese question set compatible with OpenJTalk produced labels.”, [Online], HTS Working Group, Dec. 25, 2015, [Searched on September 7, 1991], Internet <URL: https://github.com /CSTR-Edinburgh/merlin/blob/master/misc/questions/questions-japanese.hed ＞

In the method of the above-mentioned non-patent document 1, the question group regarding the context described in the above-mentioned non-patent document 3 is applied to the context-dependent label corresponding to the format described in the above-mentioned non-patent document 2 and quantified. By doing so, the language feature matrix is obtained.

However, in this method, the number of possible combinations of numerical values in the language feature matrix becomes enormous. This is because the range of possible numerical values for a plurality of elements constituting the language feature matrix is wide and different, and is not unified.

In the pre-learning process shown in FIG. 18, the statistical model is learned by performing the normalization process and the standardization process for each dimension. Further, in the speech synthesis process shown in FIG. 19, the feature amount is estimated using the statistical model by performing the normalization process and the destandardization process for each dimension. However, each of the plurality of elements constituting the language feature matrix used for these processes has a particularly wide range of continuous numerical values.

In the pre-learning process, it is not possible to cover the possible combinations of numerical values of the language feature matrix, and some elements of the language feature matrix are not continuously distributed, resulting in a sparse state and a highly accurate statistical model. Can't learn. Then, in speech synthesis processing, when a language feature matrix is obtained by inputting arbitrary text, even if the elements are within a range in which continuous numerical values can be taken, as long as such a statistical model is used, The estimation accuracy of the duration length and the acoustic feature amount for each phoneme becomes low.

In addition, since the elements of the language feature matrix are outliers that exceed the range in which continuous numerical values can be taken, an error occurs when estimating the duration length and the acoustic feature amount for each phoneme. This may deteriorate the quality of the synthesized voice signal and make the sound quality unstable.

Therefore, the present invention has been made to solve the above problems, and an object of the present invention is a learning device and voice capable of stably obtaining a high-quality voice signal when synthesizing a voice of an arbitrary text. To provide synthesizers and programs.

In order to solve the above problem, the learning device according to claim 1 uses the text and the voice signal, which are set in advance so that the voice signal corresponds to the text, and uses a time length model and an acoustic model for voice synthesis. In the learning device to be learned, a language analysis unit that performs language analysis processing on the preset text and generates a context-dependent label, and time information related to the rhyme included in the context-dependent label generated by the language analysis unit. Is relativized to generate a context-dependent relative label containing the time information of the relative value related to the rhyme, and the context-dependent relative label generated by the relative value unit is preset. Speech analysis to obtain the amount of acoustic features by performing speech analysis processing on the context question group processing unit that applies the question group related to the context and generates the language feature matrix and the voice signal corresponding to the preset text. The language feature matrix generated by the context question group processing section and the acoustic feature amount obtained by the speech analysis section are temporally associated with each other, and the duration length of each phonetic element is obtained to obtain the phonetic element. The duration length for each and the duration length for each phonetic element obtained by the matching unit for obtaining the language feature quantity from the language feature matrix, the language feature matrix generated by the context question group processing unit, and the matching unit. With the learning unit that learns the time length model and learns the acoustic model using the language feature amount obtained by the association unit and the acoustic feature amount obtained by the speech analysis unit. It is characterized by being prepared.

The learning device according to claim 2 is the learning device according to claim 1, wherein the learning unit performs deep learning (DL) on the time length model and the acoustic model. ..

Further, in the learning device according to the third aspect, in the learning device according to the first aspect, the relative valuation unit obtains time information related to the utterance, the number and position of exhalation paragraphs in the utterance, and an accent phrase in the utterance. From the number and position of, the number and position of beats in the utterance, the number and position of accent phrases in the exhalation paragraph, the number and position of beats in the exhalation paragraph, the position of beats in the accent phrase, and the accent nucleus in the accent phrase. One or more of the beat positions of the utterance, the relative value of the number of exhaled paragraphs and the relative value of the position in the utterance, the relative value of the number of accent phrases in the utterance and the relative value of the position, the above. Relative values of the number of beats and positions in the utterance, relative values of the number of accents and positions in the exhalation paragraph, relative values of the number of beats and positions in the exhalation paragraph, Includes one or more of the relative values of the beat positions in the accent phrase and the relative values of the beat positions from the accent nucleus in the accent phrase that correspond to the time information related to the utterance. It is characterized in that the context-dependent relative label is obtained.

Further, the speech synthesizer according to claim 4 is a speech synthesizer that synthesizes a speech signal for an arbitrary text by using the time length model and the acoustic model learned by the learning device according to claim 1 or 2. The language analysis unit that performs speech analysis processing on the text and generates a context-dependent label and the time information related to the rhyme included in the context-dependent label generated by the language analysis unit are made into relative values, and the relative values related to the rhyme. For the relative valuation unit that generates the context-dependent relative label including the time information of the above and the context-dependent relative label generated by the relative valuation unit, a process of applying a preset question group regarding the context is performed. Using the context question group processing unit that generates the language feature matrix, the language feature matrix generated by the context question group processing unit, and the time length model, the duration length of each phonetic element is estimated, and the duration of each phonetic element is estimated. Acoustic feature amount estimation that estimates the acoustic feature amount using the time length estimation unit that obtains the language feature amount from the duration time length and the language feature matrix, and the language feature amount and the acoustic model obtained by the time length estimation unit. It is characterized by including a unit and a voice generation unit that synthesizes the voice signal based on the acoustic feature amount estimated by the acoustic feature amount estimation unit.

Further, the speech synthesizer according to claim 5 is a speech synthesizer that synthesizes a speech signal for an arbitrary text by using the time length model and the acoustic model learned by the learning device according to claim 3, with respect to the arbitrary text. The language analysis unit that performs language analysis processing and generates a context-dependent label and the time information related to the rhyme included in the context-dependent label generated by the language analysis unit are converted into relative values, and the relative values related to the rhyme are said. For the relative valuation unit that generates the context-dependent relative label including time information and the context-dependent relative label generated by the relative valuation unit, a process of applying a preset question group regarding the context is performed, and language features are performed. Using the context question group processing unit that generates a matrix, the language feature matrix generated by the context question group processing unit, and the time length model, the duration length of each phonetic element is estimated, and the duration of each phonetic element is estimated. A time length estimation unit that obtains the language feature amount from the length and the language feature matrix, and an acoustic feature amount estimation unit that estimates the acoustic feature amount using the language feature amount and the acoustic model obtained by the time length estimation unit. A voice generation unit that synthesizes the voice signal based on the acoustic feature amount estimated by the acoustic feature amount estimation unit, and the relative valuation unit provides time information related to the rhyme in the speech. Number and position of exhalation paragraphs, number and position of accent phrases in speech, number and position of beats in speech, number and position of accent phrases in exhalation paragraph, number and position of beats in exhalation paragraph, accent phrase Relative value of the number of exhalation paragraphs in the speech, relative value of the position, accent in the speech, with information on one or more of the position of the beat in the speech and the position of the beat from the accent nucleus in the accent phrase. Relative value of number of phrases and relative value of position, relative value of number of beats in the speech and relative value of position, relative value of number of accent phrases in the exhalation paragraph and relative value of position, in the exhalation paragraph Time information related to the rhyme among the relative value of the number of beats and the relative value of the position, the relative value of the position of the beat in the accent phrase, and the relative value of the position of the beat from the accent nucleus in the accent phrase. It is characterized in that the context-dependent relative label containing the one or a plurality of the relative values corresponding to is obtained.

Further, the program of claim 6 is characterized in that the computer functions as the learning device according to any one of claims 1 to 3.

Further, the program of claim 7 is characterized in that the computer functions as the speech synthesizer according to claim 4 or 5.

As described above, according to the present invention, it is possible to stably obtain a high-quality voice signal when synthesizing a voice of an arbitrary text.

It is a block diagram which shows the structure of the learning apparatus by embodiment of this invention. It is a flowchart which shows the pre-learning process of a learning apparatus. It is a figure explaining the language analysis process of step S201 of the language analysis part, and the data structure of a context-sensitive label. It is a figure which shows the form example of the context-sensitive label described in Non-Patent Document 2. It is a figure explaining the example of the relative value processing of the time information related to the prosody when generating the context-sensitive relative label. It is a figure which shows the example of the question group about the context described in Non-Patent Document 3. It is a figure explaining the context question group application processing of step S203 of the context question group processing part, and the data structure of the language feature matrix for each phoneme. It is a figure explaining the voice analysis process of step S204 of the voice analysis unit, and the data structure of the acoustic feature amount for each frame. It is a figure explaining the phoneme alignment process of step S205 of the correspondence part, and the data structure of the duration length for each phoneme. It is a figure explaining the language feature amount extraction process of step S206 of the association part, and the data structure of the language feature amount for each frame. It is a figure explaining the time length model learning process of step S207 of a learning part. It is a figure explaining the acoustic model learning process of step S208 of a learning part. It is a block diagram which shows the structure of the voice synthesis apparatus by embodiment of this invention. It is a flowchart which shows the voice synthesis processing of a voice synthesis apparatus. It is a figure explaining the voice generation processing of a voice generation part. It is a figure which shows the experimental result of the time length model. It is a figure which shows the experimental result of an acoustic model. It is explanatory drawing which shows the flow of the conventional pre-learning process described in Non-Patent Document 1. It is explanatory drawing which shows the flow of the conventional speech synthesis processing described in Non-Patent Document 1.

Hereinafter, embodiments for carrying out the present invention will be described in detail with reference to the drawings. In the present invention, for a context-dependent label generated by linguistic analysis of text, time information related to prosody is relativized to generate a context-sensitive relative label, and a context-sensitive question group is applied to the context-dependent relative level to apply a linguistic feature matrix. Is characterized in that.

As a result, in the pre-learning process, by converting the time information related to prosody into a relative value, it is possible to continuously distribute the time information related to the prosody within the range of possible numerical values in a predetermined element of the language feature matrix, resulting in a sparse state. It can be reduced to a certain extent. Then, a highly accurate statistical model (time length model and acoustic model) can be learned.

Further, by using such a time length model and an acoustic model in the speech synthesis processing, it is possible to improve the accuracy in estimating the duration time length and the acoustic feature amount for each phoneme. Further, since the elements of the language feature matrix are less likely to be outliers, errors are less likely to occur when estimating the duration length and the acoustic feature amount for each phoneme using the time length model and the acoustic model.

Therefore, a high-quality voice signal can be stably obtained when synthesizing arbitrary text.

Hereinafter, embodiments of the present invention will be described with respect to a learning device that pre-learns a time-length model and an acoustic model, and a speech synthesizer that performs speech synthesis using the time-length model and the acoustic model learned by the learning device. .. It is assumed that the speech signal handled by the learning device and the speech synthesizer is monaural, the sampling frequency is 48 kHz, and the number of bits is 16.

In the embodiment of the present invention, Japanese will be described as an object, but in other languages as well, a method of relativizing the time information related to the prosody included in the context-sensitive label can be applied.

[Learning device]
First, the learning device according to the embodiment of the present invention will be described. FIG. 1 is a block diagram showing a configuration of a learning device according to an embodiment of the present invention, and FIG. 2 is a flowchart showing a pre-learning process of the learning device.

The learning device 1 includes a storage unit 10 in which a voice corpus is stored, a language analysis unit 11, a relative value conversion unit 12, a context question group processing unit 13, a voice analysis unit 14, a correspondence unit 15, a learning unit 16, and a time length. A storage unit 17 in which a model is stored and a storage unit 18 in which an acoustic model is stored are provided.

(Voice corpus)
A preset voice corpus is stored in the storage unit 10. The voice corpus is composed of a preset text and a corresponding preset voice signal. For example, when using a phoneme balance 503 sentence created by ATR (Advanced Telecommunications Research Institute International, Inc.), the text and the audio signal read aloud are composed of 503 pairs. For example, of the 503 pairs, 493 pairs are used for training statistical models, 8 pairs are used for evaluation, and 5 pairs are used for testing. For the voice corpus, refer to the following documents.
Kenichi Iso, Takao Watanabe, Nao Kuwahara, "Design of Sentence Set for Speech Database", Sound Lecture (Spring), pp.89-90 (March 1988)

(Language Analysis Department 11)
The language analysis unit 11 reads each text of the voice corpus from the storage unit 10 and performs a known language analysis process on the text (step S201). Then, the language analysis unit 11 obtains each information such as phoneme information, accent information, part of speech information, accent phrase information, exhalation paragraph information, and total number information for each phoneme constituting the sentence, and generates a context-dependent label. The language analysis unit 11 outputs a context-dependent label for each phoneme to the relative value conversion unit 12.

FIG. 3 is a diagram illustrating the language analysis process of step S201 of the language analysis unit 11 and the data structure of the context-sensitive label. As shown in FIG. 3, the context-sensitive label is generated by the language analysis process in step S201 of FIG. 2, and is composed of phoneme information, accent information, part-of-speech information, accent phrase information, exhalation paragraph information, and total number information for each phoneme. Will be done. This example is a context-sensitive label when the text is "Twisted all the reality towards you."

As the language analysis process, for example, the morphological analysis process described below is used.
“MeCab: Yet Another Part-of-Speech and Morphological Analyzer”, Internet <URL: http://taku910.github.io/mecab/>
Further, as the language analysis process, for example, the dependency analysis process described below is used.
“CaboCha / Pumpkin: Yet Another Japanese Dependency Structure Analyzer”, Internet <URL: https://taku910.github.io/cabocha/>

(Context-sensitive label format)
The format of the context-sensitive label differs depending on the language, but in Japanese, for example, the format described in Non-Patent Document 2 described above can be used. FIG. 4 is a diagram showing a format example of the context-sensitive label described in Non-Patent Document 2.

In the context-sensitive label format shown in FIG. 4, the information about the phoneme currently being focused on is described in one line in the context-sensitive label. The context-dependent labels for each phoneme are five phonemes (p1 to p5) that combine the phoneme currently being focused on and the two adjacent phonemes before and after it, and the beat unit in the accent phrase to which the phoneme currently being focused belongs. Position and position in beat units from the accent nucleus (a1 to a3), part of the phoneme in the accent phrase to which the phoneme currently being focused on belongs, its utilization form and utilization type (c1 to c3), currently focusing on Part of morphology in the accent phrase adjacent to the accent phrase to which the phoneme belongs, its utilization form and utilization type (b1 to b3, d1 to d3), the number of beats of the accent phrase to which the phoneme currently of interest belongs, and the beat unit of the accent nucleus. Position and accent phrase type (whether questionable or not) and the position and beat unit of the accent phrase to which the currently focused phoneme belongs in the exhalation paragraph to which the currently focused phoneme belongs. Position (f1 to f8), the number of beats of the accent phrase adjacent to the accent phrase to which the currently focused phoneme belongs, the position of the accent core in beat units, and the accent adjacent to the accent phrase to which the currently focused phoneme belongs. Presence or absence of pauses between phrases (e1 to e5, g1 to g5), the number of accent phrases and beats in the exhalation paragraph to which the phoneme currently in focus belongs, and the exhalation paragraph to which the phoneme currently in interest belongs in the utterance. Position in exhalation paragraph unit, position in accent phrase unit and beat unit (i1 to i8), number of accent phrases and beats in exhalation paragraph adjacent to the exhalation paragraph to which the phoneme currently of interest belongs (h1, h2, j1) , J2), the number of exhaled paragraphs in the utterance, the number of accent phrases and the number of beats (k1 to k3), etc. are used.

A context-sensitive label is composed of information related to phonology and information related to prosody. The time information among the information related to prosody is the target of the relative value processing by the relative value unit 12. In the context-dependent label format shown in FIG. 4, the time information related to the rhythm is a1 to a3, f1, f2, f5 to f8, e1, e2, g1, g2, i1 to i8, h1, h2, j1, j2. Is.

(Relative value conversion unit 12)
The relative valuation unit 12 inputs a context-dependent label for each phoneme from the language analysis unit 11, and performs a time information relative valuation process for relativizing the time information related to prosody for the context-dependent label for each phoneme (step). S202). Then, the relative valuation unit 12 generates a context-dependent relative label including time information of the relative value related to prosody for each phoneme, and outputs the context-dependent relative label for each phoneme to the context question group processing unit 13. The relative value here is, for example, a real value in the range of 0 to 1.

As described above, the context-sensitive label is composed of information related to phonology and information related to prosody, and the information related to this prosody is an absolute integer value. On the other hand, the context-dependent relative label is also composed of information related to phonology and information related to prosody, but among the information related to this prosody, time information is a relative real value and information other than time information. Is an absolute integer value.

Here, in the prior art, the context question group application process (step S1802 or step S1902) is performed using the context-dependent label for each phoneme generated by the language analysis process (process of step S1801 of FIG. 18 or step S1901 of FIG. 19). ) Is performed. In the language feature matrix generated by this, the numerical value of the element corresponding to the absolute integer value of the context-sensitive label is also an absolute integer value.

In the pre-learning process, even if the values of the elements of the language feature matrix are within the range of continuous numerical values, the elements of the language feature matrix are sparse, so the statistical model should be learned with high accuracy. Can't. Therefore, in the speech synthesis process, the accuracy when estimating the duration length and the acoustic feature amount for each phoneme using such a statistical model becomes low. Further, when the value of the element of the language feature matrix becomes an outlier exceeding the range in which continuous numerical values can be taken, an error occurs in the estimation of the duration length and the acoustic feature amount for each phoneme. In this case, the quality of the synthesized voice signal may deteriorate, and the sound quality becomes unstable.

Therefore, in the embodiment of the present invention, the relative valuation unit 12 relativizes the time information related to prosody for the context-dependent label for each phoneme generated by the linguistic analysis process, and the relative value (relative) related to phonology. Changed to generate a context-sensitive relative label containing time information (real value). In the language feature matrix generated by this, the numerical value of the element corresponding to the relative real value of the context-sensitive relative label is also a relative real value.

In the pre-learning process, when the values of the elements of the language feature matrix are within the range of continuous numerical values, the elements of the language feature matrix are not in a sparse state. Therefore, the statistical model can be learned with high accuracy. Then, by using such a statistical model in the speech synthesis processing, a high-quality speech signal can be stably obtained.

(Context-sensitive relative label)
The relative valuation unit 12 relativizes the time information related to the prosody among the plurality of information constituting the context-sensitive label, and does not relativize the information that is not a relative numerical value (cannot be relativized), but the context. Generate dependent relative labels. Hereinafter, the relative value processing of the time information related to the prosody will be specifically described.

The information that cannot be converted into relative values is information other than the time information related to prosody, for example, the number of exhaled paragraphs, the number of accent phrases, and the number of beats (k1 to k3) in the utterance.

<1> When the position of the exhalation paragraph in the utterance is used as the time information related to the phoneme The relative value unit 12 currently pays attention to the plurality of information constituting the context-dependent label by the following formula. About the number of accent phrases and the number of beats in the exhalation paragraph to which the phoneme belongs, the position of the exhalation paragraph to which the phoneme currently of interest belongs in the utterance in the exhalation paragraph unit, the position in the accent phrase unit and the beat unit (i1 to i8) , Divide by the number of exhaled paragraphs, the number of accent phrases and the number of beats (k1 to k3) in the utterance, and obtain the relative values (I1 to I8). Relative values (I1, I2) indicate relative numbers (ratio), and relative values (I3 to I8) indicate relative positions.
[Number 1]
I1 ＝ i1 ／ k2
I2 ＝ i2 ／ k3
In ＝ in ／ k1 for n ＝ 3,4
In ＝ in ／ k2 for n ＝ 5,6
In ＝ in ／ k3 for n ＝ 7,8 ・ ・ ・ (1)

In the above equation (1), the relative valuation unit 12 divides the number of accent phrases (i1) in the exhalation paragraph to which the phoneme currently being focused on belongs by the number of accent phrases (k2) in the utterance, thereby relating to i1. Find the relative value (I1) of.

In the above equation (1), the relative value unit 12 divides the beat number (i2) in the exhalation paragraph to which the phoneme currently being focused belongs by the beat number (k3) in the utterance, thereby making it relative to i2. Find the value (I2).

FIG. 5 is a diagram illustrating an example of relative valuation processing of time information related to prosody when generating a context-sensitive relative label. In FIG. 5, the utterance is a sentence of a conversation uttered by a person. The exhalation paragraph is the utterance section of a breath, and corresponds to the paragraph when the sentence of the utterance is divided by the silence section. An accent phrase is a grammatical or semantic group containing up to one accent. A beat is a phrase unit of a sound having a certain time length, and is also called a mora. Phonemes are the smallest basic unit of speech in the linguistic sense.

For example, in the case of the utterance "all towards him in the future", the exhalation paragraphs are "in the future" and "all towards him". In addition, the accent phrases are "in the future," "all," "his," and "toward," and the beats are "sho," "u," "ra," "i," "ha," "ze," "n," and "n." They are "bu", "ka", "re", "no", "ho", "u", and "he".

Suppose that the phoneme you are currently focusing on is in the time length of the beat "ka" in the accent phrase "his". In this case, the exhalation paragraph to which the phoneme currently being focused belongs is "all toward him", and the accent phrases corresponding to the exhalation paragraph are "all", "his", and "toward". The beats corresponding to the exhaled paragraph are "ze", "n", "bu", "ka", "re", "no", "ho", "u", and "he".

Therefore, the number of accent phrases i1 = 3 in the exhalation paragraph to which the phoneme currently being focused on belongs, the number of accent phrases k2 = 4 in the utterance, and the number of beats i2 = 9 in the exhalation paragraph to which the phoneme currently being focused belongs. , The number of beats in the utterance k3 = 14. Therefore, the relative value I1 = i1 / k2 = 3/4 = 0.75 for i1 and the relative value I2 = i2 / k3 = 9/14 = 0.64 for i2.

Returning to the above equation (1), the relative valuation unit 12 determines the forward position (i3) of the exhalation paragraph to which the phoneme currently being focused belongs in the exhalation paragraph unit, and the number of exhalation paragraphs (k1) in the utterance. By dividing by, the relative value (I3) for i3 is obtained.

In the above equation (1), the relative value unit 12 divides the position (i4) in the opposite direction in the exhalation paragraph unit of the exhalation paragraph to which the phoneme currently being focused belongs by the number of exhalation paragraphs (k1) in the utterance. By doing so, the relative value (I4) for i4 is obtained.

In the above equation (1), the relative valuation unit 12 divides the forward position (i5) in the accent phrase unit of the exhalation paragraph to which the phoneme currently being focused belongs by the number of accent phrases (k2) in the utterance. By doing so, the relative value (I5) for i5 is obtained.

In the above equation (1), the relative value unit 12 divides the position (i6) in the opposite direction in the accent phrase unit of the exhalation paragraph to which the phoneme currently being focused belongs by the number of accent phrases (k2) in the utterance. By doing so, the relative value (I6) for i6 is obtained.

In the above equation (1), the relative value unit 12 divides the forward position (i7) in the beat unit of the exhalation paragraph to which the phoneme currently being focused belongs by the number of beats (k3) in the utterance. Then, find the relative value (I7) for i7.

In the above equation (1), the relative value unit 12 divides the position (i8) in the opposite direction in beat units of the exhalation paragraph to which the phoneme currently being focused belongs by the number of beats (k3) in the utterance. Then, find the relative value (I8) for i8.

In this way, the relative value conversion unit 12 uses the number of accent phrases and the number of beats in the exhalation paragraph to which the phoneme currently being focused on belongs as the time information related to the prosody, and the expiratory paragraph to which the phoneme currently being focused in the speech belongs. When the position in the exhaled paragraph unit, the position in the accent phrase unit and the position in the beat unit (i1 to i8) are used, the relative value (I1 to I8) is obtained by the above equation (1).

In addition, the relative value conversion unit 12 uses the following formula to determine the number of accent phrases and the number of beats of the exhalation paragraph adjacent to the exhalation paragraph to which the phoneme currently being focused belongs, among the plurality of information constituting the context-dependent label. For h1, h2, j1, j2), divide by the number of accent phrases and the number of beats (k2, k3) in the utterance, respectively, to obtain the relative values (H1, H2, J1, J2). Relative values (H1, H2, J1, J2) indicate relative numbers.
[Number 2]
H1 ＝ h1 / k2
H2 ＝ h2 / k3
J1 ＝ j1 ／ k2
J2 ＝ j2 / k3 ・ ・ ・ (2)

In the above equation (2), the relative value conversion unit 12 uses the number of accent phrases (k2) in the utterance to determine the number of accent phrases (h1) in the expiratory paragraph adjacent to the expiratory paragraph to which the phoneme currently being focused belongs. By dividing, the relative value (H1) for h1 is obtained.

In the above equation (2), the relative value unit 12 divides the beat number (h2) of the expiratory paragraph adjacent to the expiratory paragraph to which the phoneme currently being focused belongs by the beat number (k3) in the utterance. By doing so, the relative value (H2) for h2 is obtained.

In the above equation (2), the relative value conversion unit 12 divides the number of accent phrases (j1) in the expiratory paragraph adjacent to the expiratory paragraph to which the phoneme currently being focused belongs by the number of accents (k2) in the utterance. By doing so, the relative value (J1) for j1 is obtained.

In the above equation (2), the relative value unit 12 divides the beat number (j2) of the expiratory paragraph adjacent to the expiratory paragraph to which the phoneme currently being focused belongs by the beat number (k3) in the speech. Then, find the relative value (J2) for j2.

In this way, the relative value conversion unit 12 determines the number of accent phrases and the number of beats (h1, h2, j1, j2) of the expiratory paragraph adjacent to the expiratory paragraph to which the phoneme currently being focused belongs as time information related to the rhyme. When used, the relative values (H1, H2, J1, J2) are obtained by the above equation (2).

<2> When the position of the accent phrase in the exhalation paragraph is used as the time information related to the phoneme The relative value unit 12 currently pays attention to the plurality of information constituting the context-dependent label by the following formula. Regarding the number of beats of the accent phrase to which the phoneme belongs and the position (f1, f2) of the accent nucleus in beat units, the number of beats and the number of beats of the accent phrase (i2, f1) in the exhalation paragraph to which the phoneme currently of interest belongs, respectively. ) To find the relative values (F1, F2). Here, the accent nucleus refers to the part of the high syllable immediately before the sound becomes low. Relative value (F1) indicates a relative number, and relative value (F2) indicates a relative position.
[Number 3]
F1 ＝ f1 / i2
F2 ＝ f2 / f1 ・ ・ ・ (3)

In the above equation (3), the relative value unit 12 divides the beat number (f1) of the accent phrase to which the phoneme currently being focused belongs by the beat number (i2) in the exhalation paragraph to which the phoneme currently being focused belongs. By doing so, the relative value (F1) for f1 is obtained.

In the above equation (3), the relative value unit 12 determines the position (f2) of the accent nucleus in the accent phrase to which the phoneme currently being focused belongs in beat units, and the beat of the accent phrase to which the phoneme currently being focused belongs. The relative value (F2) for f2 is obtained by dividing by the number (f1).

As described above, when the relative value conversion unit 12 uses the number of beats of the accent phrase to which the phoneme currently being focused on and the position (f1, f2) of the accent nucleus in beat units as the time information related to the prosody, the above-mentioned Find the relative values (F1, F2) with Eq. (3).

In addition, the relative value conversion unit 12 uses the following formula to accent the accent phrase to which the phoneme currently being focused belongs in the exhalation paragraph to which the phoneme currently being focused belongs among the plurality of information constituting the context-dependent label. The position in phrase units and the position in beat units (f5 to f8) are divided by the number of accent phrases and beats (i1, i2) in the exhalation paragraph to which the phoneme of interest belongs, respectively, and the relative values (F5 to i2). Find F8). Relative values (F5 to F8) indicate relative positions.
[Number 4]
Fn ＝ fn ／ i1 for n ＝ 5,6
Fn ＝ fn ／ i2 for n ＝ 7,8 ・ ・ ・ (4)

In the above equation (4), the relative value unit 12 determines the forward position (f5) of the accent phrase to which the phoneme currently being focused belongs in the expiratory paragraph to which the phoneme currently being focused belongs in the accent phrase unit. The relative value (F5) for f5 is obtained by dividing by the number of accent phrases (i1) in the exhalation paragraph to which the phoneme currently being focused belongs.

In the above equation (4), the relative value unit 12 determines the position (f6) in the opposite direction in the accent phrase unit of the accent phrase to which the phoneme currently being focused belongs in the exhalation paragraph to which the phoneme currently being focused belongs. The relative value (F6) for f6 is obtained by dividing by the number of accent phrases (i1) in the exhalation paragraph to which the phoneme currently being focused belongs.

In the above equation (4), the relative value unit 12 currently determines the forward position (f7) of the accent phrase to which the currently focused phoneme belongs in the exhalation paragraph to which the currently focused phoneme belongs in beat units. The relative value (F7) for f7 is obtained by dividing by the number of beats (i2) in the exhalation paragraph to which the phoneme of interest belongs.

In the above equation (4), the relative value unit 12 currently determines the position (f8) in the beat unit of the accent phrase to which the currently focused phoneme belongs in the exhalation paragraph to which the currently focused phoneme belongs. The relative value (F8) for f8 is obtained by dividing by the number of beats (i2) in the exhalation paragraph to which the phoneme of interest belongs.

In this way, the relative value conversion unit 12 uses the position of the accent phrase to which the currently focused phoneme belongs in the exhalation paragraph to which the currently focused phoneme belongs and the beat unit as the time information related to the prosody. When the position (f5 to f8) of is used, the relative value (F5 to F8) is obtained by the above equation (4).

In addition, the relative value conversion unit 12 uses the following equation to calculate the number of beats of the accent phrase adjacent to the accent phrase to which the phoneme currently being focused belongs and the beat of the accent nucleus among the plurality of information constituting the context-dependent label. Regarding the position in the unit (e1, e2, g1, g2), the number of accent phrases and the number of beats of the accent phrase (i'2,) in the expiratory paragraph to which the accent phrase that is adjacent to the accent phrase to which the phoneme currently being focused belongs belongs. Divide by e1, i "2, g1) to find the relative values (E1, E2, G1, G2). The relative values (E1, G1) indicate relative numbers, and the relative values (E2, G2) are relative. Indicates a typical position.
[Number 5]
E1 ＝ e1 ／ i'2
E2 ＝ e2 ／ e1
G1 ＝ g1 / i ”2
G2 = g2 / g1 ・ ・ ・ (5)

In the above equation (5), the relative value conversion unit 12 sets the beat number (e1) of the accent phrase adjacent to the accent phrase to which the phoneme currently being focused on belongs to the accent phrase to which the phoneme currently being focused belongs. The relative value (E1) for e1 is obtained by dividing by the number of accent phrases (i'2) in the expiratory paragraph to which the adjacent forward accent phrases belong.

In the above equation (5), the relative value unit 12 is currently paying attention to the position (e2) of the accent nucleus of the accent phrase adjacent to the accent phrase to which the phoneme currently being focused belongs in beat units. The relative value (E2) for e2 is obtained by dividing by the number of beats (e1) of the adjacent accent phrase before the accent phrase to which the phoneme belongs.

In the above equation (5), the relative value unit 12 sets the beat number (g1) of the accent phrase adjacent to the accent phrase to which the phoneme currently being focused on belongs after the accent phrase to which the phoneme currently being focused belongs. The relative value (G1) for g1 is obtained by dividing by the number of accent phrases (i "2) in the exhalation paragraph to which the adjacent accent phrase belongs.

In the above equation (5), the relative value conversion unit 12 determines the position (g2) of the accent nucleus of the accent phrase adjacent to the accent phrase to which the phoneme currently of interest belongs in beat units. The relative value (G2) for g2 is obtained by dividing by the number of beats (g1) of the adjacent accent phrase after the accent phrase to which the belongs.

In this way, the relative value conversion unit 12 uses the number of beats and the accent of the accent phrase adjacent to the accent phrase to which the phoneme currently being focused on belongs among the plurality of information constituting the context-dependent label as the time information related to the prosody. When the position of the nucleus in beat units (e1, e2, g1, g2) is used, the relative value (E1, E2, G1, G2) is obtained by the above equation (5).

<3> When the position of the beat in the accent phrase and the relative position between the beat in the accent phrase and the accent nucleus (the position of the beat from the accent nucleus in the accent phrase) are used as the time information related to the rhyme. In the following equation, the conversion unit 12 uses the following formula to describe the position in beat units of the accent phrase to which the phoneme currently being focused belongs and the position in beat units from the accent nucleus among the plurality of information constituting the context-dependent label ( Divide a1 to a3) by the number of beats (f1) of the accent phrase to which the phoneme of interest belongs to obtain the relative value (A1 to A3). Relative values (A1 to A3) indicate relative positions.
[Number 6]
An ＝ an ／ f1 for n = 1,2,3 ・ ・ ・ (6)

In the above equation (6), the relative value unit 12 determines the position (a1) in beat units of the accent phrase to which the phoneme currently being focused belongs, and the number of beats (f1) of the accent phrase to which the phoneme currently being focused belongs. ) To find the relative value (A1) for a1.

In the above equation (6), the relative value unit 12 determines the forward position (a2) in beat units from the accent nucleus in the accent phrase to which the phoneme currently being focused belongs, to which the phoneme currently being focused belongs. The relative value (A2) for a2 is obtained by dividing by the number of beats (f1) of the accent phrase.

In the above equation (6), the relative value conversion unit 12 determines the position (a3) in the opposite direction in beat units from the accent nucleus in the accent phrase to which the phoneme currently being focused belongs, to which the phoneme currently being focused belongs. The relative value (A3) for a3 is obtained by dividing by the number of beats (f1) of the accent phrase.

In this way, the relative value conversion unit 12 determines the position in beat units and the position in beat units (a1 to a3) from the accent nucleus in the accent phrase to which the phoneme currently being focused belongs as time information related to prosody. When used, the relative values (A1 to A3) are obtained by the above formula (6).

Then, the relative valuation unit 12 generates a context-sensitive relative label as follows.
p1 ^ p2-p3 + p4 = p5 / A: A1 + A2 + A3
/ B: b1-b2_b3 / C: c1_c2 + c3 / D: d1 + d2_d3
/ E: E1_E2! E3_e4-e5 / F: F1_F2 # f3_f4 @ F5_F6 | F7_F8 / G: G1_G2% g3_g4_g5
/ H: H1_H2 / I: I1-I2 @ I3 + I4 & I5-I6 | I7 + I8 / J: J1_J2
Although the context-sensitive relative label is described by dividing it into four lines for convenience, it is actually described by one line for each phoneme. For details, refer to Non-Patent Document 2 described above.

(Contextual Question Group Processing Unit 13)
Returning to FIGS. 1 and 2, the context question group processing unit 13 inputs the context-dependent relative label for each phoneme from the relative value conversion unit 12, and applies the question group regarding the context to the context-dependent relative label for each phoneme. The context question group application process is performed (step S203). Then, the context question group processing unit 13 generates a language feature matrix for each phoneme, and outputs the language feature matrix for each phoneme to the association unit 15.

FIG. 6 is a diagram showing an example of a question group regarding the context described in Non-Patent Document 3. In the question group related to this context, one question is described per line, and for each question, the first question set type (“QS (Question Set): Question Set” or “CQS (Continuous Question Set):”: "Continuous value question set"), the second item label (label represented by the character string in ""), and the third item condition (condition represented by the character string in {}).

That is, the question is described by either the question set "QS" or the continuous value question set "CQS".

When the question set type is "QS", the value of the label of the second item is "1" when the data of the context-sensitive label matches the character string specified as the condition of the third item, and "1" when it does not match. 0 "is given. That is, the data obtained when the question set type is "QS" is binary data having a binary feature amount that takes one of two values.

As the condition of the third item, by connecting a plurality of character strings with ",", "1" is given when any of the character strings is matched, and "0" is given when none of the character strings are matched. The logical sum is defined.

When the question set type is "CQS", as the value of the label of the second item, when the data of the context-sensitive label matches the regular expression that extracts the numerical value of the character string specified as the condition of the third item, that numerical value. Is given, and "0" is given when they do not match. That is, the data obtained when the question set type is "CQS" is the numerical data of the continuous feature amount.

The context-related question group example shown in FIG. 6 is described in the conventional non-patent document 3, but the same question group example is also used in the embodiment of the present invention. That is, in the context question group processing unit 13, regarding the context-dependent relative label for each phonetic element, when the question set type is "QS", the data of the context-sensitive relative label is the condition of the third item as the value of the label of the second item. By giving "1" when the character string specified as is matched and "0" when it does not match, the binary data of the language feature matrix is obtained.

Further, regarding the context-dependent relative label for each phonetic element, the context-question group processing unit 13 sets the condition that the data of the context-dependent relative label is the third item as the value of the label of the second item when the question set type is "CQS". Extract the numerical value by the character string specified as. When the regular expression is matched, the numerical value is given, and when it does not match, "0" is given to obtain the numerical data of the language feature matrix. In this case, when the context-sensitive relative label data is time information related to phonology, a relative real value is obtained as the numerical data of the language feature matrix.

As described above, when the question set type is "CQS", in the prior art, when the data of the context-dependent label is the time information related to the phonology, the time information is an absolute integer value, and therefore the corresponding language feature. The numerical data of the matrix is also an absolute integer value. On the other hand, in the embodiment of the present invention, when the data of the context-dependent relative label is the time information related to the phonology, the time information is a relative real value, so that the numerical data of the corresponding language feature matrix is also It is a relative real value.

FIG. 7 is a diagram illustrating the context question group application process of step S203 of the context question group processing unit 13 and the data structure of the language feature matrix for each phoneme. The context question group processing unit 13 performs the process of step S203 of FIG. 2 by applying the question group related to the context prepared in advance to the context-dependent relative label, and obtains the language feature matrix for each phoneme.

Specifically, the context question group processing unit 13 applies all the question groups related to the context to each information related to one phonetic element described in one line of the context-dependent relative label, and data (binary) for all the question groups. Binary data of various features and numerical data of continuous features) are obtained. Then, the context question group processing unit 13 generates a language feature matrix for each phoneme.

_{For example, let N p} be the number of rows (phonic prime numbers) of the context-sensitive relative label data, _{and N b the} number of questions starting with "QS" and _{N c the} number of questions starting with "CQS", for a total of N _q (N q). It is assumed that the question group is composed of _{N q} = N _b + N _c). In this case, the context question group processing unit 13 obtains data for _{N q} questions for each row (phoneme) of the context-sensitive relative label. Then, the context question group processing unit 13 generates a matrix (N _p , N _{q ) having a total N q dimensional vector composed of N b} dimensional binary data and N _c dimensional numerical data as a language feature matrix. ..

In the context-related question group, it is assumed that the number of questions when the question set type is "QS" is N _b = 643, and the number of questions when the question set type is "CQS" is N _{c = 25.} In this case, the context question group processing unit 13 applies all the question groups to the information about one phoneme described in one line of the context-sensitive relative label, thereby performing 643 dimensional binary data and 25 dimensional numerical data. Ask for.

Then, the context question group processing unit 13 obtains 643 dimensional binary data and 25 dimensional numerical data for information on all the phonemes constituting the context-dependent relative label, respectively, and obtains a language feature matrix for each phonetic element (643 dimensional binary). Data + 25-dimensional numerical data) is generated.

Here, in the prior art, as shown in step S1802 of FIG. 18 and step S1902 of FIG. 19, a context-sensitive label containing numerical data of absolute integer values (for example, i1 = 1-49 in FIG. 4) is used. , A language feature matrix is generated. When the question set type is "CQS", the numerical data of the absolute integer value is obtained from the context-dependent label containing the numerical data of the absolute integer value, and the language feature matrix containing the numerical data of the absolute integer value. Is generated. As described above, the numerical data of this absolute integer value is not continuously distributed within the range of possible values, so that it is in a sparse state.

On the other hand, in the embodiment of the present invention, as shown in step S203 of FIG. 2 and step S1403 of FIG. 14 described later, relative real value numerical data (for example, real value in the range of 0 to 1, in FIG. 5). A language feature matrix is generated using, for example, a context-sensitive relative label containing I1 = 0.75). When the question set type is "CQS", the relative real-valued numerical data is obtained from the context-dependent relative label containing the relative real-valued numerical data, and the language feature including the relative real-valued numerical data. A matrix is generated. Since this relative real-valued numerical data is continuously distributed within the range of possible values, the degree of sparse state is reduced as compared with the prior art. As a result, it is possible to learn a time-length model and an acoustic model with high accuracy.

(Voice analysis unit 14)
Returning to FIGS. 1 and 2, the voice analysis unit 14 reads out each voice signal corresponding to each text of the voice corpus from the storage unit 10. Then, the voice analysis unit 14 cuts out a voice signal for each frame, performs a known voice (sound) analysis process on the voice signal for each frame, and obtains an acoustic feature amount consisting of predetermined information for each frame (step S204). The voice analysis unit 14 outputs the acoustic feature amount for each frame to the association unit 15.

As the voice analysis process, for example, the process described below is used.
“A high-quality speech analysis, manipulation and synthesis system”, Internet <URL: https://github.com/mmorise/World>
Further, for example, the audio signal processing described below is used.
“Speech Signal Processing Toolkit (SPTK) Version 3.11 December 25, 2017”, Internet <URL: http://sp-tk.sourceforge.net/>
“REFERENCE MANUAL for Speech Signal Processing Toolkit Ver. 3.9”

FIG. 8 is a diagram illustrating the voice analysis process of step S204 of the voice analysis unit 14 and the data structure of the acoustic feature amount for each frame. The voice analysis unit 14 reads out each voice signal of the voice corpus from the storage unit 10, and cuts out the voice signal having a frame length of 25 ms every 5 ms of a frame shift (step S801). Then, the voice analysis unit 14 performs acoustic analysis processing on the voice signal for each frame, and obtains the spectrum, the pitch frequency, and the aperiodic component (step S802).

The voice analysis unit 14 analyzes the spectrum with mer cepstrum to obtain the mer cepstrum coefficient MGC (step S803). Further, the voice analysis unit 14 obtains the voiced / unvoiced determination information VUV from the pitch frequency, logarithms the voiced sections of the pitch frequency, and interpolates the unvoiced and unvoiced sections using the information of the voiced sections before and after the pitch frequency. The pitch frequency LF0 is obtained (step S804). Further, the voice analysis unit 14 analyzes the aperiodic component by mer cepstrum to obtain the band aperiodic component BAP (step S805).

As a result, the mer cepstrum coefficient MGC, the voiced / unvoiced determination information VUV, the logarithmic pitch frequency LF0, and the band aperiodic component BAP can be obtained for each frame as the acoustic features of the static characteristics.

The voice analysis unit 14 calculates the first-order difference Δ of the mer-cepstrum coefficient MGC to obtain the first-order difference mer-cepstrum coefficient ΔMGC (step S806), ^{and calculates the second-order difference Δ 2} to calculate the second-order difference mer-cepstrum coefficient Δ ^2. Find the MGC (step S807).

Sound analysis unit 14 calculates a primary difference delta logarithmic pitch frequency LF0 seeking primary differential logarithmic pitch frequency DerutaLF0 (step S808), secondary calculates the secondary difference delta ² differential logarithmic pitch frequency delta ² LF0 is obtained (step S809).

Sound analysis unit 14 calculates a primary difference delta band aperiodic component BAP but a first differential band aperiodic component DerutaBAP (step S810), 2 calculates the second-order difference delta ^secondary differential band aperiodic The component Δ ² BAP is obtained (step S811).

Thus, as the acoustic feature quantity of the dynamic characteristics, for each frame, the primary difference mel-cepstrum coefficients DerutaMGC, 2-order differential mel-cepstrum coefficient delta ² MGC, primary differential logarithmic pitch frequency ΔLF0,2 order difference logarithmic pitch frequency delta ² LF0 primary difference band aperiodic component ΔBAP and secondary differential band aperiodic component delta ² BAP is obtained.

The acoustic features obtained in this way are the static characteristic merkepstrum coefficient MGC, the logarithmic pitch frequency LF0 and the band aperiodic component BAP, the first-order difference merkepstrum coefficient ΔMGC of the dynamic characteristics, and the first-order difference logarithm for each frame. Pitch frequency ΔLF0, primary difference band aperiodic component ΔBAP, secondary difference mer cepstrum coefficient Δ ² MGC, secondary difference logarithmic pitch frequency Δ ² LF0 and secondary difference band aperiodic component Δ ² BAP, and static characteristic voiced / It is composed of silent judgment information VUV. This acoustic feature is composed of 199-dimensional data.

(Association unit 15)
Returning to FIGS. 1 and 2, the association unit 15 inputs the language feature matrix for each phoneme from the context question group processing unit 13, and inputs the acoustic feature amount for each frame from the voice analysis unit 14.

The associating unit 15 uses a known phoneme alignment technique to perform time associative processing between the language feature matrix for each phoneme and the acoustic feature amount for each frame (step S205). Then, the associating unit 15 calculates at which time of the audio signal each phoneme of the language feature matrix constituting the text sentence is positioned (corresponding) in the acoustic feature amount, and obtains the duration length of each phoneme. ..

The association unit 15 outputs the language feature matrix for each phoneme and the duration time for each phoneme to the learning unit 16. The language feature matrix for each phoneme and the duration time for each phoneme are used for learning the time length model.

For this duration length, the time information in milliseconds (ms) is divided by a frame shift of 5 ms, and the value obtained in 5 ms frames is used.

As the phoneme alignment technique, for example, the speech recognition process described below is used.
"The Hidden Markov Model Toolkit (HTK)", Internet <URL: http://htk.eng.cam.ac.uk>
“The HTK Book (for HTK Version 3.4)”, Cambridge University Engineering Department, Internet <URL: www.seas.ucla.edu/spapl/weichu/htkbook/>

FIG. 9 is a diagram illustrating the phoneme alignment process of step S205 of the association unit 15 and the data structure of the duration length for each phoneme. The associating unit 15 uses a language feature matrix consisting of 668-dimensional data per phoneme and an acoustic feature amount of 199 dimensions per frame, and by temporally associating the phoneme alignment processing in step S205, for each phoneme. Find the duration length. Specifically, the association unit 15 generates time information consisting of a start frame number and an end frame number in the corresponding acoustic feature amount for each phoneme in the language feature matrix, and the time length (number of frames) of the phoneme. Is generated as the duration length (one-dimensional numerical data) for each phoneme.

Returning to FIGS. 1 and 2, the associating unit 15 performs a known language feature amount extraction process for the language feature matrix for each phoneme and the duration length for each phoneme (step S206), and corresponds to the acoustic feature amount. Find the language features for each frame. Then, the association unit 15 outputs the language feature amount for each frame and the acoustic feature amount for each frame to the learning unit 16. The language features for each frame and the acoustic features for each frame are used for learning the acoustic model.

FIG. 10 is a diagram illustrating the language feature amount extraction process of step S206 of the association unit 15 and the data structure of the language feature amount for each frame. The association unit 15 has a number of frames corresponding to the duration of the phoneme in the language feature matrix for the number of frames (processing unit of the acoustic feature amount) corresponding to the duration of the phoneme currently being focused on in the speech. And by adding four-dimensional time data expressing the position in the frame, the language feature amount for each frame is generated.

That is, the linguistic feature amount is 643 dimensional binary data and 25 dimensional numerical data of the linguistic feature line example time-associated with the acoustic feature amount for each frame, and 4 dimensional according to the duration length. It consists of time data. That is, the language features are composed of a total of 672-dimensional data for each of all the frames corresponding to the number of phonemes.

The associating unit 15 deletes the silent sections at the beginning and end of each sentence after the temporal associating processing of the language features and the acoustic features.

(Learning Department 16)
Returning to FIGS. 1 and 2, the learning unit 16 inputs the language feature matrix for each phoneme and the duration time for each phoneme from the association unit 15. Then, the learning unit 16 learns the time length model using the language feature matrix for each phoneme and the duration time length for each phoneme as learning data (step S207), and stores the time length model in the storage unit 17.

The learning unit 16 inputs the language feature amount for each frame and the acoustic feature amount for each frame from the association unit 15. Then, the learning unit 16 learns the acoustic model using the language feature amount for each frame and the acoustic feature amount for each frame as learning data (step S208), and stores the acoustic model in the storage unit 18. The learning unit 16 learns the time length model and the acoustic model by, for example, deep learning.

(Time length model)
The learning process of the time length model will be described. FIG. 11 is a diagram illustrating the time length model learning process of step S207 of the learning unit 16.

The learning unit 16 uses 668-dimensional language feature matrix data consisting of 643-dimensional binary data and 25-dimensional numerical data for each phonetic element expressing the text as input data of the time-length model, and has one-dimensional integer values. The data of the duration length (the number of frames in units of 5 ms) is treated as the output data of the time length model.

The learning unit 16 obtains the maximum value and the minimum value of all the data for each dimension of the data of the language feature matrix which is the input data and stores it in the storage unit 17, and stores all the data for each dimension. Normalize using the maximum and minimum values.

The learning unit 16 obtains the average value and standard deviation of all the data for the duration length data, which is the output data, and stores the data in the storage unit 17, and uses the average value and the standard deviation for each of the data. To standardize.

The learning unit 16 uses the normalized 668-dimensional data of the language feature matrix as input data and the standardized one-dimensional data of the duration length as output data for each phonetic element, and sets the time-length model in step S207. learn. Then, the learning unit 16 stores the learned time length model in the storage unit 17.

When learning the time-length model, the techniques described at the following sites are used.
"CSTR-Edinburgh / merlin", Internet <URL: https://github.com/CSTR-Edinburgh/merlin>
The same applies to the learning of the acoustic model described later.

(Acoustic model)
The learning process of the acoustic model will be described. FIG. 12 is a diagram illustrating the acoustic model learning process of step S208 of the learning unit 16.

The learning unit 16 uses an acoustic model of a 672-dimensional language feature amount consisting of 643-dimensional binary data, 25-dimensional numerical data, and 4-dimensional time data for each frame in units of 5 ms, which are time-associated with the acoustic feature amount. Treat as input data of. Further, the learning unit 16 handles 199-dimensional acoustic features for each frame in units of 5 ms as output data of the acoustic model.

The learning unit 16 obtains the maximum value and the minimum value of all the data for each dimension of the language feature amount data which is the input data and stores it in the storage unit 18, and stores all the data for each dimension. Normalize using the maximum and minimum values.

The learning unit 16 obtains the mean value and standard deviation of all the data for each dimension of the acoustic feature amount data, which is the output data, and stores it in the storage unit 18, and stores all the data for each dimension. Standardize using mean and standard deviation.

The learning unit 16 learns the acoustic model in step S208 by using the normalized 672-dimensional data of the language features as the input data and the standardized 199-dimensional data of the acoustic features as the output data for each frame. To do. Then, the learning unit 16 stores the learned acoustic model in the storage unit 18.

The acoustic feature amount, which is the output data of the acoustic model, is composed of 199-dimensional data extracted from the audio signal every 5 ms unit frame. Specifically, it has a static characteristic of 66 dimensions, which is a combination of a 60-dimensional mel cepstrum coefficient, a 1-dimensional logarithmic pitch frequency, and a 5-dimensional band aperiodic component, and a dynamic characteristic and a voiced characteristic obtained by first-order difference and second-order difference of the static characteristic. / Including the silent judgment value, the total is 199 dimensions.

With reference to FIGS. 11 and 12, the input layer of the time length model is 668 dimensions, the input layer of the acoustic model is 672 dimensions, the hidden layer of both models is 1024 dimensions × 6 layers, and the output layer of the time length model is one dimension. , The output layer of the acoustic model is composed of a 199-dimensional forward propagation type. The activation function in the hidden layer is the bicurve tangent function, the loss error function is the mean square error function, the number of mini-batch is 64, the number of epochs is 25, the stochastic gradient descent method as the learning coefficient optimization method, the start learning rate 0.002, After 10 epochs (epochs) are passed, the learning rate is exponentially attenuated for each epoch, and learning is performed by the error back propagation method. If the evaluation error does not decrease continuously for 5 epochs after 15 epochs, the process ends early.

As described above, according to the learning device 1 according to the embodiment of the present invention, the language analysis unit 11 reads each text of the speech corpus from the storage unit 10 and performs language analysis processing to generate a context-dependent label for each phoneme. To do.

The relative valuation unit 12 performs a time information relative valuation process for relativizing the time information related to prosody with respect to the context-dependent label for each phoneme, and generates a context-dependent relative label for each phoneme. The context question group processing unit 13 performs context question group application processing for applying a context-related question group to the context-dependent relative label for each phoneme, and generates a language feature matrix for each phoneme.

The voice analysis unit 14 reads out each voice signal corresponding to each text of the voice corpus from the storage unit 10, performs voice analysis processing on the voice signal for each frame, and obtains an acoustic feature amount for each frame.

The associating unit 15 temporally associates the language feature matrix for each phoneme with the acoustic feature amount for each frame by using the phoneme alignment technique, and obtains the duration length for each phoneme. Then, the associating unit 15 performs a language feature amount extraction process on the language feature matrix for each phoneme and the duration time for each phoneme, and obtains the language feature amount for each frame corresponding to the acoustic feature amount.

The learning unit 16 learns a time length model using the language feature matrix for each phoneme and the duration time for each phoneme as learning data, and the language feature amount for each frame and the acoustic feature amount for each frame as learning data, and the acoustic model. To learn.

Here, in the prior art, the time information related to prosody is treated as numerical data of absolute integer values, and a language feature matrix is generated using a context-dependent label including time information of absolute values related to prosody. .. Therefore, the numerical data of the time information of the integer value related to the prosody included in the language feature matrix is not continuously distributed within the range that can be taken, and is in a sparse state. Then, it is not possible to learn a highly accurate time-length model and an acoustic model, and when synthesizing arbitrary texts by using these statistical models, it is not possible to stably obtain a high-quality voice signal. It was.

In the embodiment of the present invention, the time information related to the prosody is converted into a relative value to obtain the numerical data of the relative decimal value, and the language feature matrix is created by using the context-dependent relative label including the time information of the relative value related to the prosody. I tried to generate it. Therefore, the numerical data of the time information of the relative value related to the prosody included in the language feature matrix is continuously distributed within the range that can be taken, and the degree of the sparse state is reduced as compared with the prior art. Therefore, it is possible to learn a highly accurate time-length model and an acoustic model, and it is possible to stably obtain a high-quality voice signal when synthesizing an arbitrary text by using these statistical models.

[Speech synthesizer]
Next, the voice synthesizer according to the embodiment of the present invention will be described. FIG. 13 is a block diagram showing a configuration of a voice synthesizer according to the embodiment of the present invention, and FIG. 14 is a flowchart showing a voice synthesis process of the voice synthesizer.

The speech synthesizer 2 stores a language analysis unit 20, a relative value conversion unit 21, a context question group processing unit 22, a time length estimation unit 23, an acoustic feature amount estimation unit 24, a voice generation unit 25, and a time length model. It includes a storage unit 17 and a storage unit 18 in which an acoustic model is stored. The time-length model stored in the storage unit 17 and the acoustic model stored in the storage unit 18 are models learned by the learning device 1 shown in FIG.

(Language Analysis Department 20-Contextual Question Group Processing Department 22)
The language analysis unit 20 inputs the text to be subjected to the speech synthesis processing, performs the same processing as the language analysis unit 11 shown in FIG. 1 (step S1401), and outputs the context-dependent label to the relative value conversion unit 21. ..

The relative valuation unit 21 inputs a context-sensitive label from the language analysis unit 20, performs the same processing as the relative valuation unit 12 shown in FIG. 1 (step S1402), and uses the context-dependent relative label as the context question group processing unit. Output to 22.

The context question group processing unit 22 inputs a context-dependent relative label from the relative value conversion unit 21, performs the same processing as the context question group processing unit 13 shown in FIG. 1 (step S1403), and performs a language feature matrix for each phoneme. Is output to the time length estimation unit 23.

(Time length estimation unit 23)
The time length estimation unit 23 inputs the language feature matrix for each phoneme from the context question group processing unit 22, and uses the language feature matrix for each phoneme and the time length model stored in the storage unit 17 to use the duration for each phoneme. Estimate the length (step S1404).

Specifically, the time length estimation unit 23 normalizes the language feature matrix for each phoneme using the maximum value and the minimum value of the input data of the time length model stored in the storage unit 17, and normalizes each phoneme. Performs an operation using a time-length model using the language feature matrix of. Then, the time length estimation unit 23 destandardizes the duration time length of each standardized phoneme, which is the output data, by using the average value and the standard deviation of the output data of the time length model stored in the storage unit 17. Find the duration of each original phoneme.

The time length estimation unit 23 uses the language feature matrix for each phoneme and the duration time length for each phoneme to perform the same language feature extraction processing as the association unit 15 shown in FIG. 1, and the language feature extraction process for each frame. Ask for. Then, the time length estimation unit 23 outputs the language feature amount for each frame to the acoustic feature amount estimation unit 24.

(Acoustic feature amount estimation unit 24)
The acoustic feature amount estimation unit 24 inputs the language feature amount for each frame from the time length estimation unit 23, and uses the language feature amount for each frame and the acoustic model stored in the storage unit 18 to use the acoustic feature amount for each frame. Is estimated (step S1405). The acoustic feature amount estimation unit 24 outputs the acoustic feature amount for each frame to the voice generation unit 25.

Specifically, the acoustic feature amount estimation unit 24 normalizes the language feature amount for each frame by using the maximum value and the minimum value of the input data of the acoustic model stored in the storage unit 18, and normalizes each frame. The linguistic features of are used as input data, and calculations are performed using an acoustic model. Then, the acoustic feature amount estimation unit 24 destandardizes the standardized acoustic feature amount for each frame, which is the output data, by using the average value and the standard deviation of the output data of the acoustic model stored in the storage unit 18. Obtain the amount of acoustic features for each original frame.

The acoustic features estimated in this way take discrete values for each frame. Therefore, the acoustic feature amount estimation unit 24 obtains a smooth value of the acoustic feature amount by calculating the maximum likelihood estimation or the moving average for the acoustic feature amount for each continuous frame.

(Speech generator 25)
The voice generation unit 25 inputs the acoustic feature amount for each frame from the acoustic feature amount estimation unit 24, and synthesizes the voice signal based on the acoustic feature amount for each frame (step S1406). Then, the voice generation unit 25 outputs a voice signal for the text to be voice-synthesized.

FIG. 15 is a diagram illustrating a voice generation process of the voice generation unit 25. Of the acoustic features for each frame input from the acoustic feature estimation unit 24, the voice generation unit 25 has a mercepstrum coefficient MGC for each frame, a logarithmic pitch frequency LF0, and a static characteristic acoustic feature that is a band aperiodic component BAP. Is selected (step S1501).

The voice generation unit 25 converts the mel cepstrum coefficient MGC into a mer cepstrum spectrum and obtains the spectrum (step S1502). Further, the voice generation unit 25 obtains the voiced / unvoiced determination information VUV from the logarithmic pitch frequency LF0, logarithms the voiced section of the logarithmic pitch frequency LF0, sets the unvoiced and unvoiced sections to zero, and obtains the pitch frequency (step S1503). .. In addition, the voice generation unit 25 transforms the band aperiodic component BAP into a merkepstrum spectrum to obtain the aperiodic component (step S1504).

The voice generation unit 25 continuously uses the spectrum for each frame obtained in step S1502, the pitch frequency for each frame obtained in step S1503, and the aperiodic component for each frame obtained in step S1504 to continuously perform the voice waveform. Is generated (step S1505), and an audio signal is output (step S1506).

As described above, according to the speech synthesizer 2 according to the embodiment of the present invention, the language analysis unit 20 performs language analysis processing on the text to be speech synthesis processing to generate a context-dependent label for each phoneme. ..

The relative valuation unit 21 performs a time information relative valuation process for relativizing the time information related to prosody with respect to the context-dependent label for each phoneme, and generates a context-dependent relative label for each phoneme. The context question group processing unit 22 performs a context question group application process for applying a context-related question group to the context-dependent relative label for each phoneme, and generates a language feature matrix for each phoneme.

The time length estimation unit 23 estimates the duration time length for each phoneme by using the language feature matrix for each phoneme and the time length model generated by the learning device 1. Then, the time length estimation unit 23 performs the language feature amount extraction process using the language feature matrix for each phoneme and the duration time length for each phoneme, and obtains the language feature amount for each frame.

The acoustic feature amount estimation unit 24 estimates the acoustic feature amount for each frame by using the language feature amount for each frame and the acoustic model generated by the learning device 1. The voice generation unit 25 synthesizes a voice signal based on the acoustic features for each frame.

Here, in the prior art, the time information related to prosody is treated as numerical data of absolute integer values, and a language feature matrix is generated using a context-dependent label including time information of absolute values related to prosody. .. For this reason, the numerical data of the time information of the integer value related to the prosody included in the language feature matrix is not continuously distributed within the possible range and becomes a sparse state, and a highly accurate time-length model and acoustic model are learned. Can't. When arbitrary text is voice-synthesized using the statistical model in this way, a high-quality voice signal cannot be stably obtained.

In the embodiment of the present invention, the time information related to the prosody is converted into a relative value to obtain the numerical data of the relative decimal value, and the language feature matrix is created by using the context-dependent relative label including the time information of the relative value related to the prosody. I tried to generate it. Therefore, the numerical data of the time information of the relative value related to the prosody included in the language feature matrix is continuously distributed within the range that can be taken, and the degree of the sparse state is reduced as compared with the prior art. Therefore, when an arbitrary text is voice-synthesized using the statistical model learned by the learning device 1, a high-quality voice signal can be stably obtained.

〔Experimental result〕
Next, experimental results by simulation for comparing the prior art with the embodiments of the present invention will be described. In the prior art of the experimental results described below, a language feature matrix is generated by utilizing the above-mentioned Non-Patent Document 2 and Non-Patent Document 3, and a time-length model and an acoustic model are learned.

The language feature matrix, which is the input data of the time-length model, is composed of a total of 668-dimensional data consisting of 643-dimensional binary data and 25-dimensional numerical data for each phonetic element expressing the text. The linguistic feature amount, which is the input data of the acoustic model, is 643 dimensional binary data and 25 dimensional numerical data of the linguistic feature matrix time-associated with the acoustic feature amount for each frame of 5 ms, and 4 dimensional time data. It is composed of a total of 672-dimensional data.

FIG. 16 is a diagram showing the experimental results of the time length model, and shows the objective evaluation value and the training error. Specifically, this figure shows the root mean square error (RMSE) and phase between the reference data and the estimated value for each of the predetermined Develop set and Test set with respect to the duration time length, which is the output data of the time length model. It shows the number of relationships (CORR).

In addition, in this figure, regarding the duration length, the error value of the Valid set at the time of early termination so as not to overfit during training (learning) and the error value of the Train set at that time are set as error values (Error). Shown. The smaller the RMSE and Error, the higher the evaluation, and the larger the CORR, the higher the evaluation.

Comparing the prior art with the embodiments of the present invention, the RMSE and Error of the embodiments of the present invention are both smaller than those of the prior art, and the CORR of the embodiments of the present invention is larger than those of the prior art. Therefore, in the embodiment of the present invention, the evaluation value is improved as compared with the conventional technique, the estimation accuracy of the duration length using the time length model is improved, and the effectiveness can be confirmed.

FIG. 17 is a diagram showing the experimental results of the acoustic model, and shows the objective evaluation value and the training error. Specifically, this figure shows the root mean square error (MCD) between the Melkeptrum reference data and the estimated value for each of the predetermined Develop set and Test set with respect to the acoustic features that are the output data of the acoustic model. Root mean square error (BAP) between reference data and estimates of band aperiodic components, root mean square error (RMSE) between reference data and estimates of pitch frequency F0, correlation coefficient (CORR) and voiced It shows the silent judgment error rate (VUV).

In addition, this figure shows the minimum error value of the Valid set at the time of early termination so as not to overfit during training (learning), and the error value of the Train set at that time as an error value (Error). ing. The smaller the MCD, BAP, RMSE, VUV and Error, the higher the evaluation, and the larger the CORR, the higher the evaluation.

Comparing the prior art with the embodiments of the present invention, the MCD, BAP, RMSE, VUV and Error of the embodiments of the present invention are all smaller than those of the prior art, and the CORR of the embodiments of the present invention is smaller than that of the prior art. Is also big. Therefore, in the embodiment of the present invention, the evaluation value is improved as compared with the conventional technique, the estimation accuracy of the acoustic feature amount using the acoustic model is improved, and the effectiveness can be confirmed.

Although the present invention has been described above with reference to embodiments, the present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the technical idea. In the above embodiment, the relative valuation unit 12 of the learning device 1 relativizes the time information related to the prosody for the context-dependent label for each phoneme, and generates a context-dependent relative label including the time information of the relative value related to the prosody. I tried to do it.

In this case, the relative valuation unit 12 may generate the context-dependent relative label for all the time information related to the prosody included in the context-dependent label, or may generate the time information related to a part of the prosody. As a target, a context-sensitive relative label may be generated. That is, the relative value conversion unit 12 relatives the time information related to one or a plurality of preset prosody among all the time information related to the prosody, and the relative value related to one or a plurality of prosody. You may want to generate a context-dependent relative label that contains the time information of. The same applies to the relative value unit 21 of the voice synthesizer 2.

As the hardware configuration of the learning device 1 and the speech synthesizer 2 according to the embodiment of the present invention, a normal computer can be used. The learning device 1 and the speech synthesizer 2 are composed of a computer provided with a volatile storage medium such as a CPU and RAM, a non-volatile storage medium such as a ROM, and an interface.

A storage unit 10 in which a voice corpus is stored, a language analysis unit 11, a relative value conversion unit 12, a context question group processing unit 13, a voice analysis unit 14, a correspondence unit 15, a learning unit 16, and time provided in the learning device 1. Each function of the storage unit 17 in which the long model is stored and the storage unit 18 in which the acoustic model is stored is realized by causing the CPU to execute a program describing these functions.

Further, the language analysis unit 20, the relative value conversion unit 21, the context question group processing unit 22, the time length estimation unit 23, the acoustic feature amount estimation unit 24, the voice generation unit 25, and the time length model provided in the speech synthesizer 2 are stored. Each function of the stored storage unit 17 and the storage unit 18 in which the acoustic model is stored is realized by causing the CPU to execute a program describing these functions.

These programs are stored in the storage medium, read by the CPU, and executed. In addition, these programs can be stored and distributed in storage media such as magnetic disks (floppy (registered trademark) disks, hard disks, etc.), optical disks (CD-ROM, DVD, etc.), semiconductor memories, etc., and can be distributed via a network. You can also send and receive.

1 Learning device 2 Speech synthesizer 10, 17, 18 Storage unit 11, 20 Language analysis unit 12, 21 Relative value conversion unit 13, 22 Contextual question group processing unit 14 Speech analysis unit 15 Correspondence unit 16 Learning unit 23 Time length estimation Part 24 Acoustic feature amount estimation part 25 Voice generation part

Claims (7)

In a learning device that learns a time-length model and an acoustic model used for voice synthesis based on the text and the voice signal in which the voice signal is set in advance to correspond to the text.
A language analysis unit that performs language analysis processing on the preset text and generates context-sensitive labels,
A relative valuation unit that relativizes the time information related to the prosody included in the context-dependent label generated by the language analysis unit and generates a context-dependent relative label including the time information of the relative value related to the prosody.
A context-sensitive question group processing unit that generates a language feature matrix by applying a preset question group related to the context to the context-dependent relative label generated by the relative value conversion unit.
A voice analysis unit that performs voice analysis processing on the voice signal corresponding to the preset text and obtains an acoustic feature amount, and a voice analysis unit.
The language feature matrix generated by the context question group processing unit is temporally associated with the acoustic feature amount obtained by the voice analysis unit, the duration length of each phoneme is obtained, and the continuation of each phoneme is performed. A mapping unit that obtains the language features from the time length and the language feature matrix, and
The language feature matrix generated by the context question group processing unit and the duration time length for each phoneme obtained by the association unit are used to learn the time length model, and the language obtained by the association unit. A learning unit that learns the acoustic model using the feature amount and the acoustic feature amount obtained by the voice analysis unit.
A learning device characterized by being equipped with. In the learning device according to claim 1,
The learning unit
A learning device characterized in that deep learning (DL) is performed on the time length model and the acoustic model. In the learning device according to claim 1,
The relative value conversion unit
The time information related to the rhyme is the number and position of the exhalation paragraph in the utterance, the number and position of the accent phrase in the utterance, the number and position of the beat in the utterance, the number and position of the accent phrase in the exhalation paragraph, and the exhalation paragraph. Information on the number and position of beats within, the position of beats within an accent phrase, and the position of beats from the accent nucleus in the accent phrase.
Relative values and positions of the number of exhaled paragraphs in the utterance, relative values and positions of the number of accent phrases in the utterance, relative values of the number of beats and positions in the utterance, Relative values and positions of the number of accent phrases in the exhalation paragraph, relative values and positions of the number of beats in the exhalation paragraph, relative values of the position of beats in the accent phrase, and the accent. A learning device for obtaining the context-dependent relative label including one or a plurality of the relative values corresponding to the time information related to the utterance among the relative values of the positions of beats from the accent nucleus in the phrase. .. In a speech synthesizer that synthesizes a speech signal for an arbitrary text by using the time length model and the acoustic model learned by the learning device of claim 1 or 2.
A language analysis unit that performs language analysis processing on the arbitrary text and generates context-sensitive labels,
A relative valuation unit that relativizes the time information related to the prosody included in the context-dependent label generated by the language analysis unit and generates a context-dependent relative label including the time information of the relative value related to the prosody.
A context-sensitive question group processing unit that generates a language feature matrix by applying a preset question group related to the context to the context-dependent relative label generated by the relative value conversion unit.
Using the language feature matrix and the time length model generated by the context question group processing unit, the duration length of each phoneme is estimated, and the language feature quantity is calculated from the duration time length of each phoneme and the language feature matrix. The time length estimation unit to be obtained and
An acoustic feature amount estimation unit that estimates an acoustic feature amount using the language feature amount and the acoustic model obtained by the time length estimation unit, and an acoustic feature amount estimation unit.
A voice generation unit that synthesizes the voice signal based on the acoustic feature amount estimated by the acoustic feature amount estimation unit, and a voice generation unit.
A voice synthesizer characterized by being equipped with. In a speech synthesizer that synthesizes a speech signal for an arbitrary text by using the time length model and the acoustic model learned by the learning device of claim 3.
A language analysis unit that performs language analysis processing on the arbitrary text and generates context-sensitive labels,
A relative valuation unit that relativizes the time information related to the prosody included in the context-dependent label generated by the language analysis unit and generates a context-dependent relative label including the time information of the relative value related to the prosody.
A context-sensitive question group processing unit that generates a language feature matrix by applying a preset question group related to the context to the context-dependent relative label generated by the relative value conversion unit.
Using the language feature matrix and the time length model generated by the context question group processing unit, the duration length of each phoneme is estimated, and the language feature quantity is calculated from the duration time length of each phoneme and the language feature matrix. The time length estimation unit to be obtained and
An acoustic feature amount estimation unit that estimates an acoustic feature amount using the language feature amount and the acoustic model obtained by the time length estimation unit, and an acoustic feature amount estimation unit.
A voice generation unit that synthesizes the voice signal based on the acoustic feature amount estimated by the acoustic feature amount estimation unit is provided.
The relative value conversion unit
The time information related to the rhyme is the number and position of the exhalation paragraph in the utterance, the number and position of the accent phrase in the utterance, the number and position of the beat in the utterance, the number and position of the accent phrase in the exhalation paragraph, and the exhalation paragraph. Information on the number and position of beats within, the position of beats within an accent phrase, and the position of beats from the accent nucleus in the accent phrase.
Relative values and positions of the number of exhaled paragraphs in the utterance, relative values and positions of the number of accent phrases in the utterance, relative values of the number of beats and positions in the utterance, Relative values and positions of the number of accent phrases in the exhalation paragraph, relative values and positions of the number of beats in the exhalation paragraph, relative values of the position of beats in the accent phrase, and the accent. Speech synthesis characterized in that the context-dependent relative label containing one or more of the relative values of the position of the beat from the accent nucleus in the phrase corresponding to the time information related to the rhyme is obtained. apparatus. A program for causing a computer to function as the learning device according to any one of claims 1 to 3. A program for causing a computer to function as the speech synthesizer according to claim 4 or 5.

Images