JP5722295B2 - Acoustic model generation method, speech synthesis method, apparatus and program thereof - Google Patents

Description

  The present invention relates to an acoustic model generation method and a speech synthesis method for generating an acoustic model used for an HMM (Hidden Markov Model) speech synthesis method, and an apparatus and a program thereof.

  In recent years, an HMM speech synthesis method has been proposed as a speech synthesis method (for example, Non-Patent Document 1). The acoustic model (speech database) in the HMM speech synthesis method holds one model for each synthesis unit as a parameter of speech data obtained by averaging the spectrum and F0 for each synthesis unit. As a result, even with a small amount of voice data, the voice feeling is low, but stable voice synthesis is possible.

  On the other hand, as disclosed in Non-Patent Document 2, it is known that the naturalness of synthesized speech is improved by introducing a tone coupling type between accent phrases.

Masuko et al., "HMM-based speech synthesis using dynamic features", Theory of Science, vol.J79-D-II, no.12, pp.2184-2190, Dec. 1996. Hakoda et al., `` Examination of Tone Combined Derivation Rules for Sentence Voices '', IEICE Tech.

  In the conventional HMM speech synthesis method, the tone combination type cannot be considered at the time of model learning and speech synthesis. However, since it is expensive to manually add a tone coupling type to voice data for model learning, HMM speech synthesizing methods that take into account the tone coupling type are rarely used.

  The present invention has been made in view of this problem, and an acoustic model generation method, a speech synthesis method, an apparatus, and a program capable of learning and generating an acoustic model automatically provided with a tone coupling type The purpose is to provide.

The acoustic model generation method of the present invention includes a model learning process, a tone coupling type extraction process, and a tone coupling type model learning process. In the model learning process, the speech synthesis HMM is learned by inputting the learning speech data including the pitch parameter and the spectrum parameter, and the phoneme segmentation information and the speech information including the accent information of the learning speech data. The tone combination type extraction process generates a speech parameter having the same phoneme segmentation information as speech information from the speech synthesis HMM, and uses the speech parameter and the parameter of the learning speech data to generate a pitch pattern between the accent phrases. Extract the tone combination type that affects the shape . In the tone coupled model learning process, the learning speech data, the utterance information, and the tone coupled type are input, and model learning considering the tone coupled type is performed to generate a tone coupled acoustic model.

  The speech synthesis method of the present invention includes a text analysis process, a speech parameter generation process, and a speech synthesis filter process. In the text analysis process, the speech synthesis target text is input, the text synthesis target text is analyzed, and text information including reading, accent, and tone combination types is output. In the speech parameter generation process, speech parameters are generated using the tone-coupled acoustic model generated by the above-described acoustic model generation method and text information. The speech synthesis filter process generates a speech waveform using the speech parameters.

  According to the acoustic model generation method of the present invention, it is possible to automatically generate an acoustic model to which a tone coupling type is assigned, so that the cost for realizing an HMM speech synthesis method considering the tone coupling type can be reduced. Can do.

  In addition, according to the speech synthesis method of the present invention, the synthesized sound is generated using the acoustic model that takes into account the tone combination type, so that the quality of the synthesized sound can be improved over the normal HMM speech synthesis.

The figure which shows the function structural example of the acoustic model production | generation apparatus 100 of this invention. The figure which shows the operation | movement flow of the acoustic model production | generation apparatus 100. FIG. The figure which shows the example of phoneme segmentation information. The figure which shows the example of HMM for speech synthesis of 3 states. The figure which shows the function structural example of the tone combination type | mold extraction part 20. FIG. The figure which shows the concept of the audio | voice parameter which the audio | voice parameter production | generation means 201 produces | generates. The figure which shows the concept between accent phrases. The figure which shows the operation | movement flow of the tone combination type | mold extraction part 20. FIG. The figure which shows the operation | movement flow of tone combination type | mold extraction part 20 '. The figure which shows the function structural example of the speech synthesizer 200 of this invention. The figure which shows the operation | movement flow of the speech synthesizer.

  Embodiments of the present invention will be described below with reference to the drawings. The same reference numerals are given to the same components in a plurality of drawings, and the description will not be repeated.

  FIG. 1 shows an example of the functional configuration of an acoustic model generation apparatus 100 according to the present invention. The operation flow is shown in FIG. The acoustic model generation apparatus 100 includes a model learning unit 10, a tone coupling type extraction unit 20, a tone coupling type model learning unit 30, and a control unit 40. The acoustic model generation apparatus 100 is realized by a predetermined program being read into a computer including, for example, a ROM, a RAM, a CPU, and the like, and the CPU executing the program.

  The model learning unit 10 learns the speech synthesis HMM by using the speech data for learning including the pitch parameter and the spectrum parameter, and the speech information including the phoneme segmentation information and the accent information of the speech data for learning (step). S10). The voice data for learning is data in which voices in which N speakers have spoken N sentences are recorded. The speech data for learning includes a pitch parameter (basic frequency: F0) and spectrum parameters (cepstrum, mel cepstrum, etc.) obtained as a result of performing signal processing on the speech signal.

  These parameters are data at predetermined time intervals called frames. One frame is a time (10 ms) consisting of a predetermined number (for example, 160) of audio signals obtained by converting the audio signals into discrete digital signals at a sampling frequency of 16 kHz, for example. Note that the speech data itself may be given to the model learning unit 10 and the parameters may be generated by digital signal processing.

  The utterance information is information given to each utterance in the speech data for learning, and includes phoneme segmentation information including at least the start time and end time information of each phoneme constituting the utterance, accent phrase boundary, accent type, accent And accent information such as phrase length. FIG. 3 shows an example of phoneme segmentation information. The first column in FIG. 3 is the phoneme name, the second column is the start time, and the third column is the end time. The start / end time is an elapsed time when the start point of each utterance is set to 0 [seconds].

The model learning unit 10 learns an HMM for speech synthesis from learning speech data and speech information. This HMM is a three-state or five-state left-to-right type HMM. FIG. 4 shows an example of a three-state speech synthesis HMM. 1 represents the start of the HMM in the first state. 2 represents the second state, and 3 represents the third state. The HMM is represented by a probability chain including self transitions a ₁₁ , a ₂₂ , a ₃₃ and state transitions a ₁₂ , a ₂₃ to the next state.

The HMM states 1, 2, and 3 are associated with output probability distributions b ₁ (o _t ), b ₂ (o _t ), and b ₃ (o _t ), respectively. The output probability distribution is a model parameter that characterizes sounds such as F0 and cepstrum. The model learning unit 10 learns the speech synthesis HMM using, for example, a Baum-Welch algorithm according to the phoneme label. The learned speech synthesis HMM is output to the tone combination type extraction unit 20. The learned speech synthesis HMM may be stored in the recording device as the speech synthesis HMM 50.

  The tone combination type extraction unit 20 generates a speech parameter having the same phoneme segmentation information as the speech information from the speech synthesis HMM, and uses the speech parameter and a parameter of learning speech data input from the outside as an accent. The tone combination type between phrases is extracted (step S20). Detailed operation of the tone combination type extraction unit 20 will be described later.

  The tone combination type model learning unit 30 performs model learning considering the tone combination type by inputting the learning voice data and speech information input from the outside and the tone combination type extracted by the tone combination type extraction unit 20 to perform tone combination. A type acoustic model is generated (step S30). The model learning in the tone coupled model learning unit 30 is different from the model learning in the model learning unit 10 in that learning is performed by adding the tone coupled type. The control unit 40 controls the time series operation of each unit.

  As described above, the acoustic model generation apparatus 100 can automatically extract the tone combination type from the learning speech data, and learn the speech synthesis HMM with the tone combination type added. Therefore, it is possible to provide an acoustic model considering the tone coupling type at a low cost.

  FIG. 5 shows a more specific functional configuration example of the tone coupling type extraction unit 20 which is a main part of the present invention, and the operation will be described in more detail. The tone combination type extraction unit 20 includes a sound parameter generation unit 201 and a tone combination type extraction unit 202.

  The speech parameter generation unit 201 receives the speech synthesis HMM generated by the model learning unit 10 and the utterance information input from the outside, and generates a speech parameter having the same phoneme segmentation information as the utterance information ( Step S201, FIG. 2). FIG. 6 shows the concept of audio parameters generated by the audio parameter generation unit 201.

  First, the number of frames in the sth state of the pth phoneme of utterance i is obtained. The number of frames in each state is calculated by equally dividing the duration of the p-th phoneme by the number of states. For example, the duration of the phoneme “o” shown in FIG. 3 is 150 [ms]. If the number of states of the speech synthesis HMM is three, for example, a time of 50 [ms] is allocated to each state. If one frame is 10 [ms], for example, each state is composed of 5 frames (third line in FIG. 6).

Next, the voice parameter generation unit 201 generates a voice parameter series of the utterance i by assigning an average vector μ _ps of model parameters to each frame (fourth line in FIG. 6). Finally, interpolation is performed on the speech parameter series. As disclosed in Non-Patent Document 1, the speech parameter interpolation is performed using the dynamic feature amount and variance of the model parameter. Note that a general interpolation method such as spline interpolation may be used. The tone combination extraction unit 202 uses a speech parameter sequence generated by the speech parameter generation unit 201 and learning speech data input from the outside. To extract the tone combination type between accent phrases. FIG. 7 shows the concept between accent phrases. For example, a sentence “Today is a meeting” is composed of three accent phrases “Today”, “Meeting”, and “I”. The position (time) of this accent phrase can be obtained by referring to the utterance information.

The tone combination type extraction unit 202 calculates ms _ij as an average value of logarithmic values of F0 around the boundary of the j-th and j + 1-th accent phrases of the i-th utterance of the speech parameter series generated by the speech parameter generation unit 201. Then, mo _ij is obtained as an average value of logarithmic values of F0 around the boundary of the same accent phrase of the speech data for learning. Then, the difference d is calculated, and the tone coupling type when the difference d is larger than the threshold value α is extracted as weak coupling, and the case where the difference d is small is extracted as strong coupling.

  In general, when the strength of coupling between accent phrases is small (strong coupling), F0 of learning speech data near the boundary between two accent phrases tends to be low, and when coupling strength is large (weak coupling) ), F0 near the boundary between two accent phrases tends to be high. On the other hand, F0 of the generated speech parameter series is generated from the speech synthesis HMM learned without considering the tone coupling type, so that F0 having an intermediate height without considering strong coupling or weak coupling is obtained. Generated. Therefore, tone coupling is performed as strong coupling when F0 of learning speech data is low (difference is small) compared to F0 of the speech parameter series generated by the speech parameter generation unit 201, and weak coupling when high (difference d is large). The type can be determined.

  The processing performed by the tone combination type extraction unit 20 in FIG. 7 will be described more specifically with reference to FIG. The horizontal axis in FIG. 7 is the elapsed time t [ms], and the vertical axis is F0 [Hz], indicating F0 of a certain accent boundary.

  The speech parameter generation unit 201 uses the speech synthesis HMM generated by the model learning unit 10 and utterance information input from the outside to convert all speech parameter sequences having the same phoneme segmentation information as the utterance information. An utterance i is generated (step S201a of loop S201, FIG. 8).

The tone combination type extraction unit 202 obtains ms _ij as an average value of logarithmic values of F0 around the boundary of the j-th and j + 1-th accent phrases of the i-th utterance of the speech parameter series, and the same accent of the speech data for learning The average value of logarithmic values of F0 around the phrase boundary is determined as mo _ij , and the difference d is calculated (step S202a of loop S202). The average value of F0 near the boundary between accent phrases is calculated using F0 before and after the boundary of the accent phrase (see ms _ij and mo _ij in FIG. 7).

  The tone combination type extraction unit 202 weakly connects the accent phrase if the difference d is greater than the threshold value α (step S202b), and strongly connects the accent phrase if the difference d is equal to or less than the threshold value α (step S202b ′). Is determined. This tone combination type extraction is performed for all accent phrases of all utterances.

[Modification 1]
The example in which the tone combination type is obtained from the average value of F0 around t [ms] before and after the boundary of the accent phrase has been described, but the tone is determined by the difference between the average values of F0 of all utterances constituting the jth and j + 1th accent phrases. The coupling type may be determined. The average value of the jth and j + 1th accent phrases of the i-th utterance of the speech parameter series is obtained as fs _ij and fs _{ij + 1} , and the average value of the same accent phrase of the speech data for learning is obtained as fo _ij and fo _{ij + 1} , and the speech parameter series The difference between ds and do when the difference between ds and fs _{ij + 1} is ds = fs _ij −fs _{ij + 1} and learning speech data difference do = fo _ij −fo _{ij + 1} is weakly coupled. A small case may be determined as strong coupling.

[Modification 2]
Although an example in which only two types of weak coupling and strong coupling are extracted as the tone coupling type has been described, any N types of tone coupling types can be extracted. FIG. 9 shows an operation flow of the tone combination type extraction unit 20 that extracts N types of tone combination types.

9 includes a plurality of threshold values α _i compared to FIG. 8, and the step S202b for determining the difference compares the difference d with each of the plurality of threshold values α ₁ , α ₂ ,..., Α _N−1. It is different in that it is discriminated into N coupled types. Thus, by classifying into two or more types of tone coupling types, it becomes possible to make the synthesized speech more natural.

[Speech synthesizer]
FIG. 10 shows a functional configuration example of the speech synthesizer 200 of the present invention. The operation flow is shown in FIG. The speech synthesizer 200 includes a text analysis unit 210, a speech parameter generation unit 220, a tone-coupled acoustic model 230, a speech synthesis filter unit 250, and a control unit 240. The speech synthesizer 200 is realized by reading a predetermined program into a computer composed of, for example, a ROM, a RAM, a CPU, and the like, and executing the program by the CPU.

  The text analysis unit 210 receives the speech synthesis target text, performs text analysis on the speech synthesis target text, and outputs text information composed of reading, accent, and tone combination types (step S210). The tone coupled acoustic model 230 is an acoustic model that has been model-trained in consideration of the tone coupled type generated by the acoustic model generating apparatus 100 described above.

  The voice parameter generation unit 220 generates a voice parameter using the tone coupled acoustic model 230 and text information (step S220). The speech synthesis filter unit 250 generates a speech waveform using the speech parameters output from the speech parameter generation unit 220 (step S250). Steps S210 to S250 are repeated until the processing for all texts is completed (step S240). The control unit 240 performs control for this repeated operation.

  According to the speech synthesizer 200 of the present invention, since the speech is synthesized based on the acoustic model that takes into account the tone combination type, the quality of the synthesized speech can be improved over the normal HMM speech synthesis.

  When the processing means in the above apparatus is realized by a computer, the processing contents of the functions that each apparatus should have are described by a program. Then, by executing this program on the computer, the processing means in each apparatus is realized on the computer.

  The program describing the processing contents can be recorded on a computer-readable recording medium. As the computer-readable recording medium, for example, any recording medium such as a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory may be used. Specifically, for example, as a magnetic recording device, a hard disk device, a flexible disk, a magnetic tape or the like, and as an optical disk, a DVD (Digital Versatile Disc), a DVD-RAM (Random Access Memory), a CD-ROM (Compact Disc Read Only) Memory), CD-R (Recordable) / RW (ReWritable), etc., magneto-optical recording media, MO (Magneto Optical disc), etc., semiconductor memory, EEP-ROM (Electronically Erasable and Programmable-Read Only Memory), etc. Can be used.

  This program is distributed by selling, transferring, or lending a portable recording medium such as a DVD or CD-ROM in which the program is recorded. Further, the program may be distributed by storing the program in a recording device of a server computer and transferring the program from the server computer to another computer via a network.

  Each means may be configured by executing a predetermined program on a computer, or at least a part of these processing contents may be realized by hardware.

Claims (7)

A model learning process for learning and generating a speech synthesis HMM by using as input speech data for learning including pitch parameters and spectrum parameters, and speech information including phoneme segmentation information and accent information of the speech data for learning; ,
A speech parameter having the same phoneme segmentation information as the utterance information is generated from the speech synthesis HMM, and the pitch pattern shape between the accent phrases is influenced using the speech parameter and the parameter of the learning speech data. Tone combination type extraction process to extract the tone combination type to give ,
As inputs and the training speech data and the speech information and the tone-linked, and tone-linked model learning process of generating a tonal binding acoustic model performs model learning considering the tone-linked,
An acoustic model generation method comprising: The acoustic model generation method according to claim 1,
The tone combination extraction process is
A speech parameter generation step for generating a speech parameter sequence having the same phoneme segmentation information as the speech information, using the speech synthesis HMM generated in the model learning process and the speech information as inputs;
Using the parameters of the speech parameter sequence and speech data for the learning, and the tone-linked extracting the tonal coupled between accent phrases,
A method for generating an acoustic model, comprising: Tone-coupled acoustic model generated by the acoustic model generation method according to claim 1 or 2,
A text analysis process in which the speech synthesis target text is input, the text synthesis target text is analyzed, and text information consisting of reading, accent, and tone combination type is output;
Using the tone-coupled acoustic model and the text information, a speech parameter generation process for generating speech parameters;
A speech synthesis filter process for generating a speech waveform using the speech parameters;
A speech synthesis method comprising: A model learning unit that learns and generates a speech synthesis HMM by using as input speech data for learning including pitch parameters and spectrum parameters, and speech information including phoneme segmentation information and accent information of the speech data for learning; ,
A speech parameter having the same phoneme segmentation information as the utterance information is generated from the speech synthesis HMM, and the pitch pattern shape between the accent phrases is influenced using the speech parameter and the parameter of the learning speech data. A tone combination type extraction unit for extracting a tone combination type to be given ;
As inputs and the training speech data and the speech information and the tone-linked, and tone-linked model learning unit for generating a tonal binding acoustic model performs model learning considering the tone-linked,
An acoustic model generation apparatus comprising: In the acoustic model generation device according to claim 4,
The tone combination type extraction unit is
Speech parameter generation means for generating a speech parameter sequence having the same phoneme segmentation information as the speech information, using the speech synthesis HMM generated by the model learning unit and the speech information as inputs;
Using the parameters of the speech parameter sequence and speech data for the learning, and the tone-linked extracting means for extracting the tonal coupled between accent phrases,
An acoustic model generation device comprising: Tone-coupled acoustic model generated by the acoustic model generation device according to claim 4 or 5,
A text analysis unit that receives the speech synthesis target text as input and outputs text information composed of reading, accent, and tone combination type by text analysis of the speech synthesis target text;
Using the tone-coupled acoustic model and the text information, a speech parameter generating unit that generates speech parameters;
A speech synthesis filter unit that generates a speech waveform using the speech parameters;
A speech synthesizer comprising:   A program for causing a computer to function as the acoustic model generation device according to claim 4 or the voice synthesis device according to claim 6.