JP6495781B2 - Voice parameter generation device, voice parameter generation method, program - Google Patents

Description

  The present invention relates to speech synthesis technology, and more particularly to learning hidden Markov models for speech synthesis from speech data.

  As a method for generating synthetic speech from speech data, there is a speech synthesis method based on Hidden Markov Model (HMM). In HMM speech synthesis, quality deterioration occurs due to excessive smoothing processing by statistical processing. As a method of suppressing this quality deterioration, there is a post filter technology based on a modulation spectrum (Non-Patent Document 1).

  The outline of the speech synthesis apparatus of Non-Patent Document 1 will be described below with reference to FIGS. 1 to 5. FIG. 1 is a block diagram showing the configuration of the speech synthesizer 9 of Non-Patent Document 1. As shown in FIG. FIGS. 2 and 4 are block diagrams showing configurations of a model learning unit 901 and a speech synthesis unit 902 which constitute the speech synthesis device 9 of Non-Patent Document 1, respectively. FIGS. 3 and 5 are flowcharts showing the operations of the model learning unit 901 and the speech synthesis unit 902 that constitute the speech synthesis device 9 of Non-Patent Document 1, respectively. As shown in FIG. 1, the speech synthesizer 9 of Non-Patent Document 1 includes a model learning unit 901, a speech synthesis unit 902, a speech data recording unit 903, and a context data recording unit 904. Further, as shown in FIG. 2, the model learning unit 901 includes an acoustic model learning unit 910, an acoustic model recording unit 911, an audio parameter generation unit 912, a modulation spectrum correction coefficient calculation unit 913, and a modulation spectrum correction coefficient recording unit. Including 914. As shown in FIG. 4, the speech synthesis unit 902 includes a text analysis unit 920, a speech parameter generation unit 921, a post filter unit 922, and a speech waveform generation unit 923.

  The voice data recording unit 903 records voice data of one speaker (hereinafter referred to as a target speaker) to be subjected to voice synthesis. In addition, the context data recording unit 904 records context data which is information related to an utterance included in the voice data (hereinafter referred to as utterance information). The voice data is generated from voice in which a target speaker utters a plurality of sentences, and is an acoustic feature value obtained as a result of performing signal processing on a voice (voice signal). Examples of acoustic feature quantities include pitch parameters such as fundamental frequency (F0), and spectral parameters such as cepstrum and mel cepstrum. The context data is utterance information that corresponds one-to-one with the utterance in the voice data. The speech information includes at least phoneme information (pronunciation information) and accent information (accent type, accent phrase length). In addition, the utterance information may further include part of speech information.

  The model learning unit 901 generates an acoustic model and a modulation spectrum correction coefficient from the speech data recorded in the speech data recording unit 903 and the context data recorded in the context data recording unit 904, and the acoustic model recording unit 911 and the modulation spectrum correction coefficient respectively The image is recorded in the recording unit 914. The acoustic model learning unit 910 learns an acoustic model from speech data and context data (S910). More specifically, the acoustic model is learned as a hidden Markov model, using speech data of the target speaker and context data that is speech information related to an utterance included in the speech data as learning data. The speech parameter generation unit 912 generates a speech parameter sequence from the acoustic model and context data learned in S910 (S912). The modulation spectrum correction coefficient calculation unit 913 calculates a modulation spectrum correction coefficient from the speech parameter sequence generated in S912 and the speech parameter sequence of natural speech generated from speech data of the speech data recording unit 903 (S913).

  The speech synthesis unit 902 generates a speech waveform from the text to be synthesized speech using the acoustic model recorded in the acoustic model recording unit 911 and the modulation spectrum correction coefficient recorded in the modulation spectrum correction coefficient recording unit 914. Output The text analysis unit 920 analyzes the text input to the speech synthesis unit 902, and generates context data such as reading of the text and an accent (S920). The speech parameter generation unit 921 generates a speech parameter sequence from the context data obtained in S920 and the acoustic model recorded in the acoustic model recording unit 911 (S921). The post filter unit 922 applies a post filter designed based on the modulation spectrum correction coefficient recorded in the modulation spectrum correction coefficient recording unit 914 to the speech parameter sequence generated in S 921 to obtain a post filter speech parameter sequence Generate (S922). The speech waveform generation unit 923 generates a speech waveform from the post-filtered speech parameter sequence, and outputs synthetic speech (S923). Note that the speech parameter generation unit 921 may have the same function as the speech parameter generation unit 912 constituting the model learning unit 901.

Takamichi Shinnosuke, Toda Tomoki, Graham Neubig, Sakriani Sakti, Nakamura Tetsu, "Post filter based on modulation spectrum in HMM speech synthesis", IEICE Technical Report, The Institute of Electronics, Information and Communication Engineers, November 2013, Vol. 113 No. 308 pp. 19-24.

  In HMM speech synthesis technology using a post filter based on a modulation spectrum of Non-Patent Document 1, in order to suppress quality degradation and obtain high quality speech, the likelihood of speech parameter series is captured from two different viewpoints, Based on the viewpoint, the speech parameter sequence is sequentially generated and converted to output synthetic speech. The details will be described below.

  First, from the viewpoint of static and dynamic features of speech parameters relative to context data, express the likelihood of speech parameter series, that is, generate the likelihood as an acoustic model, and maximize the likelihood of speech parameter series It generates (S921). Next, from the viewpoint of time-series direction fluctuation of speech parameters, the speech quality is improved by bringing the speech parameter series closer to the speech parameter series of natural speech. That is, by the post filter based on the modulation spectrum correction coefficient, it is converted into a speech parameter series (post-filtered speech parameter series) close to the speech parameter series of natural speech (S922).

  Here, in the process of S922, the speech parameter series is uniformly converted without considering the likelihood of the acoustic model. For this reason, the voice parameter sequence may be converted to an invalid value when viewed from the viewpoint of the static and dynamic features of the voice parameter for context data, and the naturalness of the voice may not be sufficiently improved. .

  Also, once the speech parameter sequence is generated, the post filter is applied again to convert the speech parameter sequence, so the amount of operation increases, and the text from the result speech synthesis target is input until the synthesized speech is obtained. There is also the problem of a delay in time.

  Therefore, according to the present invention, a speech parameter sequence close to natural speech is valid with a small amount of operation, from the viewpoints of static and dynamic characteristics of speech parameters with respect to context data and from the viewpoint of time-series direction fluctuation of speech parameters. An object of the present invention is to provide an audio parameter generation device for generating.

One aspect of the present invention is an acoustic model which is a hidden Markov model used for generating a speech parameter sequence, and a Gaussian distribution N (Ac; 0, σ ² I) as the distribution model of the speech parameter sequence (where A is the following equation) Lower triangular matrix, c is a speech parameter sequence, σ ² I is a linear prediction coefficient (a ₁ , a ₂ ,..., A _p ) when assuming that the Gaussian distribution covariance matrix (I is an identity matrix) From the context data on the text to be subjected to speech synthesis, a speech parameter series on the text is generated using the modulation spectrum model to be expressed.

  According to the present invention, it is possible to generate a speech parameter series close to natural speech with a small amount of calculation by generating a speech parameter series using an acoustic model and a modulation spectrum model simultaneously.

FIG. 2 is a block diagram showing the configuration of a speech synthesis device 9 of Non-Patent Document 1. FIG. 7 is a block diagram showing the configuration of a model learning unit 901 of Non-Patent Document 1. 10 is a flowchart showing an operation of a model learning unit 901 of Non-Patent Document 1. FIG. 10 is a block diagram showing the configuration of a speech synthesis unit 902 of Non-Patent Document 1. 11 is a flowchart showing the operation of the speech synthesis unit 902 of Non-Patent Document 1. FIG. 1 is a block diagram showing the configuration of a speech synthesizer 1 of a first embodiment. FIG. 2 is a block diagram showing the configuration of a model learning unit 101 according to the first embodiment. 6 is a flowchart showing the operation of the model learning unit 101 according to the first embodiment. FIG. 2 is a block diagram showing the configuration of a speech synthesis unit 102 according to the first embodiment. 6 is a flowchart showing the operation of the speech synthesis unit 102 according to the first embodiment.

  Hereinafter, embodiments of the present invention will be described in detail. Note that components having the same function will be assigned the same reference numerals and redundant description will be omitted.

<The point of the invention of the first embodiment>
In order to express viewpoints of static and dynamic features of speech parameters with respect to context data, an acoustic model that is also used in Non-Patent Document 1 is used. Also, in order to express the viewpoint of the time-series directional fluctuation of speech parameters, a modulation spectrum model described using a linear prediction model is used. Then, a speech parameter sequence that maximizes the likelihood of the acoustic model and the modulation spectrum model simultaneously is determined. The algorithm for obtaining this voice parameter sequence is configured as an algorithm that is an extension of the conventional voice parameter generation method (the technique described in Reference Non-Patent Document 3 described later). As a result, it becomes possible to generate a speech parameter sequence close to natural speech with a low amount of calculation, which is appropriate from the above two points of view, and it becomes possible to synthesize synthetic speech having higher naturality than ever before.

<Specific Description of Example 1>
The speech synthesizer according to the first embodiment will be described below with reference to FIGS. FIG. 6 is a block diagram showing the configuration of the speech synthesizer 1 of the first embodiment. FIGS. 7 and 9 are block diagrams showing configurations of the model learning unit 101 and the speech synthesis unit 102 that constitute the speech synthesis device 1 according to the first embodiment, respectively. FIGS. 8 and 10 are flowcharts showing the operations of the model learning unit 101 and the speech synthesis unit 102 that constitute the speech synthesis device 1 of the first embodiment, respectively. As shown in FIG. 6, the speech synthesizer 1 of the first embodiment includes a model learning unit 101, a speech synthesis unit 102, an acoustic model learning speech data recording unit 103, and an acoustic model learning context data recording unit 104. The modulation spectrum model training speech data storage unit 105 and the modulation spectrum model training context data storage unit 106 are included. Further, as shown in FIG. 7, the model learning unit 101 includes an acoustic model learning unit 110, an acoustic model recording unit 111, a modulation spectrum model learning unit 112, and a modulation spectrum model recording unit 113. As shown in FIG. 9, the speech synthesis unit 102 includes a text analysis unit 120, a speech parameter generation unit 121, and a speech waveform generation unit 122.

  The same data as the voice data recording unit 903 and the context data recording unit 904 of Non-Patent Document 1 is recorded in the sound model learning speech data recording unit 103 and the sound model learning context data recording unit 104. On the other hand, the modulation spectrum model training speech data storage unit 105 records modulation spectrum model speech training data, which is speech data used for modulation spectrum model training. The modulation spectrum model training speech data is used by the target speaker for acoustic model training by the target speaker in addition to all or part of the speech data of the same spoken sentence as that used for the acoustic model training speech data. Voice data of an utterance text different from that used as voice data, or voice data of an utterance text different from a speaker different from the target speaker may be included. Further, context data corresponding to the audio data of the modulation spectrum model training speech data storage unit 105 is recorded in the modulation spectrum model training context data storage section 106.

(Model learning unit 101)
In the model learning unit 101, the sound data for sound model learning 103, the context data for sound model learning 104, the modulation spectrum model sound for speech data recording unit 105, and the modulation spectrum model for context data recording 106 are recorded. From each data, two models, a model for speech synthesis of the target speaker, an acoustic model and a modulation spectrum model are generated. The acoustic model learning unit 110 learns an acoustic model from the voice data of the target speaker recorded in the acoustic model learning voice data recording unit 103 and the context data recorded in the acoustic model learning context data recording unit 104 (S110). The acoustic model learning unit 110 may be the same as the acoustic model learning unit 910 of Non-Patent Document 1.

  The modulation spectrum model learning unit 112 generates a modulation spectrum model from the voice data recorded in the modulation spectrum model learning voice data recording unit 105 and the context data recorded in the modulation spectrum model learning context data recording unit 106 (S112). Hereinafter, the expression of the modulation spectrum model and the modulation spectrum model generation method (learning algorithm) will be described.

(1) Expression of Modulation Spectrum Model In Non-Patent Document 1, the modulation spectrum is defined as the power spectrum of the speech parameter sequence. In Non-Patent Document 1, it is assumed that the main cause of quality deterioration of synthetic speech in HMM speech synthesis is that the high frequency component of the modulation spectrum is lower than that of natural speech. Is designed.

In the first embodiment, unlike the non-patent document 1, the modulation spectrum is not handled in the frequency domain but in the time domain. That is, it is assumed that the generation model of the speech parameter sequence in the temporal aspect is a distribution model defined by the following Gaussian distribution.

Here, T is the time length of one _{sentence, c = (c 1, c} 2, ..., c T) ( column speech parameters c _t at each time t) speech parameter sequence _{is, a = (a 1, a} 2 ,..., A _p ) are parameters representing the modulation spectrum model as a linear prediction model (hereinafter referred to as model parameters of the modulation spectrum model), P is a dimension of the linear prediction model, and σ ² is a dispersion parameter of Gaussian distribution. Also, A is a lower triangular matrix of T rows and T columns, and I is an identity matrix of T order.

  Since A is a lower triangular matrix of T rows and T columns as described above, the assumption about the generation model of the speech parameter sequence here is equivalent to assuming the generation model of the modulation spectrum as the all-pole model .

(2) Generation method of modulation spectrum model (learning algorithm)
The modulation spectrum model is learned by maximum likelihood estimation for speech parameter series c _n given from speech data recorded in modulation spectrum model training speech data storage unit 105 and context data recorded in modulation spectrum model training context data storage section 106 It does by solving the problem. The model parameter a ^ of the modulation spectrum model is obtained by the following equation.

となる。 In addition, if model parameter A ^ of a modulation | alteration spectrum model is expressed using the lower triangular matrix A of the above-mentioned T line T column like the above-mentioned (Formula 1),

It becomes.

This is an isomorphic problem of linear prediction model estimation. For example, by using the Levinson-Durbin-Itakura algorithm (Reference Non-Patent Document 2), a modulation spectrum model which is a solution of the maximum likelihood estimation problem is efficiently determined. be able to.
(Reference Non-Patent Document 2: N. Levinson, “The Wiener RMS (Root Mean Square) Error Criterion in Filter Design and Prediction”, J. Mathematical Phys., 25 (1947), pp. 261-278.)

In the maximum likelihood estimation of the linear prediction coefficients a = (a ₁ , a ₂ , ..., a _p ) described above, the modulation spectrum of the speech parameters of natural speech is modeled with an all-pole model to minimize Itakura-Saito divergence. It is known that it is equivalent to obtaining the conversion coefficient a = (a ₁ , a ₂ ,..., A _p ).

(Voice synthesis unit 102)
The speech synthesis unit 102 generates a speech waveform from the text to be synthesized speech using the acoustic model recorded in the acoustic model recording unit 111 and the modulation spectrum model recorded in the modulation spectrum model recording unit 113, and outputs a synthesized speech. Do. The text analysis unit 120 generates context data such as reading of the text and an accent from the text input to the speech synthesis unit 102 (S120). An algorithm used to generate context data may be the same as (Reference Non-Patent Document 3) or (Reference Non-Patent Document 4).
(Reference Non-Patent Document 3) Takashi Masuko, Keiichi Tokuda, Takao Kobayashi, Sei Imai, "HMM-based speech synthesis using dynamic features", Transactions of the Institute of Electronics, Information and Communication Engineers, The Institute of Electronics, Information and Communication Engineers, December 1996. Vol. J79-D-II No. 12 pp. 2184-2190.
(Reference Non-Patent Document 4) Sei Imai, Kazuo Sumita, Chieko Furuichi, "Mel log spectral approximation (MLSA) filter for speech synthesis", Transactions of the Institute of Electronics, Information and Communication Engineers, The Institute of Electronics, Information and Communication Engineers, February 1983, Vol. .J66-A No. 2 pp. 122-129.

  The speech parameter generation unit 121 generates a speech parameter sequence from the context data obtained in S120, the acoustic model recorded in the acoustic model recording unit 111, and the modulation spectrum model recorded in the modulation spectrum model recording unit 113. (S121).

The algorithm used to generate the speech parameter sequence is obtained as an extension of the speech parameter sequence generation algorithm of (Reference Non-Patent Document 3). In the algorithm of (Reference Non-Patent Document 3), a speech parameter series c ⁺ to be generated is obtained by the following equation.

Where λ is the parameter of the acoustic model as a hidden Markov model, w is the input text, W is a matrix representing the time difference of the speech parameter sequence c = (c ₁ , c ₂ , ..., c _T ), μ _q , Σ _q is an average parameter sequence of Gaussian distribution obtained from λ and w, and a covariance matrix.

すなわち、

である。ただし、μ_q、Σ_qはλ^(a)とw^から得られるガウス分布の平均パラメータ系列、共分散行列、Aは変調スペクトルモデルのモデルパラメータa^に対応する下三角行列であるとし、

とおく。 Similarly, in the speech parameter series c ^ determined in S121, the acoustic model recorded in the acoustic model recording unit 111 is λ ^(a) , and the model parameters of the modulation spectrum model recorded in the modulation spectrum model recording unit 113 are a. ^, The text input to the speech synthesis unit 102 is obtained as w ^ and the following objective functions are simultaneously optimized.

That is,

It is. Where μ _q and _{q q} are the mean parameter sequence of the Gaussian distribution obtained from λ ^(a) and w ^, the covariance matrix, and A is the lower triangular matrix corresponding to the model parameter a ^ of the modulation spectrum model,

far.

  The first factor of the objective function corresponds to the acoustic model likelihood, and the second factor corresponds to the modulation spectrum model likelihood, and by maximizing the product of both, the naturalness of the speech viewed from two viewpoints is maximized simultaneously can do.

が得られる。 Similar to the algorithm of (Reference non-patent document 3), speech parameter sequence

Is obtained.

  The speech waveform generation unit 122 generates a speech waveform from the speech parameter sequence c ^ generated in S121, and outputs a synthesized speech (S122). An algorithm used for speech waveform generation may be the same as (Reference Non-Patent Document 3) or (Reference Non-Patent Document 4).

  According to the speech synthesizer 1 of the present embodiment, the speech parameter sequence that maximizes the product of the likelihood of the acoustic model and the likelihood of the modulation spectrum model is determined. It is possible to generate a valid speech parameter sequence from two points of view: the feature point of view and the viewpoint of time-series direction variation of speech parameters. As a result, it is possible to generate a speech parameter sequence close to natural speech with a low amount of calculation.

<Supplementary Note>
The apparatus according to the present invention is, for example, an input unit to which a keyboard or the like can be connected, an output unit to which a liquid crystal display or the like can be connected as a single hardware entity, or a communication device (for example, communication cable) capable of communicating outside the hardware entity. Communication unit that can be connected, CPU (central processing unit, cache memory, registers, etc. may be provided), RAM or ROM that is memory, external storage device that is hard disk, input unit for these, output unit, communication unit , CPU, RAM, ROM, and a bus connected so as to enable exchange of data between external storage devices. If necessary, the hardware entity may be provided with a device (drive) capable of reading and writing a recording medium such as a CD-ROM. Examples of physical entities provided with such hardware resources include general purpose computers.

  The external storage device of the hardware entity stores a program necessary for realizing the above-mentioned function, data required for processing the program, and the like (not limited to the external storage device, for example, the program is read) It may be stored in the ROM which is a dedicated storage device). In addition, data and the like obtained by the processing of these programs are appropriately stored in a RAM, an external storage device, and the like.

  In the hardware entity, each program stored in the external storage device (or ROM etc.) and data necessary for processing of each program are read into the memory as necessary, and interpreted and processed appropriately by the CPU . As a result, the CPU realizes predetermined functions (each component requirement expressed as the above-mentioned,...

  The present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the spirit of the present invention. Further, the processing described in the above embodiment may be performed not only in chronological order according to the order of description but also may be performed in parallel or individually depending on the processing capability of the device that executes the processing or the necessity. .

  As described above, when the processing function in the hardware entity (the apparatus of the present invention) described in the above embodiment is implemented by a computer, the processing content of the function that the hardware entity should have is described by a program. Then, by executing this program on a computer, the processing function of the hardware entity is realized on the computer.

  The program describing the processing content can be recorded in a computer readable recording medium. As the computer readable recording medium, any medium such as a magnetic recording device, an optical disc, a magneto-optical recording medium, a semiconductor memory, etc. may be used. Specifically, for example, as a magnetic recording device, a hard disk device, a flexible disk, a magnetic tape or the like as an optical disk, a DVD (Digital Versatile Disc), a DVD-RAM (Random Access Memory), a CD-ROM (Compact Disc Read Only) Memory), CD-R (Recordable) / RW (Rewritable), etc. as magneto-optical recording medium, MO (Magneto-Optical disc) etc., as semiconductor memory EEP-ROM (Electronically Erasable and Programmable Only Read Memory) etc. Can be used.

  Further, this program is distributed, for example, by selling, transferring, lending, etc. a portable recording medium such as a DVD, a CD-ROM or the like in which the program is recorded. Furthermore, this program may be stored in a storage device of a server computer, and the program may be distributed by transferring the program from the server computer to another computer via a network.

  For example, a computer that executes such a program first temporarily stores a program recorded on a portable recording medium or a program transferred from a server computer in its own storage device. Then, at the time of execution of the process, the computer reads the program stored in its own recording medium and executes the process according to the read program. Further, as another execution form of this program, the computer may read the program directly from the portable recording medium and execute processing according to the program, and further, the program is transferred from the server computer to this computer Each time, processing according to the received program may be executed sequentially. In addition, a configuration in which the above-described processing is executed by a so-called ASP (Application Service Provider) type service that realizes processing functions only by executing instructions and acquiring results from the server computer without transferring the program to the computer It may be Note that the program in the present embodiment includes information provided for processing by a computer that conforms to the program (such as data that is not a direct command to the computer but has a property that defines the processing of the computer).

  Further, in this embodiment, the hardware entity is configured by executing a predetermined program on a computer, but at least a part of the processing content may be realized as hardware.

Claims (6)

An acoustic model which is a hidden Markov model used to generate a speech parameter series, and a Gaussian distribution N (Ac; 0, σ ² I) as the distribution model of the speech parameter series (where A is a lower triangular matrix of the following formula, c is A speech parameter sequence, σ ² I is a modulation spectrum model represented by linear prediction coefficients (a ₁ , a ₂ ,..., A _p ) when assuming a covariance matrix (I is a unit matrix) of the Gaussian distribution. A speech parameter generation unit that generates a speech parameter sequence related to the text from context data related to the text to be subjected to speech synthesis;

Voice parameter generator including: The voice parameter generation device according to claim 1, wherein
The Gaussian distribution corresponding to the acoustic model is N (Wc; μ _q , _{q q} ) (where c is a speech parameter sequence, W is a matrix representing the time difference of the speech parameter sequence c, and μ _q and _{q q} are each) The mean parameter sequence of the Gaussian distribution, the covariance matrix),
The speech parameter sequence c ^ generated by the speech parameter generation unit is

A voice parameter generator that is The voice parameter generation device according to claim 1 or 2
An acoustic model learning unit that learns the acoustic model using, as learning data, first voice data and first context data that is speech information related to an utterance included in the first voice data;
Model parameter A of the modulation spectrum model obtained by the following equation using the second speech data and the second context data which is the speech information related to the utterance included in the second speech data as learning data Modulation spectrum model learning unit to learn as

(Where c _n is a voice parameter sequence derived from the second voice data and the second context data)
An audio parameter generator further comprising: An acoustic model which is a hidden Markov model used to generate a speech parameter series, and a Gaussian distribution N (Ac; 0, σ ² I) as the distribution model of the speech parameter series (where A is a lower triangular matrix of the following formula, c is A speech parameter sequence, σ ² I is a modulation spectrum model represented by linear prediction coefficients (a ₁ , a ₂ ,..., A _p ) when assuming a covariance matrix (I is a unit matrix) of the Gaussian distribution. A speech parameter generating step of generating a speech parameter series related to the text from context data related to the text to be subjected to speech synthesis;

Voice parameter generation method including: The speech parameter generation method according to claim 4 , wherein
The Gaussian distribution corresponding to the acoustic model is N (Wc; μ _q , _{q q} ) (where c is a speech parameter sequence, W is a matrix representing the time difference of the speech parameter sequence c, and μ _q and _{q q} are each) The mean parameter sequence of the Gaussian distribution, the covariance matrix),
The speech parameter sequence c ^ generated in the speech parameter generation step is

Voice parameter generation method. A program for causing a computer to function as the voice parameter generation device according to any one of claims 1 to 3 .

Images