JP2022081691A - Voice synthesis device and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a voice synthesis device and a program capable of obtaining high quality synthesized voice signals when generating synthesized voice signals in which reading of specific portions of a text is adjusted.

SOLUTION: A language analysis unit 20 of a voice synthesis device 2 linguistically analyzes a text to be voice-synthesized to obtain a language feature amount, and an adjustment amount addition unit 21 adds adjustment amount information of adjustment parameters to the language feature amount. An acoustic feature amount estimation unit 22 estimates an acoustic feature amount using a statistical model learned in advance based on the language feature amount to which the adjustment amount information is added. A voice generation unit 23 synthesizes a voice signal based on the acoustic feature amount, and outputs the voice signal adjusted by the adjustment parameters to the text.

SELECTED DRAWING: Figure 10

COPYRIGHT: (C)2022,JPO&INPIT

Description

The present invention relates to a speech synthesizer and a program that synthesizes speech signals using a statistical model for synthesizing speech signals from text.

Conventionally, as a method of learning a statistical model using a text and a voice signal corresponding to the text and obtaining a synthetic voice for an arbitrary text, deep learning (DL: Deep Learing) using a deep neural network (DNN) is used. ) Is known (see, for example, Non-Patent Document 1).

On the other hand, as a method of adjusting how to read out a voice signal, a technique based on a voice analysis generation process is known (see, for example, Non-Patent Document 2).

FIG. 15 is an explanatory diagram showing a conventional learning method and synthesis method described in Non-Patent Document 1. A learning device that realizes this learning method uses a text of a voice corpus prepared in advance and a voice signal corresponding thereto, and extracts a language feature amount of the text by a language analysis process (step S1501). Further, the learning device extracts an acoustic feature amount by voice analysis processing for the voice signal (step S1502).

The learning device performs time correspondence between the language features and the acoustic features (step S1503), and learns the statistical model using the language features and the acoustic features (step S1504).

Further, the speech synthesizer that realizes this synthesis method inputs an arbitrary text and extracts a language feature amount by a language analysis process of the text (step S1505). Then, the voice synthesizer estimates the acoustic feature amount from the language feature amount using the statistical model learned by the learning device (step S1506), and obtains the voice signal waveform from the acoustic feature amount by the voice generation process (step S1506). S1507). This makes it possible to obtain a synthetic speech signal corresponding to any text.

FIG. 16 is an explanatory diagram showing a conventional audio signal adjusting method described in Non-Patent Document 2. The voice adjustment device that realizes this voice signal adjustment method extracts the acoustic feature amount for each frame from the voice signal by voice analysis processing (step S1601), and obtains a desired portion of the acoustic feature amount based on the adjustment parameter. (Step S1602).

The voice adjusting device generates a voice signal from the acoustic feature amount for each frame to which the voice is adjusted by the voice generation process (step S1603). This makes it possible to obtain an audio signal with adjustments.

Zhizheng Wu, Oliver Watts, Simon King, “Merlin: An Open Source Neural Network Speech Synthesis System”, in Proc. 9th ISCA Speech Synthesis Workshop (SSW9), September 2016, Sunnyvale, CA, USA. M. Morise, F. Yokomori, and K. Ozawa, “WORLD: a vocoder-based high-quality speech synthesis system for real-time applications”, IEICE transactions on information and systems, vol. E99-D, no, 7, pp. 1877-1884, 2016

For example, when a synthetic voice signal is used for producing content such as a broadcast program, a synthetic voice signal adjusted for reading a specific part of a text may be required as an effect.

The above-mentioned method of Non-Patent Document 1 obtains a synthetic speech signal for an arbitrary text, and always obtains the same synthetic speech signal for the same text. Further, the above-mentioned method of Non-Patent Document 2 is for adjusting how to read out an audio signal.

Therefore, it is assumed that the above-mentioned non-patent documents 1 and 2 are combined as a method of obtaining a synthetic speech signal in which the reading method of a specific part of the text is adjusted.

FIG. 17 is an explanatory diagram showing a hypothetical example in which the prior arts of Non-Patent Documents 1 and 2 are combined. The learning method of this assumed example is the same as steps S1501 to S1504 shown in FIG. 15 (steps S1701 to S1704).

In the synthesis method of this assumed example, the process of step S1602 shown in FIG. 16 is inserted into the process of steps S1505 to S1507 shown in FIG. Specifically, the speech synthesizer extracts language features from arbitrary text (step S1705) and estimates acoustic features from language features using a statistical model (step S1706).

The speech synthesizer makes a desired adjustment to a desired portion of the acoustic feature amount based on the adjustment parameter (step S1707). The voice synthesizer generates a voice signal from the adjusted acoustic feature amount for each frame by the voice generation process (step S1708). This makes it possible to obtain a synthetic speech signal corresponding to any text.

However, in this assumed example, the acoustic features estimated from the language features using the statistical model in step S1706 are different from the acoustic features extracted from the actual voice signal by the voice analysis process, and are smoothed in time. Has the characteristics that have been made. Therefore, if the acoustic features estimated using the statistical model in step S1707 are adjusted and the synthesized speech signal is obtained from the adjusted acoustic features for each frame in step S1708, the sound quality of the synthesized speech signal deteriorates. It will occur.

As described above, in the assumed example shown in FIG. 17, there is a problem that a high-quality synthetic voice signal cannot be obtained. Therefore, a new method has been desired in order to obtain a high-quality synthetic speech signal in which the reading method of a specific part of the text is adjusted.

Therefore, the present invention has been made to solve the above-mentioned problems, and an object thereof is to obtain a high-quality synthetic speech signal when generating a synthetic speech signal in which a reading method of a specific part of a text is adjusted. It is an object of the present invention to provide a speech synthesizer and a program capable of the above.

In order to solve the above-mentioned problem, the speech synthesizer according to claim 1 has a language analysis unit that linguistically analyzes the text to be voice-synthesized and obtains a language feature amount, and the language feature amount obtained by the language analysis unit. A statistical model learned in advance based on the adjustment amount addition part for adding the adjustment amount information of the adjustment parameter for adjusting the acoustic feature and the language feature amount to which the adjustment amount information is added by the adjustment amount addition part. A voice signal is synthesized based on the acoustic feature amount estimation unit that estimates the acoustic feature amount and the acoustic feature amount estimated by the acoustic feature amount estimation unit, and the text is adjusted according to the adjustment parameter. It is characterized by being provided with a voice generation unit that outputs a voice signal to which adjustments have been made.

Further, in the speech synthesizer according to claim 1, the speech synthesizer according to claim 1 comprises a time-length model and an acoustic model in which the statistical model is composed of a neural network, and the acoustic feature amount estimation unit is used. Using the time length model, the time length of each phonetic element, which is the output data of the time length model, is estimated by using the language feature amount of each phone element as the input data of the time length model, and the time length of each phonetic element is used. The time length for each frame is generated, the language feature amount for each frame and the time length for each frame are used as input data using the acoustic model, and the acoustic feature amount for each frame, which is the output data of the acoustic model. It is characterized by estimating.

Further, in the speech synthesizer according to claim 1, the speech synthesizer according to claim 1 or 2 sets the adjustment parameter to any one of four parameters of speaking speed or time length, power, pitch, and intonation. It is characterized by having one or a combination of two or more.

Further, the speech synthesizer according to claim 4 has the adjustment parameters as four parameters of speech speed or time length, power, pitch, and intonation in the speech synthesizer according to claim 1 or 2. The adjustment amount of any one of the parameters is specified by an arbitrary value within a predetermined range, and the adjustment amount of the other three parameters is a fixed value.

Further, in the speech synthesizer according to claim 1, the speech synthesizer according to claim 1 has the adjustment parameters as four parameters of speaking speed or time length, power, pitch, and intonation, and the four parameters are used. Each adjustment amount in the parameter is characterized in that an arbitrary value within each predetermined range is specified.

The program according to claim 6 is characterized in that the computer functions as the speech synthesizer according to any one of claims 1 to 5.

As described above, according to the present invention, it is possible to obtain a high-quality synthetic speech signal when generating a synthetic speech signal in which the reading method of a specific part of the text is adjusted.

It is a block diagram which shows the structural example of the learning apparatus by embodiment of this invention. It is a flowchart which shows the example of the pre-learning process by a learning device. It is a figure explaining the data structure example of a language feature quantity. It is a flowchart which shows the example of the voice analysis processing by the voice analysis unit. It is a figure explaining the data structure example of the acoustic feature quantity. It is a figure explaining the data structure example of the language feature quantity to which time information is added. It is a figure explaining the data structure example of the language feature amount to which the adjustment amount information is added. It is a figure explaining the learning processing example of a time length model. It is a figure explaining the learning processing example of an acoustic model. It is a block diagram which shows the structural example of the speech synthesis apparatus by embodiment of this invention. It is a flowchart which shows the voice synthesis processing example by a voice synthesis apparatus. It is a figure explaining the time length estimation processing example using a time length model. It is a figure explaining the example of the acoustic feature amount estimation processing using an acoustic model. It is a figure explaining the example of the voice synthesis processing by the voice generation part. It is explanatory drawing which shows the conventional learning method and synthesis method described in Non-Patent Document 1. It is explanatory drawing which shows the conventional audio signal adjustment method described in Non-Patent Document 2. It is explanatory drawing which shows the hypothetical example which combined the prior arts of Non-Patent Documents 1 and 2.

Hereinafter, embodiments for carrying out the present invention will be described in detail with reference to the drawings.
[Learning device]
First, a learning device according to an embodiment of the present invention will be described. FIG. 1 is a block diagram showing a configuration example of a learning device, and FIG. 2 is a flowchart showing an example of pre-learning processing by the learning device.

The learning device 1 includes storage units 10 and 17, language analysis unit 11, voice analysis unit 12, association unit 13, adjustment amount addition unit 14, acoustic feature amount adjustment unit 15, and learning unit 16. It is assumed that the audio signal is monaural and is sampled at a sampling frequency of 48 kHz and a bit number of 16.

A preset voice corpus is stored in the storage unit 10. The voice corpus consists of text and the corresponding voice signal. For example, when using a phoneme balance 503 sentence created by ATR (International Telecommunications Research Institute, Inc.), the text and the audio signal read aloud thereof consist of 503 pairs. For the voice corpus, refer to the following documents.
Kenichi Iso, Takao Watanabe, Nao Kuwahara, "Design of Sentence Set for Speech Database", Otokoron (Spring), pp.89-90 (1988.3)

The language analysis unit 11 reads each text of the voice corpus from the storage unit 10, performs a known language analysis process on the text, and obtains a language feature amount consisting of predetermined information for each phoneme (step S201). Then, the language analysis unit 11 outputs the language feature amount for each phoneme to the association unit 13.

Specifically, the language analysis unit 11 obtains phoneme information, accent information, part of speech information, accent phrase information, exhalation paragraph information, and total number information for each phoneme constituting the sentence by language analysis processing, and from these information. Find the language feature quantity.

As the language analysis process, for example, the morphological analysis process described below is used.
"MeCab: Yet Another Part-of-Speech and Morphological Analyzer", Internet <URL: http://taku910.github.io/mecab/>
Further, as the language analysis process, for example, the dependency analysis process described below is used.
“CaboCha / Pumpkin: Yet Another Japanese Dependency Structure Analyzer”, Internet <URL: https://taku910.github.io/cabocha/>

FIG. 3 is a diagram illustrating an example of data composition of language features. As shown in FIG. 3, the language feature amount is composed of phoneme information, accent information, part-of-speech information, accent phrase information, exhalation paragraph information, and total number information for each phoneme.

Returning to FIGS. 1 and 2, the voice analysis unit 12 reads out each voice signal corresponding to each text of the voice corpus from the storage unit 10, cuts out the voice signal for each frame, and has a known sound for each frame. Perform analysis processing. Then, the voice analysis unit 12 obtains an acoustic feature amount consisting of predetermined information for each frame (step S202), and outputs the acoustic feature amount for each frame to the matching unit 13. The acoustic features are composed of 199-dimensional data, as will be described later.

As the acoustic analysis process, for example, the acoustic analysis process described below is used.
“A high-quality speech analysis, manipulation and synthesis system”, Internet <URL: https://github.com/mmorise/World>
Further, as the acoustic analysis processing, for example, the voice signal processing described below is used.
“Speech Signal Processing Toolkit (SPTK) Version 3.11 December 25, 2017”, Internet <URL: http://sp-tk.sourceforge.net/>
“REFERENCE MANUAL for Speech Signal Processing Toolkit Ver. 3.9”

FIG. 4 is a flowchart showing an example of voice analysis processing by the voice analysis unit 12. The voice analysis unit 12 reads out each voice signal of the voice corpus from the storage unit 10, and cuts out the voice signal having a frame length of 25 ms every 5 ms of a frame shift (step S401). Then, the voice analysis unit 12 performs acoustic analysis processing on the voice signal for each frame, and obtains the spectrum, the pitch frequency, and the aperiodic component (step S402).

The voice analysis unit 12 analyzes the spectrum with mer cepstrum to obtain the mer cepstrum coefficient MGC (step S403). Further, the voice analysis unit 12 obtains the voiced / unvoiced determination information VUV from the pitch frequency, logarithms the voiced sections of the pitch frequency, and interpolates the unvoiced and unvoiced sections using the information of the voiced sections before and after the pitch frequency. The pitch frequency LF0 is obtained (step S404). Further, the voice analysis unit 12 analyzes the aperiodic component by mer cepstrum to obtain the band aperiodic component BAP (step S405).

As a result, the mer cepstrum coefficient MGC, the voiced / unvoiced determination information VUV, the logarithmic pitch frequency LF0, and the band aperiodic component BAP can be obtained for each frame as the acoustic features of the static characteristics.

The voice analysis unit 12 calculates the first-order difference Δ of the mer-cepstrum coefficient MGC to obtain the first-order difference mer-cepstrum coefficient ΔMGC (step S406), and calculates the second-order difference Δ ² to calculate the second-order difference mer-cepstrum coefficient Δ ² . Find the MGC (step S407).

The voice analysis unit 12 calculates the primary difference Δ of the logarithmic pitch frequency LF0 to obtain the primary difference logarithmic pitch frequency ΔLF0 (step S408), calculates the ^secondary difference Δ2, and calculates the ^secondary difference logarithmic pitch frequency Δ2. LF0 is obtained (step S409).

The voice analysis unit 12 calculates the primary difference Δ of the band aperiodic component BAP to obtain the primary difference band aperiodic component ΔBAP (step S410), and calculates the secondary difference Δ ² to obtain the secondary difference band aperiodic. The component Δ ² BAP is obtained (step S411).

As a result, as the acoustic feature amount of the dynamic characteristics, the first-order difference mel cepstrum coefficient ΔMGC, the second-order difference mel cepstrum coefficient Δ ² MGC, the first-order difference log-pitch frequency ΔLF0, and the second-order difference log-pitch frequency Δ ² LF0 are used for each frame. The first-order difference band aperiodic component ΔBAP and the second-order difference band aperiodic component Δ ² BAP are obtained.

The voice analysis unit 12 outputs an acoustic feature amount consisting of predetermined information of static characteristics and dynamic characteristics for each frame to the association unit 13.

FIG. 5 is a diagram illustrating an example of data composition of acoustic features. As shown in FIG. 5, the acoustic feature amount is the static characteristic merkepstrum coefficient MGC, the logarithmic pitch frequency LF0 and the band aperiodic component BAP, and the dynamic characteristic first-order difference merkepstrum coefficient ΔMGC, first-order difference logarithm for each frame. Pitch frequency ΔLF0, primary difference band aperiodic component ΔBAP, secondary difference mel cepstrum coefficient Δ ² MGC, secondary difference log pitch frequency Δ ² LF0 and secondary difference band aperiodic component Δ ² BAP, and static characteristic voice / It is composed of silent judgment information VUV. As will be described later, this acoustic feature amount is composed of 199-dimensional data.

Returning to FIGS. 1 and 2, the mapping unit 13 inputs the language feature amount for each phoneme from the language analysis unit 11 and the acoustic feature amount for each frame from the voice analysis unit 12. Then, the correspondence unit 13 temporally associates the language feature amount for each phoneme with the acoustic feature amount for each frame by using a known phoneme alignment technique, so that each phoneme constituting the text sentence is voiced. It is calculated at which time of the signal the position (correspondence) is (step S203).

The association unit 13 generates time information consisting of a corresponding start frame number and end frame number for each phoneme, adds time information to predetermined information for each phoneme constituting a language feature amount, and adds time information to the phoneme. Find the time length (number of frames). Then, the correspondence unit 13 outputs the language feature amount to which the time information for each associated phoneme is added to the adjustment amount addition unit 14. Further, the matching unit 13 includes the time length of each phoneme in the acoustic feature amount, and outputs the acoustic feature amount of each associated frame (data for each phoneme for the time length) to the acoustic feature amount adjusting unit 15.

Here, the time information added to the language feature is information in milliseconds. The time length for each phonetic element is used for the output data of the time length model in the statistical model described later, and is a numerical value in 5 ms frame units obtained by dividing the time length in milliseconds in the phonetic element by a frame shift of 5 ms, that is, the sound element. The number of frames is used.

As a technique for phoneme alignment, for example, the speech recognition process described below is used.
"The Hidden Markov Model Toolkit (HTK)", Internet <URL: http://htk.eng.cam.ac.uk>
“Speech Signal Processing Toolkit (SPTK) Version 3.11 December 25, 2017”

The mapping unit 13 deletes the silent sections at the beginning and end of each sentence after the temporal mapping processing of the language features and the acoustic features.

FIG. 6 is a diagram illustrating an example of data composition of a language feature amount to which time information is added. As shown in FIG. 6, the language feature amount to which the time information is added is configured by adding the time information to the language feature amount shown in FIG. Specifically, this language feature quantity is composed of time information, phoneme information, accent information, part-of-speech information, accent phrase information, exhalation paragraph information, and total number information for each phoneme.

Returning to FIGS. 1 and 2, the adjustment amount addition unit 14 inputs the language feature amount for each phoneme from the association unit 13, and also inputs a predetermined adjustment parameter. Then, the adjustment amount addition unit 14 adds the adjustment amount information of the adjustment parameter to the predetermined information for each phoneme constituting the language feature amount (step S204). The adjustment amount addition unit 14 outputs the language feature amount to which the adjustment amount information for each phoneme is added to the learning unit 16.

The predetermined adjustment parameters are parameters for adjusting the audio signal (adjusting the acoustic characteristics), and are any of the speaking speed R _ST , the power R _PW , the pitch R _PT and the _intonation R P D, or these. It shall be a combination and shall be selected by the user. Further, the adjustment parameter is used as a part of the learning data in the learning unit 16.

Speaking speed _RST indicates the amount of adjustment of speaking speed, power _RPW indicates the amount of adjustment of power (loudness), _RPT indicates the amount of adjustment of pitch (pitch of voice), and intonation _RPD indicates the amount of adjustment. The amount of adjustment of intonation (change range of voice pitch) is shown. It should be noted that the time length may be used instead of the speaking speed.

The range of the speaking speed R _ST (range of adjusting the speaking speed) is as follows, for example.
(Slow) 0.5 <= R _ST <= 4.0 (Fast)
This means that the speaking speed _RST is slower as it is closer to 0.5 and faster as it is closer to 4.0 in the range of 0.5 to 4.0.

The range of power R _PW (power adjustment amount range) is, for example, as follows.
(Small) 1.0E-5 <= R _PW <= 2.0 (Large)
This means that the power R _PW is smaller as it is closer to 1.0E-5 and larger as it is closer to 2.0 in the range of 1.0E-5 to 2.0.

The range of pitch R _PT (pitch adjustment amount range) is, for example, as follows.
(Low) 0.5 <= R _PT <= 2.0 (High)
This means that the pitch R _PT is lower as it is closer to 0.5 and higher as it is closer to 2.0 in the range of 0.5 to 2.0.

The range of intonation _RPD (range of adjustment amount of intonation) is as follows, for example.
(Small) 1.0E-5 <= R _PD <= 2.0 (Large)
This means that the intonation _RPD is smaller as it is closer to 1.0E-5 and larger as it is closer to 2.0 in the range of 1.0E-5 to 2.0. The standard values for speaking speed R _ST , power R _PW , pitch R _PT , and intonation R _PD are all 1.0.

Further, each of these adjustment parameters shall be selected from, for example, 11 data shown below. That is, any one of 11 data is used as the adjustment parameter of the speaking speed R _ST , the power R _PW , the pitch R _PT , and the intonation R _PD in the learning device 1.
[Number 1]

話速、パワー等の調整量を変化させないで元の話速、パワー等を維持する場合、調整ベクトルは以下のとおりである。

Here, the four adjustment parameters are expressed by the following adjustment vectors.

When maintaining the original speaking speed, power, etc. without changing the adjustment amount of speaking speed, power, etc., the adjustment vector is as follows.

Assuming that one data is selected from each of 11 data in each of the ^four adjustment parameters, the total number of combinations is 114 = 14,641. Therefore, in order to train the statistical model, a huge amount of data is required, which increases the learning load and takes time.

Therefore, in the embodiment of the present invention, the user selects one data from 11 data in a predetermined range for one of the four adjustment parameters, and the other three adjustment parameters are set. The standard value 1.0 may be used as a fixed value. The same applies to the acoustic feature amount adjusting unit 15 and the voice synthesizer 2 of FIG. 10 which will be described later.

この場合、調整量追加部１４は、調整パラメータとして、ユーザにより１１個のデータのうち１個のデータが選択された話速Ｒ_ST、並びに、標準値1.0を固定値としたパワーＲ_PW、ピッチＲ_PT及び抑揚Ｒ_PDを入力する。 For example, assume that the user selects one data out of 11 data for the speaking speed _RST and uses the standard value 1.0 as the fixed value for the power _RPW , pitch RP _T and _intonation RP D, and the adjustment vector is It is as follows.

In this case, the adjustment amount addition unit 14 has a speaking speed R _ST in which one of 11 data is selected by the user as an adjustment parameter, a power R _PW with a standard value of 1.0 as a fixed value, and a pitch. Enter the R _PT and intonation R _PD .

In this way, one data is selected from 11 data for one of the four adjustment parameters, and the standard value of 1.0 is used as a fixed value for the other three adjustment parameters. , Is equivalent to plotting the adjustment amount only in the axial direction of any one element of the adjustment vector R. In this case, the number of combinations is 10 × 4 + 1 = 41. As a result, when learning the statistical model, the number of training data can be reduced, so that the load of the learning process can be reduced and the time of the learning process can be shortened.

Further, as another example in the embodiment of the present invention, the user may select the four adjustment parameters in an interlocking manner in 11 steps. The same applies to the acoustic feature amount adjusting unit 15 and the voice synthesizer 2 of FIG. 10 which will be described later.

a1，b1，・・・，c11，d11は、対応する調整パラメータの調整量範囲に含まれる値とする。 In this case, the adjustment amount addition unit 14 has the speaking speed R _ST , power R _PW , pitch R _PT , and intonation R of any of the 11 types of patterns preset as adjustment parameters, which are selected by the user. Enter the _PD . The adjustment vectors of the 11 types of patterns are as follows.

a1, b1, ..., c11, d11 are values included in the adjustment amount range of the corresponding adjustment parameter.

In this case, the number of combinations is 11. As a result, when training the statistical model, the number of training data can be further reduced, so that the load can be further reduced and the time can be further shortened.

The adjustment amount addition unit 14 may input different adjustment parameters for each sentence, each breath paragraph, or each accent phrase. The same applies to the acoustic feature amount adjusting unit 15 and the voice synthesizer 2 described later.

FIG. 7 is a diagram illustrating an example of data configuration of a language feature amount to which adjustment amount information is added. As shown in FIG. 7, the language feature amount to which the adjustment amount information is added is configured by adding the adjustment amount information of the adjustment parameter to the language feature amount shown in FIG. Specifically, this language feature amount is composed of time information, phoneme information, accent information, part-of-speech information, accent phrase information, exhalation paragraph information, total number information, and adjustment amount information for each phoneme.

The adjustment amount information is information that reflects the adjustment amount in the adjustment parameters of the speaking speed R _ST , the power R _PW , the pitch R _PT , and the intonation R _PD .

As described above, the adjustment amount addition unit 14 inputs the adjustment parameter of any one of the speaking speed R _ST , the power R _PW , the pitch R _PT and the _intonation R P D, or a combination thereof. For example, when the adjustment parameter of only the speaking speed R _ST is input, the adjustment amount adding unit 14 inputs the input speaking speed R _ST , the power R _PW , the pitch R _P T, and the pitch R P T having a standard value of 1.0, which are fixed values, in the language feature quantity. Add the adjustment amount information of the intonation _RPD . Further, when the adjustment parameters of the speaking speed _RST and the power _RPW are input to the adjustment amount addition unit 14, for example, the input speaking speed _RST and the power _RPW and the standard value 1.0 which is a fixed value are input to the language feature amount. The adjustment amount information of the pitch R _PT and the intonation R _PD of is added.

Returning to FIGS. 1 and 2, the acoustic feature amount adjusting unit 15 has an acoustic feature amount (time length) for each frame corresponding to the language feature amount for each phoneme input by the adjustment amount addition unit 14 from the matching unit 13. Is the data for each phoneme). Further, the acoustic feature amount adjusting unit 15 inputs a predetermined adjustment parameter similar to the adjustment amount adding unit 14.

The acoustic feature amount adjusting unit 15 adjusts the acoustic feature amount for each frame according to the adjustment parameter, and outputs the adjusted acoustic feature amount for each frame (data for each phoneme for the time length) to the learning unit 16.

When the speaking speed is adjusted according to the adjustment parameter of the speaking speed _RST , the acoustic feature amount adjusting unit 15 multiplies the time length _DUR input from the corresponding unit 13 by the reciprocal of the speaking speed RST as shown in the following equation. , Adjust the time length by converting the multiplication result into an integer and finding the new time length DUR'.
[Number 2]
DUR'= int (DUR x 1 / R _ST ) ・ ・ ・ (2)
The time length input from the matching unit 13 is DUR, and the adjusted time length is DUR'.

When the time length is adjusted according to the time length adjustment parameter R _DR (= 1 / R _ST ) instead of the speaking speed R _ST , the acoustic feature amount adjusting unit 15 uses the time length DUR input from the corresponding unit 13. On the other hand, instead of the reciprocal of the speaking speed R _ST , the time length is adjusted by multiplying the time length adjustment parameter R _DR , converting the multiplication result into an integer, and obtaining a new time length DUR'.

The acoustic feature amount adjusting unit 15 repeats or thins out the acoustic feature amount of the frame input from the corresponding unit 13 according to the time length after adjustment, and arranges the number of frames of the acoustic feature amount, thereby aligning the acoustic feature amount. To adjust. In this way, the number of frames of the acoustic feature amount is arranged according to the adjustment of the time length for each phoneme.

The acoustic feature amount adjusting unit 15 uses the acoustic feature amounts of the front and rear frames when adjusting the acoustic feature amount by repeating or thinning out the acoustic feature amount of the corresponding frame according to the time length after the adjustment. May be performed by interpolation. As a result, high-quality acoustic features can be obtained. Further, when the speaking speed or the like is adjusted according to the adjustment parameter of the speaking speed _RST and other adjustment parameters, the acoustic feature amount adjusting unit 15 adjusts by other adjustment parameters before adjusting the speaking speed.

Further, when the voice power is adjusted according to the adjustment parameter of the power R _PW , the acoustic feature amount adjusting unit 15 is the 0th dimension in the mer cepstrum coefficient MGC of the static characteristic included in the acoustic feature amount input from the matching unit 13. The logarithmic value of the power R _PW is added to the value MGC [0] of.

As shown in the following equation, the acoustic feature amount adjusting unit 15 compares the added value with 0 and finds the larger one as the 0th-dimensional value MGC [0]'in the new static characteristic mercepstrum coefficient MGC. By doing so, the amount of acoustic features is adjusted.
[Number 3]
MGC [0]'= max (0, MGC [0] + logR _PW ) ・ ・ ・ (3)
The 0th-dimensional value in the mer cepstrum coefficient MGC of the static characteristic included in the acoustic feature amount input from the corresponding unit 13 is MGC [0], and the adjusted value is MGC [0]'.

Further, when the pitch frequency of the voice is adjusted according to the adjustment parameter of the pitch R _PT , the acoustic feature amount adjusting unit 15 is 0-dimensional in the logarithmic pitch frequency LF0 of the static characteristic included in the acoustic feature amount input from the matching unit 13. Add the logarithmic value of pitch R _PT to the eye value LF0 [0].

As shown in the following equation, the acoustic feature amount adjusting unit 15 compares the added value with 0 and finds the larger one as the 0th-dimensional value LF0 [0]'at the logarithmic pitch frequency LF0 of the new static characteristic. By doing so, the amount of acoustic features is adjusted.
[Number 4]
LF0 [0]'= max (0, LF0 [0] + logR _PT ) ・ ・ ・ (4)
The 0th dimension value in the logarithmic pitch frequency LF0 of the static characteristic included in the acoustic feature amount input from the corresponding unit 13 is LF0 [0], and the adjusted value is LF0 [0]'.

Further, when the intonation of voice is adjusted according to the adjustment parameter of the intonation RPD, the acoustic feature amount adjusting unit 15 calculates in advance from the logarithmic pitch frequency _LF0 of the static characteristic included in the acoustic feature amount input from the matching unit 13. Subtract the average value μ _LF0 that has been set. Then, the acoustic feature amount adjusting unit 15 divides the subtraction result by the standard deviation Σ _LF0 calculated in advance, and obtains the division result. The average value μ _LF0 is the average value of the logarithmic pitch frequency LF0 of the static characteristics included in the acoustic feature quantity input from the mapping unit 13, and the standard deviation Σ _LF0 is the standard deviation thereof.

［数６］

As shown in the following equation, the acoustic feature amount adjusting unit 15 sets the mean value μ _LF0 and the standard deviation Σ _LF0 of the logarithmic pitch frequency LF0 of the static characteristics included in the acoustic feature amount input from the matching unit 13 for each sentence. It shall be calculated. N is the number of frames corresponding to the sentence.
[Number 5]

[Number 6]

The acoustic feature amount adjusting unit 15 adds the logarithmic value of the intonation _RPD to the standard deviation Σ _LF0 , compares the addition result with 0, and obtains the larger one. Then, the acoustic feature amount adjusting unit 15 multiplies the division result by the larger value, and adds the average value μ _LF0 to the multiplication result.

The acoustic feature amount adjusting unit 15 compares the added value with 0 and obtains the larger one as the logarithmic pitch frequency LF0'of the new static characteristic. The formula of the arithmetic processing by the acoustic feature amount adjusting unit 15 is as follows.
[Number 7]
LF0'= max (0, ((LF0-μ _LF0 ) / Σ _LF0 ) × max (0, Σ _LF0 + logR _PD ) + μ _LF0 )
... (7)
The logarithmic pitch frequency of the static characteristics included in the acoustic feature quantity input from the mapping unit 13 is LF0, the average value is μ _LF0 , the standard deviation is Σ _LF0 , and the logarithmic pitch frequency of the static characteristics after adjustment is LF0'. ..

The acoustic feature amount adjusting unit 15 calculates the first-order difference Δ of the new static characteristic calculated according to each adjustment parameter as described above, and obtains the first-order difference of the new dynamic characteristic. Further, the acoustic feature amount adjusting unit 15 calculates the quadratic difference Δ ² to obtain the quadratic difference of the new dynamic characteristic. In this way, the acoustic feature amount adjusting unit 15 adjusts the acoustic feature amount.

The adjustment process of the acoustic feature amount by the acoustic feature amount adjusting unit 15 is linked to the addition process of the adjustment amount information to the language feature amount by the adjustment amount addition unit 14.

The learning unit 16 inputs the language feature amount for each phoneme from the adjustment amount addition unit 14, and also inputs the acoustic feature amount for each frame (data for each phoneme for the time length) from the acoustic feature amount adjustment unit 15. Then, the learning unit 16 standardizes these data and learns the time-length model and the acoustic model, which are statistical models.

(Learning of time length model)
Next, the learning process of the time length model by the learning unit 16 will be described. FIG. 8 is a diagram illustrating an example of learning processing of the time length model. The learning unit 16 generates 312-dimensional binary values and 13-dimensional numerical data representing linguistic features, and one-dimensional adjustment data based on the linguistic feature amount for each phone element input from the adjustment amount addition unit 14. The one-dimensional adjustment data is speech speed data, and the number of dimensions of the language feature is 326.

Here, the 312-dimensional binary value and the 13-dimensional numerical data in the language feature are generated based on the phoneme information, the accent information, the part of speech information, the accent phrase information, the exhalation paragraph information, and the total number information included in the language feature. To. The one-dimensional adjustment data in the language feature amount is the adjustment amount of the speech speed among the adjustment amount information (speaking speed adjustment amount, power adjustment amount, pitch adjustment amount, and intonation adjustment amount) included in the language feature amount. Is generated based on.

The learning unit 16 handles 326-dimensional data consisting of 312-dimensional binary values of language features, 13-dimensional numerical data, and one-dimensional adjustment data (speech speed data) as input data for a time-length model (step S801). ).

The learning unit 16 obtains the maximum value and the minimum value for each dimension using all the data of the 326 dimensions of the language feature amount and stores them in the storage unit 17, and for each of the all data, the maximum value for each dimension. Standardize using the value and the minimum value (step S802).

Further, the learning unit 16 has one-dimensional data of the time length of each phoneme in the acoustic feature amount (data for each phoneme for the time length) for each frame input from the acoustic feature amount adjustment unit 15. Is treated as the output data of the time model (step S803). This time length is the number of frames in units of 5 ms, and consists of a one-dimensional integer value for each phoneme expressing the text.

The learning unit 16 obtains the mean value and the standard deviation using all the one-dimensional data of the time length and stores them in the storage unit 17, and standardizes all the data using the mean value and the standard deviation. (Step S804).

The learning unit 16 shifts from steps S802 and S804, and uses 326-dimensional standardized data of language features as input data and one-dimensional standardized data of time length as output data for each phonetic element. Learn the model (step S805). Then, the learning unit 16 stores the learned time length model in the storage unit 17.

When learning the time length model in step S805, the technique described at the following site is used.
"CSTR-Edinburgh / merlin", Internet <URL: https://github.com/CSTR-Edinburgh/merlin>
The same applies to the learning of the acoustic model in step S905 of FIG. 9, which will be described later.

The time-length model is composed of, for example, a forward propagation type neural network in which the input layer is 326 dimensions, the hidden layer is 1024 dimensions, and the output layer is one dimension. The hyperbolic tangential function is used as the activation function in the hidden layer, and the mean square error function is used as the loss error function. In addition, the number of mini-batch is 64, the number of epochs is 100, the dropout rate is 0.5, the stochastic gradient descent method is used as the learning coefficient optimization method, and the start learning rate is 0.01. The learning rate is exponentially attenuated for each epoch, and learning is performed by the error back propagation method. If the evaluation error does not decrease for 5 consecutive epochs after 15 epochs, the learning is terminated early.

As a result, the time length model is stored in the storage unit 17 as a statistical model. Further, as a statistical model, the storage unit 17 has 326 dimensions including 312-dimensional binary values of language features, which are input data of the time-length model, 13-dimensional numerical data, and 1-dimensional adjustment data (speaking speed data). The maximum and minimum values for each dimension related to the data of are stored. Further, the storage unit 17 stores, as a statistical model, the mean value and the standard deviation of the time-length one-dimensional data which is the output data of the time-length model.

(Learning of acoustic model)
Next, the learning process of the acoustic model by the learning unit 16 will be described. FIG. 9 is a diagram illustrating an example of learning processing of an acoustic model. The learning unit 16 has a 312-dimensional binary value representing a language feature, a 13-dimensional numerical data, a 4-dimensional time data, and a 3-dimensional adjustment data based on the language feature amount for each phone element input from the adjustment amount addition section 14. To generate.

The four-dimensional time data consists of the number of frames of the sound element corresponding to the frame (one-dimensional data) and the position of the frame in the sound element (three-dimensional data). The three-dimensional adjustment data are power data, pitch data, and intonation data. These adjustment data are the power adjustment amount, the pitch adjustment amount, and the adjustment amount information (speaking speed adjustment amount, power adjustment amount, pitch adjustment amount, and intonation adjustment amount) included in the language feature amount. Generated based on the amount of intonation adjustment. The number of dimensions of the language feature is 332.

The learning unit 16 is a 332-dimensional binary value consisting of a 312-dimensional binary value, a 13-dimensional numerical data, a 4-dimensional time data, and a 3-dimensional adjustment data (power data, pitch data, and intonation data) in the language feature quantity for each phonetic element. From the data, 332-dimensional data in the language feature quantity for each frame is generated.

Regarding the language feature amount for each frame, the learning unit 16 has a 312-dimensional binary value of the language feature amount, a 13-dimensional numerical data, a 4-dimensional time data, and a 3-dimensional adjustment data (power data, pitch data, and intonation data). The 332-dimensional data consisting of the above is treated as the input data of the acoustic model (step S901).

The learning unit 16 obtains the maximum value and the minimum value for each dimension using all the data of the 332 dimensions of the language feature amount and stores them in the storage unit 17, and for each of the all data, the maximum value for each dimension. Standardize using the value and the minimum value (step S902).

Further, the learning unit 16 obtains 199-dimensional data for the acoustic feature amount excluding the time length of the acoustic feature amount for each frame (data for each phoneme for the time length) input from the acoustic feature amount adjusting unit 15. It is treated as output data of an acoustic model (step S903).

Here, as described above, the acoustic feature amount excluding the time length is the static characteristic mel cepstrum coefficient MGC, the logarithmic pitch frequency LF0 and the band aperiodic component BAP, and the dynamic characteristic first-order difference mel cepstrum coefficient ΔMGC, the first-order difference log. Pitch frequency ΔLF0, primary difference band aperiodic component ΔBAP, secondary difference mel cepstrum coefficient Δ ² MGC, secondary difference log pitch frequency Δ ² LF0 and secondary difference band aperiodic component Δ ² BAP, and static characteristic voice / It consists of silent judgment information VUV.

Specifically, the acoustic feature amount excluding the time length includes the static characteristic 60-dimensional Melkeptrum coefficient, the static characteristic 66-dimensional data that combines the one-dimensional log pitch frequency and the five-dimensional band aperiodic component. It consists of 132-dimensional data of dynamic characteristics obtained by first-order difference and second-order difference of these static characteristic data, and one-dimensional voiced / unvoiced determination data. That is, the number of dimensions of the acoustic feature amount excluding the time length is 199.

The learning unit 16 obtains the mean value and standard deviation for each dimension using all the 199-dimensional data of the acoustic feature quantity and stores them in the storage unit 17, and also for each of the data, the average for each dimension. Standardize using the value and standard deviation (step S904).

The learning unit 16 shifts from steps S902 and S904, and for each frame, the 332-dimensional standardized data of the language feature amount is used as input data, and the 199-dimensional standardized data of the acoustic feature amount is used as output data. Learn the model (step S905). Then, the learning unit 16 stores the learned acoustic model in the storage unit 17.

The acoustic model is composed of, for example, a forward propagation type neural network in which the input layer is 332 dimensions, the hidden layer is 1024 dimensions, and the output layer is 199 dimensions. The hyperbolic tangential function is used as the activation function in the hidden layer, and the mean square error function is used as the loss error function. In addition, the number of mini batches is 256, the number of epochs is 100, and the dropout rate is 0.5.
As a method for optimizing the learning coefficient, it is assumed that the learning is performed by the stochastic gradient descent method, the starting learning rate is 0.001, the learning rate is exponentially attenuated for each epoch after passing 10 epochs, and the learning is performed by the error back propagation method. If the evaluation error does not decrease for 5 consecutive epochs after 15 epochs, the learning is terminated early.

As a result, the acoustic model is stored in the storage unit 17 as a statistical model. Further, in the storage unit 17, as statistical models, 312-dimensional binary values of language features, which are input data of acoustic models, 13-dimensional numerical data, 4-dimensional time data, and 3-dimensional adjustment data (power data, The maximum value and the minimum value for each dimension of the 332-dimensional data (pitch data and intonation data) are stored. Further, in the storage unit 17, as a statistical model, the mean value and the standard deviation for each dimension of the 199-dimensional data of the acoustic features, which are the output data of the acoustic model, are stored.

As described above, according to the learning device 1 of the embodiment of the present invention, the language analysis unit 11 performs a known language analysis process on the text of the speech corpus, and obtains the language features for each phoneme. The voice analysis unit 12 cuts out a voice signal corresponding to the text of the voice corpus for each frame, performs a known acoustic analysis process on the voice signal for each frame, and obtains an acoustic feature amount for each frame.

The associating unit 13 temporally associates the language feature amount for each phoneme with the acoustic feature amount for each frame by using a known phoneme alignment technique, and obtains the time length for each phoneme. Then, the association unit 13 generates a language feature amount for each phoneme to which time information is added, and generates an acoustic feature amount for each associated frame (data for each phoneme for the time length).

The adjustment amount addition unit 14 adds the adjustment amount information of the adjustment parameter to the language feature amount for each phoneme to which the time information is added. The acoustic feature amount adjusting unit 15 adjusts the acoustic feature amount for each frame (data for each phoneme for the time length) according to the adjustment parameter.

The learning unit 16 has a maximum value and a minimum value for each dimension based on 326-dimensional data consisting of 312-dimensional binary values of language features, 13-dimensional numerical data, and 1-dimensional adjustment data (speech speed data). And standardize each of all the data. Further, the learning unit 16 obtains the mean value and the standard deviation based on the one-dimensional data of the time length, and standardizes the one-dimensional data of the time length.

The learning unit 16 learns a time length model using 326-dimensional standardized data of language features as input data and one-dimensional standardized data of time length as output data for each phonetic element.

The learning unit 16 is based on 332-dimensional data consisting of 312-dimensional binary values of language features, 13-dimensional numerical data, 4-dimensional time data, and 3-dimensional adjustment data (power data, pitch data, and intonation data). Then, the maximum value and the minimum value are obtained for each dimension, and each of all the data is standardized. Further, the learning unit 16 obtains the mean value and the standard deviation for each dimension based on the 199-dimensional data of the acoustic feature amount, and standardizes each of all the data.

The learning unit 16 learns an acoustic model for each frame using 332-dimensional standardized data of language features as input data and 199-dimensional standardized data of acoustic features as output data.

As a result, the storage unit 17 stores, as a learned statistical model, a time-length model, an acoustic model, a maximum value, and the like that reflect the adjustment amount information of the adjustment parameters.

Then, by the speech synthesizer 2 described later, the acoustic feature amount is estimated based on the language feature amount to which the adjustment amount information of the adjustment parameter is added by using the learning model in which the adjustment amount information of the adjustment parameter is reflected, and the acoustic feature amount is estimated for each frame. A synthetic speech signal is generated from the acoustic features of.

In the assumed example in which the prior arts of Non-Patent Documents 1 and 2 shown in FIG. 17 are combined, the frame after adjustment is made by adjusting the acoustic feature amount having the characteristic smoothed in time by estimation using the learning model. Since the synthetic voice signal is generated from each acoustic feature amount, the sound quality of the synthetic voice signal deteriorates. Furthermore, since the acoustic features corresponding to the specific part of the input text are adjusted and the synthesized speech signal is generated from the adjusted acoustic features for each frame, the adjusted part and the adjustment adjacent to it are adjusted. Discontinuity occurs in the synthesized speech signal at the connection portion between the portion not added.

On the other hand, the speech synthesizer 2 according to the embodiment of the present invention estimates the acoustic feature amount using the learning model reflecting the adjustment amount information of the adjustment parameter and generates the synthesized speech signal, so that the learning model is used. It is not necessary to make adjustments to the acoustic features that have the characteristics smoothed in time by the estimation. In addition, since the linguistic features adjusted for a specific part of the input sentence are input to the learning model to obtain the acoustic features and the synthetic speech signal is generated, the adjusted part and the adjacent part thereof No discontinuity in the synthesized speech signal at the connection between the unadjusted part

Therefore, it is possible to obtain a high-quality synthetic speech signal when generating a synthetic speech signal in which the reading method of a specific part of the text is adjusted.

Further, in the embodiment of the present invention, the adjustment parameter is any one of the speaking speed R _ST , the power R _PW , the pitch R _PT and the _intonation R P D, or a combination thereof, and is selected by the user. In this case, for example, the user selects one data from 11 data for one of the four adjustment parameters, and uses the standard value 1.0 as a fixed value for the other three adjustment parameters. .. Alternatively, the user selects, for example, four adjustment parameters in an interlocking manner in 11 steps.

By limiting the selection range of the adjustment parameter in this way, it is possible to reduce the training data when learning the statistical model, and it is possible to learn the statistical model with a low load and in a short time.

[Speech synthesizer]
Next, the speech synthesizer according to the embodiment of the present invention will be described. FIG. 10 is a block diagram showing a configuration example of a voice synthesizer, and FIG. 11 is a flowchart showing an example of voice synthesis processing by the voice synthesizer.

The voice synthesizer 2 includes a language analysis unit 20, an adjustment amount addition unit 21, an acoustic feature amount estimation unit 22, a storage unit 17, and a voice generation unit 23. The storage unit 17 corresponds to the storage unit 17 shown in FIG. 1, and stores a time length model, an acoustic model, a maximum value, and the like as statistical models learned by the learning device 1.

The statistical model learned by the learning device 1 may be read from the storage unit 17 provided in the learning device 1 and stored in the storage unit 17 provided in the speech synthesis device 2. Further, the voice synthesizer 2 may directly access the storage unit 17 provided in the learning device 1 via the Internet.

The language analysis unit 20 inputs a text to be voice-synthesized, performs a known language analysis process on the text in the same manner as the language analysis unit 11 shown in FIG. 1, and obtains a language feature amount consisting of predetermined information for each phoneme. (Step S1101). Then, the language analysis unit 20 outputs the language feature amount for each phoneme to the adjustment amount addition unit 21.

The adjustment amount addition unit 21 inputs a language feature amount for each phoneme from the language analysis unit 20, and also inputs a predetermined adjustment parameter. Then, the adjustment amount addition unit 21 adds the adjustment amount information of the adjustment parameter to the predetermined information for each phoneme constituting the language feature amount, similarly to the adjustment amount addition unit 14 shown in FIG. 1 (step S1102). The adjustment amount addition unit 21 outputs the language feature amount to which the adjustment amount information for each phoneme is added to the acoustic feature amount estimation unit 22.

As described above, the predetermined adjustment parameter shall be any one of the speaking speed R _ST , the power R _PW , the pitch R _PT and the _intonation R P D, or a combination thereof, and shall be specified by the user. The value of the adjustment parameter shall be an arbitrary real number within the range of the adjustment described above. That is, the predetermined adjustment parameter is any one or a combination of two or more of the speaking speed R _ST , the power R _PW , the pitch R _PT , and the intonation R _PD .

The predetermined adjustment parameters are speaking speed R _ST , power R _PW , pitch R _PT , and intonation R _PD , and the adjustment amount of any one of these four parameters is arbitrary within the predetermined range. A value is specified, and a fixed value may be used as the adjustment amount of the other three parameters. Further, the predetermined adjustment parameters may be the above-mentioned four parameters, and any value within the respective predetermined range may be specified for each adjustment amount.

The adjustment amount addition unit 21 may input different adjustment parameters in sentence units, exhalation paragraph units, or accent phrase units, as in the adjustment amount addition unit 14 shown in FIG.

The acoustic feature amount estimation unit 22 inputs the language feature amount for each phoneme from the adjustment amount addition unit 21, performs standardization and destandardization processing using the maximum value and the like stored in the storage unit 17, and creates a time-length model. Use to estimate the time length for each phoneme.

The acoustic feature amount estimation unit 22 performs standardization and destandardization processing using the maximum value and the like stored in the storage unit 17, and estimates the acoustic feature amount for each frame using the acoustic model (step S1103). The acoustic feature amount estimation unit 22 outputs the acoustic feature amount for each frame to the voice generation unit 23.

(Estimation of time length using time length model)
Next, the time length estimation process using the time length model by the acoustic feature amount estimation unit 22 will be described. FIG. 12 is a diagram illustrating an example of time length estimation processing using a time length model. The acoustic feature amount estimation unit 22 has 312-dimensional binary values and 13-dimensional numerical data representing language features, and one-dimensional adjustment data (talk) based on the language feature amount for each phone element input from the adjustment amount addition unit 21. (Speed data) is generated. The number of dimensions of language features is 326.

The acoustic feature amount estimation unit 22 handles 326-dimensional data consisting of a 312-dimensional binary value of a language feature amount, a 13-dimensional numerical data, and a one-dimensional adjustment data (speech speed data) as input data of a time-length model. (Step S1201).

From the storage unit 17, the acoustic feature amount estimation unit 22 is 326 composed of a 312-dimensional binary value of the language feature amount, which is input data of the time-length model, a 13-dimensional numerical data, and a one-dimensional adjustment data (speech speed data). Read the maximum and minimum values for each dimension related to the dimension data. Then, the acoustic feature amount estimation unit 22 standardizes each of the 326-dimensional data of the language feature amount by using the maximum value and the minimum value for each dimension (step S1202).

The acoustic feature amount estimation unit 22 is the output data of the time length model using the time length model stored in the storage unit 17 and using the 326-dimensional standardized data of the language feature amount as the input data of the time length model. Estimate one-dimensional standardized data of time length (step S1203).

The acoustic feature amount estimation unit 22 reads out the average value and the standard deviation of the time-length one-dimensional data, which is the output data of the time-length model, from the storage unit 17. Then, the acoustic feature amount estimation unit 22 reverse-standardizes the one-dimensional standardized data of the time length estimated in step S1203 using the mean value and the standard deviation (step S1204), and the one-dimensional time length. Data is obtained (step S1205).

As a result, the time length model stored in the storage unit 17, the maximum and minimum values for each dimension of the 326-dimensional data of the language feature amount which is the input data of the time length model, and the output data of the time length model. The time-length one-dimensional data for each phonetic element can be obtained from the 326-dimensional data of the language feature quantity for each phonetic element by using the average value and the standard deviation for the time-length one-dimensional data.

(Estimation of acoustic features using an acoustic model)
Next, the acoustic feature estimation process using the acoustic model by the acoustic feature estimation unit 22 will be described. FIG. 13 is a diagram illustrating an example of acoustic feature amount estimation processing using an acoustic model. The acoustic feature amount estimation unit 22 is based on the one-dimensional data of the time length for each phoneme obtained in step S1205, and similarly to step S901 in FIG. 9, the time data for each of the plurality of frames corresponding to the phonemes. Generate four-dimensional data (step S1301).

The acoustic feature amount estimation unit 22 has a 312-dimensional binary value representing a language feature, a 13-dimensional numerical data, and a three-dimensional adjustment data (power data) based on the language feature amount for each phone element input from the adjustment amount addition unit 21. , Pitch data and intonation data). Then, the acoustic feature amount estimation unit 22 is a 328-dimensional data composed of a 312-dimensional binary value, a 13-dimensional numerical data, and a three-dimensional adjustment data (power data, pitch data, and intonation data) in the language feature amount for each phone element. From, 328-dimensional data in the language feature quantity for each frame is generated.

The acoustic feature amount estimation unit 22 has a 328-dimensional data consisting of a 312-dimensional binary value of the language feature amount for each frame, a 13-dimensional numerical data, and a 3-dimensional adjustment data (power data, pitch data, and intonation data), and a 328-dimensional data. The four-dimensional data of the time data generated in step S1301 is treated as the input data of the acoustic model (step S1302).

From the storage unit 17, the acoustic feature amount estimation unit 22 contains 312-dimensional binary values of language features, which are input data of the acoustic model, 13-dimensional numerical data, 4-dimensional time data, and 3-dimensional adjustment data (power data). , Pitch data and intonation data), and the maximum and minimum values for each dimension of the 332-dimensional data are read out. Then, the acoustic feature amount estimation unit 22 standardizes each of the 332-dimensional data consisting of the 328-dimensional data of the language feature amount and the 4-dimensional data of the time data by using the maximum value and the minimum value for each dimension. (Step S1303).

The acoustic feature estimation unit 22 is 332-dimensional standardized consisting of 328-dimensional standardized data of language features and 4-dimensional standardized data of time data using an acoustic model stored in the storage unit 17. Using the collected data as input data of the acoustic model, 199-dimensional standardized data of acoustic features, which is output data of the acoustic model, is estimated (step S1304).

The acoustic feature amount estimation unit 22 reads out the average value and the standard deviation of the 199-dimensional data of the acoustic feature amount, which is the output data of the acoustic model, from the storage unit 17. Then, the acoustic feature amount estimation unit 22 reverse-standardizes the 199-dimensional standardized data of the acoustic feature amount estimated in step S1304 by using the mean value and the standard deviation for each dimension (step S1305). The acoustic feature amount estimation unit 22 generates 199-dimensional data of the acoustic feature amount for each frame (step S1306).

The acoustic features estimated and destandardized in this way take discrete values for each frame. Therefore, the acoustic feature estimation unit 22 obtains maximum likelihood estimation or moving average for the 199-dimensional data of the acoustic features for each continuous frame, and obtains the 199-dimensional data of the new acoustic features for each frame. Ask. As a result, the acoustic feature amount for each frame becomes a smooth value.

As a result, the acoustic model stored in the storage unit 17, the maximum and minimum values for each dimension of the 332-dimensional data of the language feature amount which is the input data of the acoustic model, and the acoustic feature amount which is the output data of the acoustic model. Obtaining 199-dimensional data of acoustic features per frame from 328-dimensional data of language features and 4-dimensional data of time data for each frame using the average value and standard deviation of the 199-dimensional data of. Can be done.

Returning to FIGS. 10 and 11, the voice generation unit 23 inputs the acoustic feature amount for each frame from the acoustic feature amount estimation unit 22, and synthesizes the voice signal based on the acoustic feature amount for each frame (step S1104). .. Then, the voice generation unit 23 outputs a voice signal adjusted by the adjustment parameter to the text to be voice-synthesized.

FIG. 14 is a diagram illustrating an example of voice synthesis processing by the voice generation unit 23. Of the acoustic features for each frame input from the acoustic feature estimation unit 22, the voice generation unit 23 has a mercepstrum coefficient MGC for each frame, a logarithmic pitch frequency LF0, and a static characteristic acoustic feature that is a band aperiodic component BAP. Is selected (step S1401).

The voice generation unit 23 converts the mel cepstrum coefficient MGC into a mel cepstrum spectrum and obtains a spectrum (step S1402). Further, the voice generation unit 23 obtains the voiced / unvoiced determination information VUV from the logarithmic pitch frequency LF0, indexes the voiced section of the logarithmic pitch frequency LF0, sets the silent and silent sections to zero, and obtains the pitch frequency (step S1403). .. Further, the voice generation unit 23 converts the band aperiodic component BAP into a merkepstrum spectrum and obtains the aperiodic component (step S1404).

The voice generation unit 23 continuously uses the spectrum for each frame obtained in step S1402, the pitch frequency for each frame obtained in step S1403, and the aperiodic component for each frame obtained in step S1404. Is generated (step S1405), and an audio signal is output (step S1406).

As a result, it is possible to obtain a voice signal adjusted by a predetermined adjustment parameter with respect to the text to be voice-synthesized.

As described above, according to the speech synthesizer 2 of the embodiment of the present invention, the language analysis unit 20 performs a known language analysis process on the text to be speech-synthesized, obtains a language feature amount for each phoneme, and adjusts the amount. The addition unit 21 adds the adjustment amount information of the adjustment parameter to the language feature amount for each phoneme.

The acoustic feature amount estimation unit 22 stores a maximum of 326-dimensional data including a 312-dimensional binary value of a language feature amount, a 13-dimensional numerical data, and a one-dimensional adjustment data (speech speed data) in the storage unit 17. Standardize using values. Then, the acoustic feature amount estimation unit 22 uses the time length model stored in the storage unit 17 to use these standardized data as input data, and uses the time length standardized data which is the output data. presume.

The acoustic feature amount estimation unit 22 destandardizes the one-dimensional standardized data of the time length using the average value or the like stored in the storage unit 17, and obtains the time data for each frame. The acoustic feature amount estimation unit 22 has a 328-dimensional binary value, a 13-dimensional numerical data, and a three-dimensional adjustment data (power data, pitch data, and intonation data) out of the 329-dimensional data of the language feature amount. The four-dimensional data of the data and the time data are standardized by using the maximum value and the like stored in the storage unit 17. Then, the acoustic feature amount estimation unit 22 uses the acoustic model stored in the storage unit 17 to use these standardized data as input data, and 199-dimensional standardized data of the acoustic feature amount which is the output data. presume.

The acoustic feature amount estimation unit 22 destandardizes the 199-dimensional standardized data of the acoustic feature amount using the average value and the like stored in the storage unit 17, and obtains the acoustic feature amount for each frame. Then, the voice generation unit 23 synthesizes a voice signal based on the acoustic feature amount for each frame, and generates a synthesized voice signal.

In the assumed example in which the prior arts of Non-Patent Documents 1 and 2 shown in FIG. 17 are combined, the frame after adjustment is made by adjusting the acoustic feature amount having the characteristic smoothed in time by estimation using the learning model. Since the synthetic voice signal is generated from each acoustic feature amount, the sound quality of the synthetic voice signal is deteriorated. Furthermore, since the acoustic features corresponding to the specific part of the input text are adjusted and the synthesized speech signal is generated from the adjusted acoustic features for each frame, the adjusted part and the adjustment adjacent to it are adjusted. Discontinuity occurs in the synthesized speech signal at the connection portion between the portion not added.

On the other hand, the speech synthesizer 2 according to the embodiment of the present invention estimates the acoustic feature amount using the learning model reflecting the adjustment amount information of the adjustment parameter and generates the synthesized speech signal, so that the learning model is used. It is not necessary to make adjustments to the acoustic features that have the characteristics smoothed in time by the estimation. In addition, the adjusted linguistic features corresponding to a specific part of the input sentence are input to the learning model to obtain the acoustic features, and the synthetic speech signal is generated. There is no discontinuity in the synthetic speech signal at the connection between the unadjusted portion.

Therefore, it is possible to obtain a high-quality synthetic speech signal when generating a synthetic speech signal in which the reading method of a specific part of the text is adjusted.

Although the present invention has been described above with reference to embodiments, the present invention is not limited to the above-described embodiment and can be variously modified without departing from the technical idea.

As the hardware configuration of the learning device 1 and the speech synthesizer 2 according to the embodiment of the present invention, a normal computer can be used. The learning device 1 and the speech synthesis device 2 are composed of a computer provided with a volatile storage medium such as a CPU and RAM, a non-volatile storage medium such as a ROM, and an interface.

The functions of the storage units 10 and 17, the language analysis unit 11, the voice analysis unit 12, the association unit 13, the adjustment amount addition unit 14, the acoustic feature amount adjustment unit 15, and the learning unit 16 provided in the learning device 1 are these. This is realized by having the CPU execute a program that describes the function. Further, each function of the language analysis unit 20, the adjustment amount addition unit 21, the acoustic feature amount estimation unit 22, the storage unit 17, and the voice generation unit 23 provided in the speech synthesizer 2 also has a program describing these functions in the CPU. It is realized by executing each.

These programs are stored in the storage medium, read by the CPU, and executed. In addition, these programs can be stored and distributed in storage media such as magnetic disks (floppy (registered trademark) disks, hard disks, etc.), optical disks (CD-ROM, DVD, etc.), semiconductor memories, etc., and can be distributed via a network. You can also send and receive.

1 Learning device 2 Speech synthesizer 10, 17 Storage unit 11, 20 Language analysis unit 12 Speech analysis unit 13 Correspondence unit 14, 21 Adjustment amount addition unit 15 Acoustic feature amount adjustment unit 16 Learning unit 22 Acoustic feature amount estimation unit 23 Voice Generator

Claims (6)

A language analysis unit that linguistically analyzes the text to be voice-synthesized and obtains linguistic features.
An adjustment amount addition unit that adds adjustment amount information of adjustment parameters for adjusting acoustic characteristics to the language feature amount obtained by the language analysis unit, and an adjustment amount addition unit.
An acoustic feature amount estimation unit that estimates an acoustic feature amount using a statistical model learned in advance based on the language feature amount to which the adjustment amount information is added by the adjustment amount addition unit.
A voice generation unit that synthesizes a voice signal based on the acoustic feature amount estimated by the acoustic feature amount estimation unit and outputs a voice signal adjusted by the adjustment parameter to the text is provided. A voice synthesizer characterized by the fact that. In the voice synthesizer according to claim 1,
The statistical model consists of a time-length model and an acoustic model composed of a neural network.
The acoustic feature amount estimation unit is
Using the time length model, the time length of each phoneme, which is the output data of the time length model, is estimated by using the language feature amount for each phoneme as the input data of the time length model.
The time length for each frame is generated from the time length for each phoneme.
Using the acoustic model, the language feature amount for each frame and the time length for each frame are used as input data, and the acoustic feature amount for each frame, which is the output data of the acoustic model, is estimated. Speech synthesizer. In the speech synthesizer according to claim 1 or 2.
A speech synthesizer comprising the adjustment parameter as any one or a combination of two or more of the four parameters of speaking speed or time length, power, pitch, and intonation. In the speech synthesizer according to claim 1 or 2.
The adjustment parameters are set as four parameters of speaking speed or time length, power, pitch, and intonation.
The adjustment amount of any one of the four parameters is specified as an arbitrary value within a predetermined range, and the adjustment amount of the other three parameters is a fixed value. Synthesizer. In the speech synthesizer according to claim 1 or 2.
The adjustment parameters are set as four parameters of speaking speed or time length, power, pitch, and intonation.
A speech synthesizer characterized in that an arbitrary value within a predetermined range is specified for each adjustment amount in the four parameters. A program for operating a computer as the speech synthesizer according to any one of claims 1 to 5.

Images