JP2001350491A - Method and device for voice processing - Google Patents

Abstract

PROBLEM TO BE SOLVED: To learn a rhythm estimation model in which fluctuation in uttering environment is being considered. SOLUTION: Rhythm information of learning data is obtained every time the data are inputted (S3), estimation factors of the data are obtained (S4), and uttering date and time information of the data is obtained (S5). Then, a rhythm estimation model is learned to estimate prescribed rhythm information employing the rhythm information of the data, the estimation factors and the uttering date and time information (S8).

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

The present invention relates to a speech processing method and apparatus, and more particularly to a technique for improving the quality of synthesized speech.

[0002]

2. Description of the Related Art A speech rule synthesis process includes a prosody generation step for estimating prosody information such as a duration, a fundamental frequency (F0), and power from an input text, and a waveform for generating a speech waveform based on the generated prosody information. It is roughly divided into a generation step.

[0003] In the prosody generation step, a corpus-based technique has recently been used. This is a method of statistically learning the relationship between text and prosody information based on a large amount of learning data. By using this method and assuming a model (prosody estimation model) that outputs predetermined prosody information with predetermined estimation factors as input, it is possible to estimate predetermined prosody information.

[0004]

However, the above method has the following problems.

Since prosody information such as duration, fundamental frequency, power and the like fluctuates under the influence of the utterance environment, in order to create a highly accurate prosody estimation model using the above method, the utterance environment must be stable. It is necessary to prepare a large amount of training data. However, in order to prepare a large amount of training data, it takes a long time to record the training data, and unless a stable vocal environment is maintained for a long time, it is necessary to generate an accurate prosodic estimation model. There is a problem that can not be. It is very difficult to maintain a stable vocal environment over a long period of time, and even for the same vocal speaker, the vocal environment fluctuates due to the physical condition, familiarity, fatigue, etc. of the day. The synthesized speech generated using the prosody estimation model affected by such fluctuations is unnatural and unnatural.

SUMMARY OF THE INVENTION The present invention has been made in view of the above problems, and has an object to learn a prosody estimation model in consideration of fluctuations in the utterance environment, and to generate a natural and natural synthesized speech. With the goal.

[0007]

According to one aspect of the present invention, there is provided a voice processing method for inputting voice information, and a first obtaining step of obtaining prosody information from the voice information. A second acquisition step of acquiring information indicating a change in the utterance environment of the voice information as one of the estimation factors; and a prosody estimation model for estimating the prosody information using the prosody information and the estimation factors. And a learning step of learning.

According to another aspect of the present invention, there is provided a speech processing method for analyzing character information, an acquiring step for acquiring a speech unit corresponding to the character information, and a speech environment. A setting step of setting the information indicating the change as one of the estimation factors, and an estimation step of estimating the prosody information of the speech unit using the estimation factors and a prosody estimation model for estimating predetermined prosody information. Prepare.

According to another aspect of the present invention, there is provided a voice processing apparatus for inputting voice information, first obtaining means for obtaining prosody information from the voice information, Second acquisition means for acquiring information indicating a change in the utterance environment as one of estimation factors, and learning means for learning a prosody estimation model for estimating the prosody information using the prosody information and the estimation factors. And

According to another aspect of the present invention, there is provided a speech processing apparatus for analyzing character information, acquiring means for acquiring a speech unit corresponding to the character information, Setting means for setting information indicating a change in environment as one of the estimation factors; and estimation means for estimating the prosody information of the speech unit using the estimation factors and a prosody estimation model for estimating predetermined prosody information. Is provided.

[0011]

Preferred embodiments of the present invention will be described below with reference to the accompanying drawings.

[First Embodiment] FIG. 1 is a block diagram showing a hardware configuration of a speech synthesizer according to the present embodiment. In FIG. 1, reference numeral 11 denotes a central processing unit that performs processing such as numerical calculation and control, and controls the processing procedure described in the flowcharts of FIGS. 2 and 4 using various control programs. A learning unit 12 learns a prosody estimation model for estimating predetermined prosody information using a large amount of learning data. The learning unit 12 includes a model management unit 19 that manages the learned prosody estimation model. The learning unit 12 can be configured by hardware or software. When configured with software, a control program for implementing the software is stored in the recording device 13. Then, the central processing unit 11 realizes the function of the model learning unit 12 based on the control program.

Reference numeral 13 denotes a storage device including a semiconductor memory, a hard disk, and the like, and a control program for realizing the processing procedure described in the flowcharts of FIGS.
A control program for controlling a graphical user interface for supporting input of learning data and text for speech synthesis is stored. The storage device 13 includes a learning data storage unit 18 that stores a large amount of learning data.

Reference numeral 14 denotes an output device including a display, a speaker, and the like. The speaker outputs synthesized voice, and the display displays the above-described graphical user interface. Reference numeral 15 denotes an input device including a keyboard, a microphone, and the like. The keyboard inputs or specifies a text (in Japanese or another language) to be synthesized, and the microphone inputs learning data. 16 is an internal bus.

Reference numeral 17 denotes a speech synthesis unit, which generates a synthesized speech from the input text according to the processing procedure described with reference to FIG. The speech synthesis unit 17 can be configured by hardware or software.
When configured with software, a control program for implementing the software is stored in the recording device 13. Then, the central processing unit 11 realizes the function of the speech synthesis unit 17 based on the control program.

Next, the processing operation of the speech synthesizing apparatus of the present embodiment having the above configuration will be described.

FIG. 2 is a flowchart for explaining a learning procedure of the prosody estimation model in the present embodiment. In FIG. 2, a procedure for learning a prosody estimation model for estimating a duration time, which is one of the prosody information, will be described.

First, in step S1, the learning unit 12 initializes a loop counter i to zero.

In step S2, the microphone of the input device 15 inputs a speech waveform of a learning text in units of words or sentences, and supplies this speech waveform to the learning unit 12 as i-th learning data. The speech waveform is stored in the learning data storage unit 18.

In step S3, the learning unit 12
Performs acoustic analysis of the i-th learning data in real time,
A phoneme sequence having phonemes as phoneme units and prosodic information of each phoneme are acquired, and these are stored in the learning data storage unit 18. For example, when the speech waveform of the learning text “storm” is subjected to acoustic analysis, the phoneme sequence “/ a /, / r” is obtained from the speech waveform.
/, / A /, / sh /, / i / ". The prosody information includes the duration of each phoneme (phoneme length),
Acquire the fundamental frequency (phoneme height), power (phoneme size), and the like. FIG. 3 shows an example in which the duration of each phoneme is stored as prosody information.

In step S4, the learning unit 12
Acquires the estimated factors from the i-th learning data and stores them in the learning data storage unit 18. In this embodiment,
The presence or absence of an accent nucleus, the position of a mora, etc. are obtained as estimation factors. FIG. 3 shows an example in which the presence or absence of an accent nucleus, the mora position, and the like are accumulated as estimation factors.

In step S5, the learning unit 12
Obtains the utterance date and time information of the i-th learning data, and stores it in the learning data storage unit 18. As the utterance date and time information, for example, the date and time when the learning data is stored in the learning data storage unit 18 in step S2 is used. The utterance date and time information is an estimation factor for considering a temporal change in the utterance environment of the learning data. FIG. 3 shows an example in which the utterance date and time information of each phoneme is stored as one of the estimation factors.

In step S6, the learning unit 12
Adds 1 to the value of the loop counter i. Then, in step S7, it is determined whether or not the value of the loop counter i is equal to a preset total number of learning data. If the two are not equal, it is determined that there is uninput learning data, the process returns to step S2, and the above processing is repeated. Thus, the learning unit 12 accumulates the speech waveform, the prosodic information, the estimation factor, and the utterance date and time information of a large amount of learning data in the learning data accumulating unit 18 as shown in FIG.

In step S8, the learning unit 12
Learns a prosody estimation model for estimating a duration, which is one of the prosody information, using the data stored in the learning data storage unit 18. In learning the prosody estimation model, a statistical method such as quantification class I or a regression tree is used. The learned prosody estimation model is stored in the model management unit 19 of the learning unit 12.

As described above, according to the present embodiment,
A prosody estimation model for estimating a duration, which is one of the prosody information, can be learned using estimation factors such as the presence or absence of an accent nucleus, a mora position, and utterance date and time information. In particular,
By using the utterance date and time information as one of the estimation factors, it is possible to learn a prosody estimation model that considers the temporal variation of the utterance environment of the learning data.

In the above embodiment, the presence or absence of an accent nucleus,
Although an example has been described in which a prosody estimation model for estimating a duration length, which is one of the prosody information, is learned using estimation factors such as a mora position and utterance date / time information, the present embodiment is not limited to this. By using the above estimation factors, it is possible to learn a prosody estimation model in consideration of the temporal variation of the utterance environment of the learning data even for prosody information such as a fundamental frequency and power.

FIG. 4 is a flowchart for explaining the procedure of the speech synthesis processing in this embodiment. FIG. 4 illustrates a procedure of synthesizing an input text using a prosody estimation model for estimating a duration, which is one of prosody information.

First, in step S11, the speech synthesizing unit 17 analyzes an input text (a character string including units of words, phrases, sentences, and the like).

In step S12, the speech synthesizing unit 17 sets estimation factors such as the presence or absence of an accent nucleus and the mora position based on the analysis result in step S11.

In step S13, the speech synthesis unit 17 searches the learning data storage unit 18 to obtain a plurality of speech units corresponding to the phoneme sequence of the input text and the prosodic information of each speech unit. The prosody information acquired here is
For example, duration, fundamental frequency, power, and the like.

In step S14, the speech synthesis unit 17 sets the utterance date and time information of the predetermined speech unit obtained in step S13. In the present embodiment, for example, the utterance date and time information of the first speech unit acquired in step S13 is set.

In step S15, the speech synthesizing unit 17 determines the estimated factor obtained in step S12, the utterance date and time information set in step S14, and the learning unit 1
Using the prosody estimation model held by the second model management unit 19, the duration time, which is one of the prosody information, is estimated.

In step S16, the speech synthesizing unit 17 replaces the duration obtained in step S13 with the duration estimated in step S15, and executes step S1 based on the replaced duration and other prosody information.
The waveform of the speech unit obtained in step 3 is edited and connected. In the present embodiment, PSOLA (Pitch-Synchronous Overlap Add meth
od "Pitch-synchronized waveform superposition method") to edit the waveform of each speech unit.

In step S17, the speech synthesis unit 17 supplies the synthesized speech generated in step S16 to the output device 14. This synthesized voice is output from a speaker included in the output device 14.

As described above, according to this embodiment,
Using the prosody estimation model learned in consideration of the temporal fluctuation of the utterance environment, it is possible to estimate the duration time, which is one of the prosody information. As a result, it is possible to generate a natural and comfortable speech that is not affected by fluctuations in the utterance environment from the input text.

In the above embodiment, an example has been described in which the duration of an input text is estimated using a prosody estimation model that has been learned in consideration of temporal fluctuations in the utterance environment. However, the present embodiment is not limited to this. Absent. It is possible to estimate prosody information such as fundamental frequency and power as well as duration time.

In the above embodiment, an example was described in which the date and time when the learning data was actually input is used as the utterance date and time information, which is one of the estimation factors. However, the present embodiment is not limited to this. For example, as shown in FIG. 6, a label representing a predetermined period (unit of time, date, etc.) may be used as the utterance date and time information. FIG. 6 is a diagram showing an example of learning data for learning a prosody estimation model for estimating a duration, which is one of the prosody information. The presence / absence of an accent nucleus, the mora position, the utterance date / time information, and the like in FIG. 6 are estimation factors of the prosody estimation model. The utterance date and time information of each learning data includes, for example, a different label (set 1
To 7).

In the above embodiment, an example is described in which the presence or absence of an accent nucleus, the mora position, the utterance date and time information, and the like are used as estimation factors, but the present embodiment is not limited to this. If it is an estimated factor that can take into account the influence of the fluctuation of the utterance environment of the learning data, in addition to the utterance date and time information, further utter the gender of the speaker and the sentence whether the learning data is obtained by uttering a word. It is also possible to use, as the estimation factor, whether the learning data is obtained from the learning data, the learning data obtained from the reading sentence or the learning data obtained from the dialogue sentence. By increasing the types of such estimation factors, a more accurate prosody estimation model can be generated. Also, by using such a prosody estimation model, it is possible to generate a synthesized speech that is extremely natural and does not cause any discomfort. It is also possible to use the accent type and the number of mora as estimation factors.

Further, an object of the present invention is to supply a storage medium (or a recording medium) in which a program code of software for realizing the functions of the above-described embodiments is recorded to a system or an apparatus, and to provide a computer (a computer) of the system or the apparatus. Or a CPU or MPU) reads out and executes the program code stored in the storage medium,
Needless to say, this is achieved. In this case, the program code itself read from the storage medium implements the functions of the above-described embodiment, and the storage medium storing the program code constitutes the present invention.
In addition, by the computer executing the readout program code, not only the functions of the above-described embodiments are realized, but also based on the instructions of the program code,
The operating system (OS) running on the computer performs part or all of the actual processing,
It goes without saying that a case where the function of the above-described embodiment is realized by the processing is also included.

Further, after the program code read from the storage medium is written into a memory provided in a function expansion card inserted into the computer or a function expansion unit connected to the computer, the program code is read based on the instruction of the program code. , The CPU provided in the function expansion card or the function expansion unit performs part or all of the actual processing,
It goes without saying that a case where the function of the above-described embodiment is realized by the processing is also included.

[0041]

As described above, according to the present invention,
It becomes possible to learn a prosody estimation model that takes into account fluctuations in the utterance environment.

Further, according to the present invention, it is possible to generate a natural and uncomplicated synthesized speech in which the influence of the fluctuation of the utterance environment is suppressed.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a hardware configuration of a speech synthesis device according to an embodiment.

FIG. 2 is a flowchart illustrating a learning procedure of a prosody estimation model in the embodiment.

FIG. 3 is a diagram illustrating examples of prosody information, estimated factors, and utterance date and time information according to the embodiment;

FIG. 4 is a flowchart illustrating a procedure of a speech synthesis process according to the embodiment.

FIG. 5 is a diagram illustrating examples of prosody information, estimated factors, and utterance date and time information according to another embodiment.

Claims (25)

[Claims] An input step of inputting voice information; a first obtaining step of obtaining prosody information from the voice information; and a second obtaining step of obtaining information indicating a change in a vocal environment of the voice information as one of estimation factors. 2. A speech processing method comprising: an acquisition step; and a learning step of learning a prosody estimation model for estimating the prosody information using the prosody information and the estimation factor. 2. The voice processing method according to claim 1, wherein the first obtaining step obtains the prosodic information for each phoneme. 3. The audio processing method according to claim 1, wherein the first acquisition step acquires any one of a duration time, a fundamental frequency, and power as the prosodic information. 4. The audio processing method according to claim 1, wherein the information indicating a change in the utterance environment of the audio information is information indicating an utterance date and time of the audio information. 5. The audio processing method according to claim 1, wherein said second obtaining step obtains accent information of said learning data as one of estimated factors. 6. The audio processing method according to claim 1, wherein the second obtaining step obtains mora information of the learning data as one of estimation factors. 7. An analyzing step of analyzing character information; an obtaining step of obtaining a speech unit corresponding to the character information; and a setting step of setting information indicating a change in the vocal environment as one of estimation factors. An estimating step of estimating the prosody information of the speech unit using the estimation factor and a prosody estimation model for estimating predetermined prosody information. 8. The speech processing method according to claim 7, wherein the prosody estimation model estimates one of a duration time, a fundamental frequency, and power. 9. The speech processing method according to claim 7, wherein the information indicating a change in the utterance environment is information indicating the utterance date and time of the speech unit. 10. The voice processing method according to claim 7, wherein said setting step sets accent information of said character information as one of estimated factors. 11. The audio processing method according to claim 7, wherein the setting step sets the mora information of the character information as one of the estimation factors. 12. A synthesizing step of synthesizing a speech corresponding to the character information using the speech segment acquired in the acquiring step and the prosodic information estimated in the estimating step. 12. The audio processing method according to any one of 7 to 11. 13. An input unit for inputting voice information; a first obtaining unit for obtaining prosody information from the voice information; and a second obtaining unit for obtaining information indicating a change in the vocal environment of the voice information as one of estimation factors. 2. An audio processing apparatus comprising: an acquisition unit; and a learning unit that learns a prosody estimation model for estimating the prosody information using the prosody information and the estimation factor. 14. The speech processing device according to claim 13, wherein the first acquisition unit acquires the prosodic information for each phoneme. 15. The speech processing device according to claim 13, wherein the first acquisition unit acquires any one of a duration time, a fundamental frequency, and power as the prosody information. 16. The audio processing apparatus according to claim 13, wherein the information indicating a change in the utterance environment of the audio information is information indicating an utterance date and time of the audio information. 17. The speech processing apparatus according to claim 13, wherein said second acquisition means acquires accent information of said learning data as one of estimated factors. 18. The speech processing apparatus according to claim 13, wherein said second acquisition unit acquires the mora information of the learning data as one of estimation factors. 19. An analyzing unit for analyzing character information, an obtaining unit for obtaining a speech unit corresponding to the character information, a setting unit for setting information indicating a change in the utterance environment as one of estimation factors, An audio processing apparatus comprising: an estimation unit configured to estimate prosody information of the speech unit using the estimation factor and a prosody estimation model for estimating predetermined prosody information. 20. The prosody estimation model comprising: a duration length;
20. The audio processing device according to claim 19, wherein one of a fundamental frequency and power is estimated. 21. The speech processing apparatus according to claim 19, wherein the information indicating the change in the utterance environment is information indicating the utterance date and time of the speech unit. 22. The speech processing device according to claim 19, wherein said setting means sets accent information of said character information as one of estimated factors. 23. The audio processing apparatus according to claim 19, wherein said setting means sets mora information of said character information as one of estimated factors. 24. The apparatus according to claim 24, further comprising a synthesizing unit for synthesizing a voice corresponding to the character information using the speech unit acquired by the acquiring unit and the prosody information estimated by the estimating unit. 23. The audio processing device according to any one of 19 to 22. 25. A storage medium storing a control program for realizing a sound processing method according to any one of claims 1 to 12 by a computer.