JP3918606B2 - Speech synthesis apparatus, speech synthesis method, speech synthesis program, and computer-readable recording medium storing the program - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a speech synthesizer that synthesizes speech based on input performance data, a speech synthesis method, a speech synthesis program, and a computer-readable recording medium on which the program is recorded. The present invention relates to a speech synthesizer, a speech synthesis method, a speech synthesis program, and a computer-readable recording medium on which the program is recorded.
[0002]
[Prior art]
Breathiness is a term that describes the characteristics of human speech. Breathability is an index that represents the loudness of a breath. Speaking of breathability means that the sound of breathing is felt loud. Since this breathability is one of the characteristics of a speaker or a singer, it is preferable to perform speech synthesis in consideration of breathability in the speech synthesizer.
[0003]
It has been found that the dynamics, which are breathiness and the volume of sound in the sense of sound, change with the change of the ratio of the harmonic component and the nonharmonic component of the voice. Here, the harmonic component is a periodic voice component caused by the vocal cord vibration, and the non-harmonic component is a noisy voice generated by the narrowing of the glottis and the vocal cords due to the air flow from the lungs. It is an ingredient.
[0004]
[Problems to be solved by the invention]
2. Description of the Related Art Conventionally, a speech synthesizer that can change the ratio of harmonic components and non-harmonic components is known (see, for example, JP-A-10-187180).
Even with the method described in this publication, it is possible to control breathability and dynamics as a result. However, in this method, the breathability only changes as a result of changing the ratio of the harmonic component and the nonharmonic component, and the breathability or the like cannot be positively controlled.
The present invention has been made in view of this point, and has a speech synthesizer, a speech synthesis method, a speech synthesis program, and a program for recording the program, which can easily control the level of breathability as desired. An object of the present invention is to provide a computer-readable recording medium.
[0005]
[Means for Solving the Problems]
To achieve the above object, a speech synthesizer according to a first invention of the present application is a speech synthesizer that synthesizes and outputs speech based on input performance data. A performance data input unit to which performance data including data Br and dynamics data Dy indicating dynamics is input, and a harmonic / nonharmonic component generation unit that generates a harmonic component H and an anharmonic component NH of the voice based on the performance data. When,
Using the breathability data Br and the dynamics data Dy , the magnitudes of the harmonic component H and the anharmonic component NH are respectively changed to the harmonic component H ′ after change and the anharmonic component NH ′ after change according to the following equations. An air breathing unit that changes the air to provide breathability;
H ′ = H + ΔH × Br
NH ′ = NH + (ΔNH1 + ΔNH2 × Dy) × Br
(Here, ΔH, ΔNH1, and ΔNH2 are numbers representing the degree of influence due to the increase and decrease of breathability data and dynamics data.)
Characterized in that a mixer for outputting the harmonic component H synthesized and synthesized speech signals a 'and inharmonic components NH after change' after the output from the breath-imparting portion changes.
[0010]
To achieve the above object, a speech synthesis method according to the second invention of the present application is a speech synthesis method for synthesizing and outputting speech by a speech synthesizer based on input performance data. the voice and performance data input step to let inputting performance data to the speech synthesizer, and a harmonic component H and the stochastic component NH speech based on said performance data including the dynamics data Dy showing a breathiness data Br and dynamics indicated harmony / stochastic component generating step Ru is generated by synthesizer, using the breathiness data Br and the dynamics data Dy, respectively below the magnitude of the harmonic component H and the stochastic component NH by the speech synthesizer breath of formula by is changed to 'stochastic component NH and after change' harmonic component H after change of Ru to impart breathiness to the audio And grant step,
H ′ = H + ΔH × Br
NH ′ = NH + (ΔNH1 + ΔNH2 × Dy) × Br
(Here, ΔH, ΔNH1, and ΔNH2 are numbers representing the degree of influence due to the increase and decrease of breathability data and dynamics data.)
And a synthesis step we leave for is synthesized to output the synthesized speech signal by a 'stochastic component NH and after the change' harmonic component H after the output from the breath-imparting step changes the speech synthesizer It is characterized by that.
[0011]
To achieve the above object, a speech synthesis program according to a third invention of the present application is a speech synthesis program for causing a computer to execute a procedure for synthesizing and outputting speech based on input performance data. and performance data input step of Ru is inputting performance data to the computer including the dynamics data Dy showing a breathiness data Br and dynamics indicating the magnitude of the harmonic component H of the speech based on the performance data and the stochastic component NH respectively harmony / stochastic component generating step Ru is generated by the computer, using the breathiness data Br and the dynamics data Dy, by the speech synthesizer the magnitude of the harmonic component H and the stochastic component NH wherein by changing the following harmonic component H after the change by the equation of 'and the stochastic component NH the changed' And breath-imparting step that Ru was granted a breathiness to voice,
H ′ = H + ΔH × Br
NH ′ = NH + (ΔNH1 + ΔNH2 × Dy) × Br
(Here, ΔH, ΔNH1, and ΔNH2 are numbers representing the degree of influence due to the increase and decrease of breathability data and dynamics data.)
That a synthesizing step of Ru and 'stochastic component NH and after the change' harmonic component H after change which is output from the breath-imparting step to output the synthesized speech signal by combining by the computer Features.
[0012]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, an embodiment of the present invention will be described by taking a singing voice synthesis device as an example.
As shown in FIG. 1, the singing voice synthesizer according to the present embodiment includes a performance data input unit 10, a harmonic / anharmonic component generator 20, an air breathing unit 30, and a mixer 40. These components can be realized by a normal computer and a computer program, but can of course be configured independently by hardware.
The performance data input unit 10 is a part for inputting various performance data for synthesizing the singing voice. In this embodiment, it is assumed that the performance data includes pitch data P, lyrics data L, singer name data S, dynamics data Dy, breathability data Br, and volume data V.
[0013]
The pitch data P is data indicating the pitch (pitch) of the singing voice. The lyrics data L is data representing the lyrics to be sung. The singer name data S is a singer identification number for reflecting the characteristics of the singer's voice in the synthesized singing voice. The breathability data Br is for representing the magnitude of breathability, and is represented by a numerical value between 0 and 1 here. The way of changing the harmonic component H and the anharmonic component NH is changed by increasing / decreasing the breath data Br. Details will be described later.
[0014]
The dynamics data Dy is for representing a sense of dynamics in the sense of hearing, and is represented by a numerical value between 0 and 1 here. When the dynamics data Dy is 0, the synthesized singing voice has the least dynamic feeling (the voice when a person sings with the smallest voice), and when the dynamics data Dy is 1, the synthesized singing voice is the largest. Of the dynamics (sound when a person sings with the loudest voice).
[0015]
The volume data V is for determining the volume of the synthesized singing voice and is expressed by a numerical value between 0 and 1. When the volume is 0, the volume of the synthesized singing voice is minimized, and when the volume is 1, the volume of the synthesized singing voice is maximized.
[0016]
The harmonic / anharmonic component generator 20 is a part that outputs a harmonic component H and an anharmonic component NH that match the input performance data. Here, the harmonic component H and the anharmonic component NH are assumed to be expressed by a frequency spectrum, but can also be expressed as a time waveform. The harmonic / anharmonic component generator 20 includes a database DB that stores different harmonic component data and anharmonic component data for each type of performance data. The harmonic / anharmonic component generator 20 acquires appropriate harmonic components and anharmonic components that match the performance data input from the performance data input unit 10 from the database DB and outputs them. If there is no harmonic component and anharmonic component that match the input performance data in the database DB, the approximate harmonic component and the anharmonic component may be read out and adjustment such as linear interpolation may be performed.
[0017]
Further, the breathing unit 30 changes the harmonic component H and the harmonic component NH output from the harmonic / nonharmonic component generator 20 based on the breath data Br input in the performance data input unit 10. Part. The mixer 40 is a part that synthesizes and outputs an audio signal by synthesizing the changed harmonic component and the non-harmonic component output from the breathing unit 30.
[0018]
Next, the operation of this embodiment will be described based on the flowchart shown in FIG.
First, various performance data are input in the performance data input unit 10 (S1).
[0019]
The harmonic / nonharmonic component generator 20 receives input of pitch data P, lyric data L, singer name data S, and dynamics data Dy among performance data input from the performance data input unit 10, and matches these data. The harmonic component data and the anharmonic component data are read from the database DB to generate the harmonic component H and the anharmonic component NH of the voice (S2). The harmonic component H generated here increases as the dynamics data Dy increases, as shown in FIG. On the other hand, the anharmonic component NH is substantially constant regardless of the change in the size of the dynamics data Dy, as shown in FIG. The reason why such a curve is formed is that the harmonic / anharmonic component generator 20 does not consider the breathability data Br as a factor.
[0020]
The breathability imparting device 30 receives the harmonic component H and the anharmonic component NH, and based on the singer name data S, the dynamics data Dy, and the breathability data Br input from the performance data input unit 10. The sizes of the component H and the anharmonic component NH are changed (S3).
[0021]
The magnitude H ′ of the harmonic component after the change, the magnitude NH ′ of the anharmonic component after the change, are the following in relation to the magnitude H of the harmonic component before the change, and the magnitude NH ′ of the anharmonic component before the change. It is expressed by the following formula.
[0022]
[Expression 1]
H´ ＝ H ＋ ΔH (S) × Br [dB] …… (1)
NH´ ＝ NH + (ΔNH1 (S) + ΔNH2 (S) × Dy) × Br [dB] …… (2)
[0023]
However, ΔH (S), ΔNH1 (S), and ΔNH2 (S) are coefficients determined by the singer name data S. As is clear from the equations (1) and (2), the greater the ΔH (S), the greater the influence on H ′ due to the increase / decrease in the breath data Br. Further, as ΔNH1 (S) increases, the degree of influence on NH ′ due to increase / decrease in breathability data Br increases, but the degree of influence on NH ′ due to increase / decrease in dynamics data Dy does not change. Further, as ΔNH2 (S) increases, the degree of influence on NH ′ due to the increase / decrease in breathability data Br and the degree of influence on NH ′ due to the increase / decrease in dynamics data Dy increase.
[0024]
The change amount (ΔH (S) × Br) of H ′ represented by the equation (1) in the above [Equation 1] is shown in the graph of FIG. 4A, and the change amount represented by the equation (2) ((ΔNH1 (S ) + ΔNH 2 (S) × Dy) × Br) is shown in the graph of FIG.
4 (a) and 4 (b), the horizontal axis represents the dynamics data Dy and the vertical axis represents the magnitude of change (dB).
[0025]
FIG. 5 is a graph showing how the magnitude H ′ of the harmonic component after change and the magnitude NH ′ of the anharmonic component after change are changed with respect to the change in the dynamics data Dy.
As shown in FIG. 5A, the slope of the straight line indicating the relationship between the dynamics data Dy and the harmonic component H ′ does not change even when the breathability data Br changes, but the intercept of the vertical axis changes. That is, the dynamics data Dy-harmonic component H ′ straight line is translated in the vertical axis direction by the change of the breath data Br.
[0026]
On the other hand, as shown in FIG. 5B, when the breathing data Br is 0, the magnitude of the anharmonic component NH ′ is constant regardless of the increase / decrease in the dynamics data Dy. When it is greater than 0, the anharmonic component NH ′ increases with an increase in the dynamics data Dy, and as the breathability data Br increases, the degree of change in the anharmonic component NH ′ with an increase in the dynamics data Dy also increases. . That is, as shown in FIG. 5B, as the breathability data Br increases, the slope of the change curve of the dynamics data Dy-anharmonic component NH ′ increases.
[0027]
FIG. 6 shows the relationship between the harmonic component H ′ and the anharmonic component NH ′ and the dynamics data Dy when the breath data Br = 0.0 (minimum) (FIG. 6A), the breath data Br = 1. The relationship between the harmonic component H, the anharmonic component NH ′, and the dynamics data Dy in the case of 0 (maximum) is shown (FIG. 5B).
[0028]
As shown in FIG. 6 (a), in the case of breathability data Br = 0.0, the harmonic component H ′ is increased as the dynamics data Dy increases. It is constant regardless of the dynamics data Dy. On the other hand, as shown in FIG. 6B, in the case of breathability data Br = 1.0, the harmonic component H ′ is increased as the dynamics data Dy increases, and the anharmonic component NH ′ is also increased. It is made to increase with the increase of the dynamics data Dy. As described above, when the size of the breath data Br differs, the manner in which the ratio of the harmonic component H ′ and the non-harmonic component NH ′ changes is changed even if the dynamics data Dy changes in the same manner.
[0029]
In actual human vocalization, when the glottal closure period is long, or when the closed period is incomplete and the ratio of DC airflow from the lungs is high, the voice is said to be “highly breathable”. In such a case, when speaking to increase the dynamics, the magnitude of the direct current air flow from the lungs itself increases, so the anharmonic component also increases as the dynamics increase.
When the degree of breathing is small, there is almost no DC airflow from the lungs, so the anharmonic component remains low and constant regardless of the dynamics.
The graph of FIG. 6 is in common with the characteristics of the utterance of an actual human voice.
Finally, the mixer 40 combines the changed harmonic component H ′ and the non-harmonic component NH ′ output from the breather 30 and synthesizes and outputs an audio signal (S4).
[0030]
As described above, according to the singing voice synthesizer of the present embodiment, it is possible to control a harmonic component and a non-harmonic component of a voice to be synthesized by breath data and dynamics data, thereby easily producing a natural and characteristic voice. It becomes possible to synthesize.
In addition, since the degree of breathing of dynamics can be controlled independently, even when synthesizing voices that are gradually made larger or smaller by changing the dynamics, natural breathing closer to human singing is achieved. It is possible to synthesize the voice that you have.
[0031]
In addition, by appropriately setting the degree of dynamics and breathability, it becomes possible to easily give a difference in breathability by the singer.
Moreover, even if there is no harmonic component and anharmonic component that match the input performance data in the database, it is possible to synthesize the desired performance data by adjusting the approximate harmonic component and the anharmonic component by interpolation. . For this reason, it is not necessary to store harmonic components and non-harmonic components that take a combination of all dynamics and breathability in the database, and the database can be made smaller.
[0032]
【The invention's effect】
As described above, according to the present invention, the magnitude of breathability can be easily controlled as desired.
[Brief description of the drawings]
FIG. 1 shows a configuration of a singing voice synthesizer according to an embodiment of the present invention.
FIG. 2 is a flowchart showing a state of processing by the apparatus of FIG.
3 is a graph showing the relationship between the harmonic component H and the anharmonic component NH of the audio output from the harmonic / anharmonic component generator 20 of FIG. 1 and the dynamics data Dy. FIG.
4 is a graph showing a relationship between a change amount for changing the harmonic component H and the anharmonic component NH and dynamics data Dy in the breathability imparting device 30 of FIG. 1; FIG.
FIG. 5 is a graph showing the relationship between the changed harmonic component H ′ and the anharmonic component NH ′ output from the breathability imparting device 30 and the dynamics data Dy.
FIG. 6 is a graph for explaining how the relationship between the harmonic component H ′ and the anharmonic component NH ′ and the dynamics data Dy changes when the breathability data Br is different.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 10 ... Performance data input part 20 ... Harmonic / nonharmonic component generator 30 ... Breathing applicator 40 ... Mixer

Claims (5)

In a speech synthesizer that synthesizes and outputs speech based on input performance data,
A performance data input unit to which performance data including breathability data Br indicating the level of breathability and dynamics data Dy indicating dynamics is input;
A harmonic / nonharmonic component generating unit that generates a harmonic component H and an anharmonic component NH of the voice based on the performance data;
Using the breathability data Br and the dynamics data Dy , the magnitudes of the harmonic component H and the anharmonic component NH are respectively changed to the harmonic component H ′ after change and the anharmonic component NH ′ after change according to the following equations. An air breathing unit that changes the air to provide breathability;
H ′ = H + ΔH × Br
NH ′ = NH + (ΔNH1 + ΔNH2 × Dy) × Br
(Here, ΔH, ΔNH1, and ΔNH2 are numbers representing the degree of influence due to the increase and decrease of breathability data and dynamics data.)
Speech synthesis is characterized in that a mixer for outputting the synthesized and the synthesized speech signal and a 'non-harmonic component NH and after the change' harmonic component H after output from the breath-imparting unit the change apparatus. The speech synthesizer according to claim 1 , wherein ΔH, ΔNH1, and ΔNH2 are coefficients determined by singer name data S. In a speech synthesis method for synthesizing and outputting speech by a speech synthesizer based on input performance data,
A performance data input step for causing the speech synthesizer to input performance data including breathability data Br indicating the level of breathability and dynamics data Dy indicating dynamics;
Harmony / stochastic component generating step Ru is generated by the speech synthesizer and a harmonic component H and the stochastic component NH voice on the basis of the performance data,
Using the breathability data Br and the dynamics data Dy , the speech synthesizer uses the following equations to change the magnitudes of the harmonic component H and the nonharmonic component NH, respectively, according to the following formulas. a breath-imparting step conditioner is changed into components NH 'and Ru is imparting breathiness to the audio,
H ′ = H + ΔH × Br
NH ′ = NH + (ΔNH1 + ΔNH2 × Dy) × Br
(Here, ΔH, ΔNH1, and ΔNH2 are numbers representing the degree of influence due to the increase and decrease of breathability data and dynamics data.)
And a synthesis step we leave for is synthesized to output the synthesized speech signal by a 'stochastic component NH and after the change' harmonic component H after the output from the breath-imparting step changes the speech synthesizer A speech synthesis method characterized by the above. In a speech synthesis program for causing a computer to execute a procedure of synthesizing and outputting speech based on input performance data,
And performance data input step of performance data Ru is input to the computer including the dynamics data Dy showing a breathiness data Br and dynamics showing the breathiness magnitude,
Harmony / stochastic component generating step Ru is generated by the computer and a harmonic component H and the stochastic component NH voice on the basis of the performance data,
Using the breathability data Br and the dynamics data Dy, the speech synthesizer uses the following equations to change the magnitudes of the harmonic component H and the nonharmonic component NH, respectively, according to the following formulas. a breath-imparting step conditioner is changed into components NH 'and Ru is imparting breathiness to the audio,
H ′ = H + ΔH × Br
NH ′ = NH + (ΔNH1 + ΔNH2 × Dy) × Br
(Here, ΔH, ΔNH1, and ΔNH2 are numbers representing the degree of influence due to the increase and decrease of breathability data and dynamics data.)
That a synthesizing step of Ru and 'stochastic component NH and after the change' harmonic component H after change which is output from the breath-imparting step to output the synthesized speech signal by combining by the computer A special speech synthesis program. A computer-readable recording medium on which the speech synthesis program according to claim 4 is recorded.

Images