JP2004061793A - Apparatus, method, and program for singing synthesis - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To keep tone quality such as a timbre as it is when a singing voice having the same pitch as a singing voice stored as speech phoneme data for singing synthesis in a database and to change the tone quality such as the timbre only when a singing voice having a different pitch is synthesized. <P>SOLUTION: When speech phoneme data corresponding to MIDI data (pitch data, lyrics data, etc.) inputted to a synthesis parameter generator 12 are selected from the database 11, synthesis parameters PA as a set of various parameters PAn and pitch data (database pitch data) DP included in the speech phoneme data are extracted. Further, synthesis pitch data SP representing the pitch of a speech to be synthesized in a time series are generated on the basis of the inputted pitch data. The individual parameters PAn are given a key scaling effect on the basis of the synthesis pitch data SP and database pitch data DP by using a key scaling function fn. <P>COPYRIGHT: (C)2004,JPO

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a singing voice synthesizing device, a singing voice synthesizing method, and a singing voice synthesizing program that synthesize voice based on input performance data.
[0002]
[Prior art]
In electronic musical instruments, pitch conversion is performed by a function called key scaling. Specifically, tone conversion is performed by adding or subtracting a certain value depending on a pitch corresponding to a pressed key among the keys of an electronic piano to a tone parameter (for example, Japanese Patent Laid-Open No. 58-211786). Reference).
For example, in a karaoke apparatus or the like, a singing / synthesizing apparatus having a pitch conversion function is provided for purposes such as converting a male voice into a female voice. In this case, simply converting the pitch of the original singing voice to a different pitch results in an unnatural voice, so other parameters used for singing synthesis, such as timbre, can also be adjusted using the key scaling function similar to electronic musical instruments. Is converted to match.
[0003]
[Problems to be solved by the invention]
In the singing voice synthesizing device, the parameters used for singing voice synthesis are obtained by extracting the actually recorded singing data for each phoneme, and the pitch at the time of recording differs for each phoneme. For this reason, in the key scaling of the parameter simply according to the pitch to be synthesized, the timbre changes even when a singing voice having the same pitch as the pitch at the time of recording the phoneme is synthesized. In order to solve this, each parameter must hold a key scaling function that specifies the scaling amount according to the pitch for each phoneme, so that the parameter does not change when the pitch is the same as when recording the phoneme. there were. However, in this case, since the number of key scaling functions becomes enormous, there is a problem that it is difficult to create and change the key scaling functions.
The present invention introduces this key scaling function into a singing voice synthesizing device, a singing voice synthesizing method, and a singing voice synthesizing program. When trying to synthesize a singing voice, keep the tone and other sound quality as it is, and change the tone and other sound quality only when trying to synthesize a singing voice with a pitch different from the pitch of the original singing voice. The purpose is to:
[0004]
[Means for Solving the Problems]
To achieve the above object, a singing voice synthesizing device according to a first invention of the present application includes a voice information input unit for inputting voice information indicating the content of a voice to be synthesized, and voice segment data for synthesizing voice. And a selection unit that selects the speech unit data stored in the phoneme database based on the speech information, and outputs synthesized pitch data indicating a pitch of a speech to be synthesized in a time series. A synthesis pitch data output unit, a synthesis parameter output unit for extracting and outputting a synthesis parameter from the speech unit data selected by the selection unit, and a speech unit data used for extracting the synthesis parameter. A database pitch data generating unit for extracting pitch data to be output and outputting the same as database pitch data; A key scaling function storage unit for storing a scaling function, a function value obtained by substituting the synthesized pitch data into the key scaling function, and a function value obtained by substituting the database pitch data into the key scaling function. And a key scaling unit that corrects the synthesis parameter based on the difference and outputs a correction parameter, and a waveform synthesis unit that synthesizes a waveform based on the correction parameter.
[0005]
According to the singing voice synthesizing device according to the first aspect, the speech unit data stored in the phoneme database is selected by the selection unit based on the input voice information. Then, synthesized pitch data indicating the pitch of the voice to be synthesized in a time series is output. In addition, a synthesis parameter for each time is extracted from the speech unit data selected by the selection unit. Further, pitch data constituting the speech unit data used in extracting the synthesis parameters is extracted and output as database pitch data.
If there is a difference between the synthesized pitch data and the database pitch data, the key scaling function corrects the synthesized parameter in accordance with the change in pitch, thereby enhancing the naturalness of the synthesized output voice. On the other hand, if there is no difference between them, the correction of the synthesis parameter is not performed. Therefore, it is possible to enhance the naturalness of the output singing voice without preparing a key scaling function for each phoneme or time.
[0006]
To achieve the above object, a singing voice synthesizing method according to a second invention of the present application includes a voice information inputting step of inputting voice information indicating the content of a voice to be synthesized, and voice segment data for voice synthesis. In the phoneme database in advance, and selecting speech unit data stored in the phoneme database based on the speech information, and outputting synthesized pitch data indicating the pitch of the speech to be synthesized in a time series. Synthesizing pitch data outputting step, synthesizing parameter outputting step of extracting and outputting a synthesizing parameter from the speech unit data selected in the selecting step, and speech unit data used in extracting the synthesizing parameter. Database pitch data that extracts the constituent pitch data and outputs it as database pitch data And a key scaling function is prepared for each of the synthesis parameters, and a function value obtained by substituting the synthesized pitch data for the key scaling function and the database pitch data are substituted for the key scaling function. A key scaling step of correcting the synthesis parameter based on a difference from the obtained function value and outputting a correction parameter; and a waveform synthesis step of synthesizing a waveform based on the correction parameter.
[0007]
To achieve the above object, a singing voice synthesizing program according to a third invention of the present application includes a voice information inputting step of inputting voice information indicating the content of a voice to be synthesized, and a voice unit for synthesizing voice. A step of storing data in advance in a phoneme database and selecting speech unit data stored in the phoneme database based on the speech information, and synthesizing pitch data indicating a pitch of a speech to be synthesized in a time series. A synthesis pitch data output step for outputting, a synthesis parameter output step for extracting and outputting a synthesis parameter from the speech unit data selected in the selection step, and a speech unit data used for extracting the synthesis parameter. Database pitch that extracts pitch data that constitutes In the data generation step, a key scaling function is prepared for each of the synthesis parameters, and a function value obtained by substituting the synthesized pitch data into the key scaling function and the database pitch data are substituted into the key scaling function. The computer is configured to execute a key scaling step of correcting the synthesis parameter based on the difference between the obtained function values and outputting a correction parameter, and a waveform synthesis step of synthesizing a waveform based on the correction parameter. It is characterized by the following.
[0008]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
FIG. 1 is a block diagram showing a schematic configuration of a singing voice synthesizing device according to an embodiment of the present invention. As shown in FIG. 1, the singing voice synthesizing device 1 according to the present embodiment includes a voice database 11, a synthesizing parameter generating device 12, a key scaling function storage unit 13, a key scaling device 14, a waveform synthesizing device 15, It has.
[0009]
The speech database 11 is data obtained by cutting out a signal such as singing data or the like which is actually recorded or acquired for each phoneme. As shown in FIG. 2, a phoneme transition portion (a connection portion between phonemes and a phoneme (example: a -E, ai, etc.) and a stationary part (a part where a phoneme (eg, a, i, etc.) is steadily pronounced) is stored. The segment data is represented by a synthesis parameter PA.
The synthesis parameter PA includes a plurality of types (here, N types) of parameters PAn (n = 1, 2,..., N) such as a formant center frequency, a bandwidth, and a gain. Each parameter PAn is stored as a data array or function that can be represented by a graph with the horizontal axis representing time t (t = 0 to T) and the vertical axis representing parameter values.
Different speech unit data stores different parameters PAn. For example, parameters PAn having different formant center frequencies and gains are stored in the phoneme transition units ae and ai.
[0010]
Further, in addition to these parameters PAn, pitch data DP (hereinafter, referred to as database pitch DP) of the speech unit at each time is stored in the data of each phoneme transition portion and the steady portion one by one. I have.
[0011]
The synthesis parameter generation device 12 includes an input unit for inputting MIDI data (pitch, lyrics, etc.) representing a singing voice to be synthesized, and speech unit data (phonemic transition data or stationary partial data) corresponding to the input data. ) Functions as a synthesis parameter generation unit that reads out the synthesis parameter PA from the audio database 11.
[0012]
Further, the synthesis parameter generation device 12 determines an accurate pitch at each time of the output singing voice to be synthesized based on the pitch data and the lyrics data input to the synthesis parameter generation device 12 as MIDI data, and synthesizes this. It functions as a synthesized pitch data output unit that outputs the pitch data SP to the key scaling device 14. This pitch determination process is performed in consideration of data such as phonemes and pitches of lyrics for voices positioned before and after the voice to be synthesized.
Further, the synthesis parameter generation device 12 functions as a database pitch data generation unit that reads the database pitch DP included in the read speech unit data and outputs the read data to the key scaling device 14.
[0013]
The key scaling function storage unit 13 stores a number (here, N) of key scaling functions fn corresponding to the number of each synthesis parameter PAn.
The key scaling device 14 reads out the key scaling function fn (n = 1, 2,... N) corresponding to each composite parameter PAn stored in the key scaling function storage unit 13 and inputs the key scaling function fn. A key scaling effect is applied to each of the synthesis parameters PAn.
With this key scaling effect, for example, the synthesis parameter is adjusted according to the pitch of the singing voice to be synthesized, such as raising the pitch of the voice when generating a high sound, or making the voice difficult to hear when generating a low sound. Can be adjusted, so that natural speech synthesis is possible. Further, since it is not necessary to provide a voice database for each pitch to be synthesized, the size of the voice database can be small.
Then, when using the database pitch DP to synthesize a singing voice having the same pitch as that stored in the voice database 11 with the synthesis parameter PAn provided with the key scaling effect, the synthesis parameter PAn is used as it is. When synthesizing singing voices of different pitches, a process of changing the synthesis parameter PAn is performed. Specifically, after obtaining the function values fn (SP) and fn (DP) (see FIG. 3) obtained by substituting the synthesized pitch SP and the database pitch DP into the function fn, the following [Equation 1] is obtained. As shown, a key scaling effect is applied to each parameter PAn using the difference between the two function values (fn (DP) −fn (SP) = 0.1 in FIG. 3) to obtain a correction parameter PAn ′. That is, the correction parameter PAn ′ is a parameter obtained by correcting the parameter PAn with the composite pitch PAn using the composite pitch SP and the database pitch DP.
[0014]
(Equation 1)
PAn '= PAn + fn (SP) -fn (DP)
[0015]
Further, as shown in the following [Equation 2], a key scaling effect is provided by multiplying each parameter PAn by a value obtained by adding a constant (for example, 1) to the difference between the function values fn (SP) and fn (DP). You may.
(Equation 2)
PAn ′ = PAn × (fn (SP) −fn (DP) +1.0)
[0016]
Which of the above [Equation 1] and [Equation 2] is selected depends on the properties of each parameter PAn. For example, when the value of the parameter PAn is expressed by a logarithm such as a gain, [Equation 1] is appropriate, and when the value of the parameter PAn is expressed by a linear value such as a frequency (Hz), [ Equation 2] is appropriate.
[0017]
In both cases of [Equation 1] and [Equation 2], when SP = DP, PAn ′ = PAn. That is, when the pitch SP to be synthesized is equal to the database pitch DP stored in the database 11, the singing voice synthesizing device 1 of the present embodiment does not change the synthesis parameter PA. .
The waveform synthesizing device 15 has a function of synthesizing the speech unit data represented by the correction parameter PAn ′ given the key scaling effect and the synthesized pitch SP, superimposing the synthesized speech unit data, and outputting an output singing voice waveform.
[0018]
Next, the operation of the singing voice synthesizing apparatus 1 according to the present embodiment will be described based on the flowchart shown in FIG. FIG. 4 is a flowchart showing a procedure of processing within one piece of speech unit data, which is used for all speech selected from the speech database 11 based on MIDI data input to the synthesis parameter generation device 12. By sequentially executing the segment data and synthesizing them by the waveform synthesizing device 15, a synthesized singing voice is obtained.
[0019]
In the flowchart shown in FIG. 4, each parameter PAn (n = 1, 2,..., N) in one piece of speech unit data is processed in order from 1 to N. Further, since each parameter PAn is represented by a function of time t, in the processing of each parameter PAn, the synthesized pitch data SP (t) and the database pitch DP (t) are acquired and corrected at each time t. The value of the parameter PAn (t) 'is obtained. More specifically, first, a variable n indicating the type of each parameter PAn is initialized to 1 (S1), and a scaling function fn corresponding to the value of n is read from the scaling function storage unit 13 (S2). Subsequently, the time t is initialized to 0 (S3), and the parameter PAn (t) at the time t is obtained (S4).
[0020]
Next, the values of the synthesized pitch data SP (t) and the database pitch data DP (t) at the time t are acquired (S5). Then, based on the values acquired in S4 and S5, the correction parameter PAn (t) 'is calculated by [Equation 1] or [Equation 2] (S6). In S6, either one of [Equation 1] or [Equation 2] may be fixedly used, or either [Equation 1] or [Equation 2] may be used depending on the type of the parameter PAn. It may be made to automatically select whether to use.
[0021]
In this way, for all times t (t = 0 to T), the correction parameter PAn ′ (t) is calculated by repeating the procedure of S4 to S6 while incrementing t by Δt (S7, S8). In the flowchart of FIG. 4, the discrete correction parameter value PAn ′ (t) is calculated for each Δt, so that when an intermediate value of each data is required, the calculation may be performed by interpolation.
The procedure described above is repeated until calculation is performed for all parameters PAn (n = 1, 2,... N) (S9, S10). The singing voice is synthesized and output by synthesizing the waveform obtained based on the data obtained by the waveform synthesizing device 15.
[0022]
Although the embodiment of the present invention has been described above, the present invention is not limited to this. For example, in the speech database 101, in addition to the phoneme transition portion and the steady portion, data (timbre) of a specific phoneme portion ) May be held.
The database pitch DP may store pitch data at each time of the speech unit data as described above, but may be stored as one pitch data representative of each time of the speech unit data. May be.
The MIDI data input to the synthesis parameter generation device 12 is not limited to the pitch and lyrics, but may be data such as dynamics and vibrato.
Further, instead of determining the synthesized pitch data SP based on the pitch data and the lyrics data as MIDI data, the synthesized pitch data SP data may be stored in advance as MIDI data.
The data input to the synthesis parameter generation unit 12 is not limited to MIDI data, but may be any data as long as performance data can be specified in a time series so as to generate a singing voice to be synthesized.
[0023]
【The invention's effect】
As described above, according to the present invention, since the pitch of the speech unit is stored as the database pitch, the singing voice having the same pitch as the pitch of the original singing voice is synthesized with a small key scaling function. In such a case, the tone quality such as the tone color can be changed only when the tone quality such as the tone color is kept as it is, and the singing voice having a pitch different from the pitch of the original singing voice is to be synthesized.
[Brief description of the drawings]
FIG. 1 is a block diagram showing an entire configuration of a singing voice synthesizing apparatus according to an embodiment of the present invention.
FIG. 2 conceptually shows the contents of data stored in a voice database shown in FIG.
FIG. 3 shows the contents of a key scaling function fn.
FIG. 4 is a flowchart showing the operation of the singing voice synthesizing device 1 shown in FIG.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 11 ... Speech database, 12 ... Synthesis parameter generation device, 13 ... Key scaling function storage part 14 ... Key scaling device, 15 ... Waveform synthesis device

Claims (6)

A voice information input unit for receiving voice information indicating the content of voice to be synthesized;
A phoneme database storing speech unit data for synthesizing speech,
A selection unit that selects the speech unit data stored in the phoneme database based on the speech information,
A synthesized pitch data output unit that outputs synthesized pitch data indicating a pitch of a voice to be synthesized in a time series,
A synthesis parameter output unit that extracts and outputs synthesis parameters from the speech unit data selected by the selection unit,
A database pitch data generation unit that extracts pitch data constituting speech unit data used when extracting the synthesis parameter and outputs the extracted data as database pitch data,
A key scaling function storage unit that stores a key scaling function prepared for each of the synthesis parameters;
Correcting the synthesis parameter based on a difference between a function value obtained by substituting the synthesized pitch data into the key scaling function and a function value obtained by substituting the database pitch data into the key scaling function. A key scaling unit that outputs correction parameters,
A singing voice synthesizing device, comprising: a waveform synthesizing unit that synthesizes a waveform based on the correction parameter. The singing voice synthesizing apparatus according to claim 1, wherein the synthesized pitch data output unit generates synthesized pitch data based on pitch information included in the voice information, and outputs the generated synthesized pitch data. The singing voice synthesizing device according to claim 1, wherein the key scaling unit adds a difference between the two function values to the various parameters. The singing voice synthesizer according to claim 1, wherein the key scaling unit multiplies the various parameters by a value obtained by adding 1 to a difference between the two function values. A voice information input step of inputting voice information indicating the content of the voice to be synthesized;
A selection step of storing speech unit data for synthesizing speech in advance in a phoneme database, and selecting speech unit data stored in the phoneme database based on the speech information,
A synthesized pitch data output step of outputting synthesized pitch data indicating a pitch of a voice to be synthesized in a time series;
A synthesis parameter output step of extracting and outputting a synthesis parameter from the speech unit data selected in the selection step,
A database pitch data generating step of extracting pitch data constituting speech unit data used when extracting the synthesis parameter and outputting the extracted pitch data as database pitch data;
A key scaling function is prepared for each of the synthesis parameters, and a function value obtained by substituting the synthesized pitch data into the key scaling function and a function value obtained by substituting the database pitch data into the key scaling function A key scaling step of correcting the synthesis parameter and outputting a correction parameter based on the difference between
And a waveform synthesizing step of synthesizing a waveform based on the correction parameter. A voice information input step of inputting voice information indicating the content of the voice to be synthesized;
A selection step of storing speech unit data for synthesizing speech in advance in a phoneme database, and selecting speech unit data stored in the phoneme database based on the speech information,
A synthesized pitch data output step of outputting synthesized pitch data indicating a pitch of a voice to be synthesized in a time series;
A synthesis parameter output step of extracting and outputting a synthesis parameter from the speech unit data selected in the selection step,
A database pitch data generating step of extracting pitch data constituting speech unit data used when extracting the synthesis parameter and outputting the extracted pitch data as database pitch data;
A key scaling function is prepared for each of the synthesis parameters, and a function value obtained by substituting the synthesized pitch data into the key scaling function and a function value obtained by substituting the database pitch data into the key scaling function. A key scaling step of correcting the synthesis parameter and outputting a correction parameter based on the difference
And a waveform synthesizing step for synthesizing a waveform based on the correction parameter.

Images