JP3979213B2 - Singing synthesis device, singing synthesis method and singing synthesis program - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a singing voice synthesizing device, a singing voice synthesis method, and a singing voice synthesis program for synthesizing voice based on input performance data.
[0002]
[Prior art]
In an electronic musical instrument, pitch conversion is performed by a function called key scaling. Specifically, the tone color conversion is performed by adding or subtracting a certain value depending on the pitch corresponding to the pressed key among the keys of the electronic piano to the tone color parameter (for example, Japanese Patent Laid-Open No. 58-211786). Issue gazette).
For example, in a karaoke apparatus or the like, a singing synthesizer having a pitch conversion function is provided for use such as converting a male voice into a female voice. In this case, simply converting the pitch of the original singing voice to a different pitch will result in unnatural voice, so other parameters used for singing synthesis, such as timbre, will also use the same key scaling function as the electronic musical instrument. It is converted to match.
[0003]
[Problems to be solved by the invention]
In the singing voice synthesizing apparatus, the parameters used for singing voice synthesis are obtained by cutting out and extracting the actually recorded singing data for each phoneme, and the pitch at the time of recording is different for each phoneme. For this reason, in the key scaling of the parameter according to the pitch to be synthesized simply, the timbre changes even when the singing voice having the same pitch as the pitch at the time of recording the phoneme is synthesized. In order to solve this problem, each parameter must hold a key scaling function that defines a scaling amount corresponding to the pitch for each phoneme so that the parameter does not change at the same pitch as when recording the phoneme. there were. However, if this is done, the number of key scaling functions becomes enormous, which makes it difficult to create and change the key scaling functions.
The present invention introduces this key scaling function into a singing synthesizer, a singing synthesis method, and a singing synthesis program, and the pitch of the same singing voice as that of the original singing voice can be obtained without increasing the number of key scaling functions. When trying to synthesize a singing voice, keep the tone quality such as the timbre as it is, and only when trying to synthesize a singing voice with a pitch different from the pitch of the original singing voice, the tone quality such as the timbre can be changed For the purpose.
[0004]
[Means for Solving the Problems]
In order to achieve the above object, a singing voice synthesizing apparatus according to the first invention of the present application includes a voice information input unit for inputting voice information indicating the contents of voice to be synthesized, and voice unit data for synthesizing voice Is stored, a selection unit that selects the speech segment data stored in the phoneme database based on the speech information, and synthesized pitch data that indicates the pitch of the speech to be synthesized in time series A synthesis pitch data output unit, a synthesis parameter output unit for extracting and outputting a synthesis parameter from the speech unit data selected by the selection unit, and a speech unit data used for extracting the synthesis parameter Database pitch data generation unit for extracting pitch data to be output and outputting it as database pitch data, and prepared for each of the synthesis parameters A key scaling function storage unit for storing a scaling function, a function value obtained by substituting the synthesized pitch data into the key scaling function, and a function value obtained by substituting the database pitch data into the key scaling function, A key scaling unit that corrects the synthesis parameter and outputs a correction parameter based on the difference, and a waveform synthesis unit that synthesizes a waveform based on the correction parameter.
[0005]
According to the singing voice synthesizing apparatus according to the first aspect of the invention, the speech unit data stored in the phonological database is selected by the selection unit based on the input speech information. Then, synthesized pitch data indicating the pitch of the voice to be synthesized in time series is output. Also, synthesis parameters for each time are extracted from the speech segment data selected by the selection unit. Further, pitch data constituting the speech segment data used when extracting the synthesis parameters is extracted and output as database pitch data.
When there is a difference between the synthesized pitch data and the database pitch data, the synthesis parameter is corrected in accordance with the change of the pitch by the key scaling function, and the naturalness of the synthesized output voice is enhanced. On the other hand, when there is no difference between them, the synthesis parameter is not corrected. For this reason, the naturalness of the output singing voice can be improved without preparing a key scaling function for each phoneme and time.
[0006]
In order to achieve the above object, a singing synthesis method according to the second invention of the present application includes a speech information input step for inputting speech information indicating the content of speech to be synthesized, and speech segment data for synthesizing speech. Is stored in the phoneme database in advance, and a selection step of selecting speech segment data stored in the phoneme database based on the speech information, and synthetic pitch data indicating the pitch of speech to be synthesized in time series are output. A synthesis pitch data output step, a synthesis parameter output step for extracting and outputting a synthesis parameter from the speech unit data selected in the selection step, and a speech unit data used for extracting the synthesis parameter. Database pitch data that extracts the pitch data to be configured and outputs it as database pitch data Preparing a key scaling function for each synthesis parameter, and substituting the function value obtained by substituting the synthetic pitch data into the key scaling function and the database pitch data into the key scaling function. A key scaling step for correcting the synthesis parameter and outputting a correction parameter based on a difference from the obtained function value, and a waveform synthesis step for synthesizing a waveform based on the correction parameter are provided.
[0007]
To achieve the above object, a singing synthesis program according to the third invention of the present application includes a speech information input step for inputting speech information indicating the content of speech to be synthesized, and a speech unit for synthesizing speech. A step of selecting the speech segment data stored in the phonological database based on the speech information, and the synthesized pitch data indicating the pitch of the speech to be synthesized in time series. A synthesis pitch data output step to output; a synthesis parameter output step to extract and output synthesis parameters from the speech segment data selected in the selection step; and speech segment data used when extracting the synthesis parameters. A database pin that extracts the pitch data that make up the database and outputs it as database pitch data. A data generation step, and a key scaling function is prepared for each synthesis parameter, and a function value obtained by substituting the synthetic pitch data into the key scaling function and the database pitch data are substituted into the key scaling function. Based on the obtained function value difference, the computer is configured to execute a key scaling step for correcting the synthesis parameter and outputting a correction parameter, and a waveform synthesis step for synthesizing a waveform based on the correction parameter. It is characterized by that.
[0008]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
FIG. 1 is a block diagram showing a schematic configuration of a singing voice synthesizing apparatus according to an embodiment of the present invention. As shown in FIG. 1, the singing voice synthesis apparatus 1 according to the present embodiment includes a voice database 11, a synthesis parameter generation apparatus 12, a key scaling function storage unit 13, a key scaling apparatus 14, and a waveform synthesis apparatus 15. It has.
[0009]
The speech database 11 is data obtained by cutting out a signal such as singing data actually recorded or acquired for each phoneme, and as shown in FIG. 2, a phoneme transition portion (a connection portion between phonemes and phonemes (example: a -E, a-i, etc.)) and stationary segments (portions where phonemes (eg, a, i) etc. are steadily pronounced) are stored. The segment data is expressed by a synthesis parameter PA.
The synthesis parameter PA is composed of a plurality of types of parameters PAn (n = 1, 2,..., N) such as formant center frequency, bandwidth, and gain. Each parameter PAn is stored as a data array or function that can be expressed by a graph with the horizontal axis representing time t (t = 0 to T) and the vertical axis representing parameter values.
Different speech unit data stores different parameters PAn. For example, parameters PAn having different formant center frequencies and gains are stored in the phoneme transition part ae and the phoneme transition part ai.
[0010]
In addition to these parameters PAn, pitch data DP at each time of the speech segment (hereinafter referred to as database pitch DP) is stored one by one in the data of each phoneme transition part and stationary part. Yes.
[0011]
The synthesis parameter generation device 12 inputs an input unit for inputting MIDI data (pitch, lyrics, etc.) expressing a singing voice to be synthesized, and speech unit data (phoneme transition unit data or stationary partial data) corresponding to the input data. ) From the voice database 11 as a synthesis parameter PA.
[0012]
Further, the synthesis parameter generation device 12 determines an accurate pitch at each time of the output singing voice to be synthesized based on the pitch data and lyrics data input to the synthesis parameter generation device 12 as MIDI data, and synthesizes this. It functions as a synthesized pitch data output unit that outputs the pitch data SP to the key scaling device 14. This pitch determination process is performed in consideration of data such as the phonological and pitch of the lyrics of the speech positioned before and after the speech to be synthesized.
In addition, the synthesis parameter generation device 12 functions as a database pitch data generation unit that reads the database pitch DP included in the read speech segment data and outputs it to the key scaling device 14.
[0013]
The key scaling function storage unit 13 stores a number (here, N) of key scaling functions fn corresponding to the number of each synthesis parameter PAn.
The key scaling device 14 reads the key scaling function fn (n = 1, 2,... N) corresponding to each synthesis parameter PAn stored in the key scaling function storage unit 13, and inputs this key scaling function fn. Applied to each synthesized parameter PAn to give a key scaling effect.
Due to this key scaling effect, for example, when generating a high sound, the voice becomes high-pitched, and when generating a low voice, the voice is difficult to hear, so that the synthesis parameter matches the pitch of the singing voice to be synthesized. Can be adjusted, so that natural speech synthesis is possible. In addition, since it is not necessary to provide an audio database for each pitch to be synthesized, the size of the audio database can be small.
Then, when synthesizing a singing voice having the same pitch as that stored in the voice database 11 with respect to the synthesis parameter PAn given the key scaling effect using the database pitch DP, the synthesis parameter PAn is used as it is. In order to synthesize singing voices with different pitches, the synthesis parameter PAn is changed. Specifically, after obtaining function values fn (SP) and fn (DP) (see FIG. 3) obtained by substituting the synthetic pitch SP and the database pitch DP into the function fn, the following [Equation 1] is obtained. As shown, a correction parameter PAn ′ is obtained by applying a key scaling effect to each parameter PAn using the difference between both function values (fn (DP) −fn (SP) = 0.1 in the case of FIG. 3). That is, the correction parameter PAn ′ is a parameter obtained by correcting the parameter PAn with the synthetic pitch PAn by the synthetic pitch SP and the database pitch DP.
[0014]
[Expression 1]
PAn ′ = PAn + fn (SP) −fn (DP)
[0015]
Further, as shown in [Expression 2] below, each parameter PAn is multiplied by a value obtained by adding a constant (for example, 1) to the difference between the function values fn (SP) and fn (DP), thereby providing a key scaling effect. May be.
[Expression 2]
PAn ′ = PAn × (fn (SP) −fn (DP) +1.0)
[0016]
Which of the above [Equation 1] and [Equation 2] is selected is determined by the property of each parameter PAn. For example, when the value of the parameter PAn is expressed by a logarithm such as a gain, [Equation 1] is appropriate, and when expressed by a linear value such as a frequency (Hz), [ Equation 2] is appropriate.
[0017]
In both cases of [Equation 1] and [Equation 2], when SP = DP, PAn ′ = PAn. That is, the singing voice synthesizing apparatus 1 of the present embodiment does not change the synthesis parameter PA when the pitch SP to be synthesized is equal to the database pitch DP stored in the database 11. .
The waveform synthesizer 15 has a function of synthesizing and superimposing the speech segment data expressed by the correction parameter PAn ′ given the key scaling effect and the synthesized pitch SP and outputting the synthesized speech segment waveform as an output singing voice waveform.
[0018]
Next, the effect | action of the song synthesizing | combining apparatus 1 which concerns on this Embodiment is demonstrated based on the flowchart shown in FIG. The flowchart of FIG. 4 shows the procedure of processing within one speech segment data, and all the speeches selected from the speech database 11 based on the MIDI data input to the synthesis parameter generation device 12 are shown. Synthetic singing voices are obtained by sequentially executing the segment data and synthesizing them by the waveform synthesizer 15.
[0019]
Further, in the flowchart shown in FIG. 4, each parameter PAn (n = 1, 2,... N) in one speech segment data is processed in order from 1 to N. Further, since each parameter PAn is expressed by a function at time t, in the processing of each parameter PAn, the synthesized pitch data SP (t) and the database pitch DP (t) are acquired and corrected at each time t. The value of the parameter PAn (t) ′ is obtained. More specifically, first, a variable n indicating the type of each parameter PAn is initialized to 1 (S1), and a scaling function fn corresponding to the value of n is read from the scaling function storage unit 13 (S2). Subsequently, time t is initialized to 0 (S3), and parameter PAn (t) at that time t is acquired (S4).
[0020]
Next, the values of the synthesized pitch data SP (t) and database pitch data DP (t) at the time t are acquired (S5). Then, based on the values acquired in S4 and S5, the correction parameter PAn (t) ′ is calculated by [Equation 1] or [Equation 2] (S6). In S6, either [Equation 1] or [Equation 2] may be used in a fixed manner, and either [Equation 1] or [Equation 2] is selected depending on the type of parameter PAn. You may make it select automatically whether to use.
[0021]
Thus, for all times t (t = 0 to T), the correction parameter PAn ′ (t) is calculated by repeating the steps S4 to S6 while incrementing t by Δt (S7, S8). In the flowchart of FIG. 4, since the discrete correction parameter value PAn ′ (t) is calculated for each Δt, if an intermediate value of each data is required, it may be calculated by interpolation.
The above-described procedure is repeated until all parameters PAn (n = 1, 2,... N) are calculated (S9, S10). By synthesize | combining the waveform obtained based on the data obtained by this by the waveform synthesizer 15, a singing voice is synthesize | combined and output.
[0022]
The embodiment of the present invention has been described above, but the present invention is not limited to this. For example, in the speech database 101, in addition to the phoneme transition portion and the steady portion, the data (timbre portion) of a specific phoneme portion is described. ) May be held.
The database pitch DP may store the pitch data at each time of the speech unit data as described above, but is stored as one pitch data representing each time of the speech unit data. May be.
Further, the MIDI data input to the synthesis parameter generation device 12 is not limited to pitch and lyrics, but may be data such as dynamics and vibrato.
Further, instead of determining the synthesized pitch data SP based on the pitch data and lyrics data as MIDI data, the synthesized pitch data SP data may be held in advance as MIDI data.
Further, the data input to the synthesis parameter generation unit 12 is not limited to MIDI data, but may be any data that can specify performance data in time series so that the singing voice to be synthesized is generated.
[0023]
【The invention's effect】
As described above, according to the present invention, since the pitch of the speech segment is stored as the database pitch, the singing voice having the same pitch as that of the original singing voice is synthesized with a small key scaling function. In this case, it is possible to change the tone quality such as the tone color only when trying to synthesize a singing voice having a pitch different from the pitch of the original singing voice while maintaining the tone quality such as the timbre, and natural singing synthesis is possible.
[Brief description of the drawings]
FIG. 1 is a block diagram showing the overall configuration of a singing voice synthesizing apparatus according to an embodiment of the present invention.
FIG. 2 conceptually shows the contents of data stored in the voice database shown in FIG.
FIG. 3 shows the contents of a key scaling function fn.
4 is a flowchart showing the operation of the singing voice synthesizing apparatus 1 shown in FIG.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 11 ... Speech database, 12 ... Synthesis parameter production | generation apparatus, 13 ... Key scaling function memory | storage part 14 ... Key scaling apparatus, 15 ... Waveform synthesis apparatus

Claims (6)

A voice information input unit that receives voice information indicating the content of the voice to be synthesized;
A phonetic database in which speech segment data for synthesizing speech is stored;
A selection unit for selecting the speech segment data stored in the phoneme database based on the speech information;
A synthesized pitch data output unit for outputting synthesized pitch data indicating the pitch of the voice to be synthesized in time series;
A synthesis parameter output unit that extracts and outputs a synthesis parameter from the speech unit data selected by the selection unit;
A database pitch data generating unit that extracts pitch data constituting the speech segment data used when extracting the synthesis parameter and outputs it as database pitch data;
A key scaling function storage unit for storing a key scaling function prepared for each of the synthesis parameters;
Based on the difference between the function value obtained by substituting the synthetic pitch data into the key scaling function and the function value obtained by substituting the database pitch data into the key scaling function, the synthetic parameter is corrected. A key scaling unit that outputs correction parameters;
A singing voice synthesizing apparatus comprising a waveform synthesizing unit that synthesizes a waveform based on the correction parameter. The singing voice synthesizing apparatus according to claim 1, wherein the synthetic pitch data output unit generates synthetic pitch data based on pitch information included in the audio information and outputs the generated synthetic pitch data. The singing voice synthesizing apparatus according to claim 1, wherein the key scaling unit adds a difference between the two function values to the various parameters. The singing voice synthesizing apparatus according to claim 1, wherein the key scaling unit multiplies the various parameters by a value obtained by adding 1 to a difference between the two function values. A voice information input step for inputting voice information indicating the contents of the voice to be synthesized;
A selection step of storing speech segment data for synthesizing speech in the phoneme database in advance and selecting speech segment data stored in the phoneme database based on the speech information;
A synthesized pitch data output step for outputting synthesized pitch data indicating the pitch of the voice to be synthesized in time series;
A synthesis parameter output step for extracting and outputting a synthesis parameter from the speech segment data selected in the selection step;
A database pitch data generation step of extracting pitch data constituting the speech segment data used when extracting the synthesis parameter and outputting it as database pitch data;
A key scaling function is prepared for each synthetic parameter, and a function value obtained by substituting the synthetic pitch data into the key scaling function and a function value obtained by substituting the database pitch data into the key scaling function A key scaling step for correcting the composite parameter and outputting a correction parameter based on the difference between
And a waveform synthesis step of synthesizing a waveform based on the correction parameter. A voice information input step for inputting voice information indicating the contents of the voice to be synthesized;
A selection step of storing speech segment data for synthesizing speech in the phoneme database in advance and selecting speech segment data stored in the phoneme database based on the speech information;
A synthesized pitch data output step for outputting synthesized pitch data indicating the pitch of the voice to be synthesized in time series;
A synthesis parameter output step for extracting and outputting a synthesis parameter from the speech segment data selected in the selection step;
A database pitch data generation step of extracting pitch data constituting the speech segment data used when extracting the synthesis parameter and outputting it as database pitch data;
A key scaling function is prepared for each synthetic parameter, and a function value obtained by substituting the synthetic pitch data into the key scaling function and a function value obtained by substituting the database pitch data into the key scaling function A key scaling step for correcting the composite parameter and outputting a correction parameter based on the difference between
A singing composition program configured to cause a computer to execute a waveform synthesis step of synthesizing a waveform based on the correction parameter.

Images