JP4353174B2 - Speech synthesizer - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a voice synthesizer in which database size is reduced while deterioration in tone quality is suppressed to a minimum. <P>SOLUTION: The voice synthesizer has a storing means which stores the featured values of voice at a specific time while making phoneme and pitch as indexes, a phoneme template storing means which stores a plurality of templates, which are templates representing time variation of the featured values of the pitch and voice and are obtained by analyzing the voice of the portion where the featured values are normal, and a plurality of templates, which are obtained by analyzing the voice of the connecting portion of phoneme, while making respective phoneme and pitch as indexes, an inputting means which receives voice information that is used for voice synthesis and includes a phoneme flag indicating whether the pitch and the phoneme are in the normal portion or in the transition portion of the phoneme and a voice synthesizing means which reads the featured values of the voice from the storing means while making the pitch and the phoneme included in the inputted voice information as indexes and synthesizes voice based on the featured values and the pitch of the voice. <P>COPYRIGHT: (C)2006,JPO&amp;NCIPI

Description

  The present invention relates to a speech synthesizer, and more particularly to a speech synthesizer that synthesizes a human singing voice.

  Human speech is composed of phonemes (phonemes), and each phoneme is composed of a plurality of formants. Therefore, for the synthesis of human singing voice, first, prepare all the phonemes that can be generated by humans to generate and synthesize all the formants that constitute each phoneme, and then add the necessary phonemes. Generate. Next, the plurality of generated phonemes are sequentially connected, and the pitch is controlled according to the melody. This technique is applicable not only to human speech but also to synthesis of musical sounds having formants, for example, musical sounds generated from wind instruments.

  A speech synthesizer using this method has been conventionally known. For example, in Japanese Patent Publication No. 2504172, a formant sound configured so as not to generate an unnecessary spectrum even when a formant sound having a high pitch is generated. A generator is disclosed.

  Further, it is known that the formant frequency depends on the pitch. As described in the example of Japanese Patent Laid-Open No. 6-308997, several pieces of phoneme are stored in the database for each pitch frequency. A technique for selecting an appropriate phoneme piece according to the pitch of speech is known.

  However, in the conventional database as described above, it is necessary to have phonemes having a certain number of pitch frequencies or more for each phoneme, and the size of the database becomes relatively large.

  Further, since it is necessary to extract phonemes from speech generated at many different pitches, it takes time to construct a database.

  Furthermore, the formant frequency does not depend only on the pitch, but the amount of data increases to the square, the cube, etc. due to the addition of other elements such as dynamics.

  An object of the present invention is to provide a speech synthesizer that reduces the size of a database while minimizing degradation of sound quality.

  Another object of the present invention is to provide a speech synthesizer using the database.

According to one aspect of the present invention, a storage unit that stores a feature amount of speech at a specific time including at least one of excitation resonance or formant as a phoneme and pitch as an index, and represents a temporal change in the feature amount of the pitch and speech. Template storage means for storing a template as a phoneme and a pitch as an index, input means for inputting speech information for speech synthesis including at least the pitch and phoneme, and the feature amount and template of the speech according to the input speech information Read means for reading from the storage means and the template storage means respectively, and applying the read template to the pitch included in the feature value of the read sound and the input sound information. Speech synthesis means for synthesizing speech based on the feature and pitch of speech In the speech synthesizer, when the pitch included in the input speech information exceeds the value of the highest index in the storage unit, the reading unit is configured to transfer the pitch from the pitch included in the input speech information to the storage unit. A feature in which a pitch difference obtained by subtracting a pitch of the highest index to be stored is obtained, and the pitch difference is added to the frequency of the excitation resonance or the formant included in the feature amount read from the storage unit by the highest pitch index. The quantity is output to the speech synthesis means.
In addition, according to another aspect of the present invention, storage means for storing a feature amount of speech at a specific time including at least one of excitation resonance or formant as an index of phoneme and pitch, and time of feature amount of pitch and speech Template storage means for storing a template representing a change using phonemes and pitches as indexes, input means for inputting speech information for speech synthesis including at least the pitch and phonemes, and the feature values and templates of the speech are input Read means for reading from the storage means and the template storage means respectively by voice information, applying the read template to the pitch feature included in the read voice feature and the input voice information, Speech synthesis means for synthesizing speech based on feature and pitch of speech after application When the pitch included in the input speech information is less than the lowest index value in the storage unit, the read-out device has the storage from the pitch included in the input speech information. A pitch difference obtained by subtracting the pitch of the lowest index stored in the means is obtained, and the pitch difference is designated as the frequency of the excitation resonance or formant included in the feature amount read from the storage means by the lowest pitch index. A feature amount obtained by adding the ratio is output to the speech synthesizer.

  According to the present invention, it is possible to provide a speech synthesis database with a reduced size while minimizing deterioration in sound quality.

  Further, according to the present invention, it is possible to provide a speech synthesizer capable of synthesizing a more realistic human singing voice and singing a song in a natural state with no sense of incongruity.

  FIG. 1 is a block diagram showing the configuration of the speech synthesizer 1.

  The speech synthesizer 1 includes a data input unit 2, a feature parameter generation unit 3, a database 4, and an EpR speech synthesis engine 5.

  Input data Score input to the data input unit 2 is sent to the feature parameter generation unit 3 and the EpR speech synthesis engine 5. The feature parameter generation unit 3 reads feature parameters and various templates, which will be described later, from the database 4 based on the input data Score. The feature parameter generation unit 3 further applies various templates to the read feature parameters, generates final feature parameters, and sends them to the EpR speech synthesis engine 5.

  The EpR speech synthesis engine 5 generates a pulse based on the pitch, dynamics, and the like of the input data Score, and synthesizes and outputs speech by applying a feature parameter to the generated pulse.

  FIG. 2 is a conceptual diagram illustrating an example of the input data Score. It is composed of a phonological track PHT, a note track NT, a pitch track PIT, a dynamics track DYT, and an opening track OT, and is music data in which data that changes with time of a phrase of the music or the entire music is stored.

  The phoneme track PHT includes a phoneme name and its duration of pronunciation. Further, each phoneme is classified into two types: articulation (Articulation) indicating a transition part between phonemes and phoneme and stationery (Stationary) indicating other stationary parts. Each phoneme includes a flag indicating which of these phonemes is classified. Since articulation is a transition part, it has a plurality of phoneme names including a head phoneme name and a subsequent phoneme name. On the other hand, since stationery is a stationary part, it consists of only one phoneme name.

  In the note track NT, a flag indicating any of a note attack, a note-to-note, and a note release is recorded. A note attack is a command for instructing music expression at the time of pronunciation, note-to-note at a change in pitch, and note release at the time of sound fall.

  The pitch track PIT records the fundamental frequency of the sound to be generated at each time. Note that the pitch of the sound that is actually sounded is calculated using other information based on the pitch information recorded in the pitch track PIT, so that the pitch of the sound that is actually sounded is recorded here. The pitch may be different.

  In the dynamics track DYT, a dynamics value at each time, which is a parameter indicating the strength of sound, is recorded. The dynamics value takes a value from 0 to 1.

  In the opening track OT, the opening value at each time, which is a parameter indicating the degree of lip opening (lip opening), is recorded. The opening value takes a value from 0 to 1.

  The feature parameter generation unit 3 reads data from the database 4 based on the input data Score input from the data input unit 2, and generates feature parameters based on the input data Score and the data read from the database 4 as will be described later. To the EpR speech synthesis engine 5.

  The characteristic parameters generated by the characteristic parameter generation unit 3 can be classified into, for example, an envelope of an excitation waveform spectrum, an excitation resonance, a formant, and a difference spectrum. These four characteristic parameters are obtained by decomposing the spectral envelope (original spectrum) of the harmonic component obtained by analyzing actual human speech or the like (original speech).

励起レゾナンスは、胸部による共鳴を表す。中心周波数（ＥＲＦｒｅｑ）、バンド幅（ＥＲＢＷ）、アンプリチュード（ＥＲＡｍｐ）の３つのパラメータで構成され、２次フィルター特性を有している。 The envelope of the excitation waveform spectrum (Excitation Curve) is EGain indicating the magnitude (dB) of the vocal cord waveform, ESlope Depth indicating the slope of the spectrum envelope of the vocal cord waveform, and the minimum to maximum value of the spectrum envelope of the vocal cord waveform. It is comprised by three parameters of ESlope showing the depth (dB) of this, It can represent with the following formula | equation (A).

Excited resonance represents resonance by the chest. It consists of three parameters: center frequency (ERFreq), bandwidth (ERBW), and amplitude (ERAmp), and has secondary filter characteristics.

Formants represent resonances due to the vocal tract by combining 1 to 12 resonances. It consists of three parameters: center frequency (FormantFreq _i ), bandwidth (FormantBW _i ), and amplitude (FormantAmp _i ). “I” is a value from 1 to 12 (1 ≦ i ≦ 12).

  The difference spectrum is a characteristic parameter having a spectrum that is different from the original spectrum that cannot be expressed by the envelope, excitation resonance, and formant of the excitation waveform spectrum.

  The database 4 includes at least a Timbre database TDB, a phoneme template database PDB, and a note template database NDB.

  In general, when speech is synthesized using only feature parameters obtained from a specific time stored in the Timbre database TDB, the speech becomes very monotonous and mechanical. In addition, when phonemes are continuous, the sound at the transition part actually changes gradually, so when only the steady parts of phonemes are simply connected, the speech at the connection point is very unnatural. It becomes. Therefore, the phoneme template and the note template are stored as a database and used at the time of speech synthesis, so that those drawbacks can be reduced.

  Timbre is a timbre tone color and is expressed by a characteristic parameter (set of excitation spectrum, excitation resonance, formant, difference spectrum) at one point in time. FIG. 3 shows an example of the Timbre database TDB. This database has phoneme names and pitches as indexes.

  In the following description, the Timbre database TDB shown in FIG. 3 is used in this specification. However, as shown in FIG. 4, the phoneme name, the pitch, the dynamics, and the opening are provided as indexes so that the feature parameters can be specified more finely. A database may be prepared.

なお、ｔ＝０、Δｔ、２Δｔ、３Δｔ、…、Ｔであり、本実施例では、Δｔは５ｍｓとする。 The phoneme template database PDB includes a stationery template database and an articulation template database. Here, the template is a set of a sequence in which pairs of feature parameters P and pitch pitch are arranged at regular intervals and a length T (sec.) Of the section, and can be expressed by the following formula (B). I can do it.

Note that t = 0, Δt, 2Δt, 3Δt,..., T, and in this embodiment, Δt is 5 ms.

  If Δt is reduced, the time resolution is improved and the sound quality is improved, but the database size is increased. Conversely, if Δt is increased, the sound quality is deteriorated but the database size is reduced. What is necessary is just to determine in consideration of the priority of sound quality and the size of a database, when determining (DELTA) t.

  FIG. 5 is an example of a stationery template database. The stationery template database has stationery templates for all voiced phonemes, using phoneme names and representative pitches as indexes. A stationery template can be obtained by analyzing speech of a stable phoneme and pitch using an EpR model.

  When a certain voiced sound, for example “A”, is extended for a long time and uttered at a certain pitch, for example C4, the characteristic parameters such as pitch and formant frequency are almost constant and can be said to be stationary. In practice, however, there are some fluctuations. If there is no change and the sound is completely constant, the sound becomes inorganic and mechanical. Conversely, it can be said that the change represents humanity and naturalness.

  When synthesizing voiced sound, instead of using only Timbre, that is, a feature parameter at one point in time, the time variation and pitch variation of the feature parameter extracted from the actual human voice in the stationery template are added to it. By doing so, the voiced sound can be given naturalness.

  In the case of singing voice synthesis, it is necessary to change the time of sound generation according to the length of a note, but only one sufficiently long template is prepared. When synthesizing a voiced sound longer than the template, the template is applied from the beginning of the voiced sound without changing the time axis of the template.

  When the end of the template is reached, the same template is then applied again. It is also possible to apply a template in which the template time is reversed when the end of the template is reached. This method eliminates the discontinuity at the connection point of the template.

  The reason why the time axis of the template is not expanded / contracted is that the naturalness is lost if the speed of variation of the characteristic parameter and the pitch is greatly changed. It is preferable not to expand and contract from the viewpoint that the fluctuation of the steady portion is not something that humans consciously control.

  The stationery template does not have the time series of the characteristic parameters of the stationary part as they are, but the variation of the characteristic parameters of the stationary part, which is a structure having the typical characteristic parameters of the phoneme and the fluctuation amount, is small. Compared with having the characteristic parameters as they are, the amount of variation is smaller and the amount of information is smaller, which has the effect of reducing the database size.

  FIG. 6 is an example of an articulation template database. The articulation template database uses the first phoneme name, the subsequent phoneme name, and the representative pitch as indexes. The articulation template database stores articulation templates for practically possible phoneme combinations in a certain language.

  The articulation template can be obtained by analyzing the speech of the connected part of the phoneme having a stable pitch using the EpR model.

人間が２つの音素を連続して発音する場合には、突然変化するのではなくゆるやかに移行していくので、例えば、「あ」という母音の後に区切りを置かないで連続して「え」という母音を発音する場合には、最初に「あ」が発音され「あ」と「え」の中間に位置する発音を経て「え」に変化する。 The feature parameter P (t) may be an absolute value as it is, but a difference value may be used. As will be described later, at the time of synthesis, the absolute values of these template values are not used as they are, but the relative change amounts of the parameters are used. Therefore, according to the template application method, the following formula (C1) Characterized in the form of a difference from P (t = T), a difference from P (0), or a difference between P (0) and P (T) connected by a straight line as shown in FIG. Record the parameters.

When a person pronounces two phonemes in succession, it changes slowly, not suddenly. For example, it is called “e” continuously without a break after the vowel “a”. When a vowel is pronounced, “a” is first pronounced, and changes to “e” through a pronunciation located between “a” and “e”.

  This phenomenon is a phenomenon generally called articulation coupling. In order to perform speech synthesis so that the phoneme combination part becomes natural, it is preferable to have some form of speech information of the combination part for the combination of phonemes that can be combined in a certain language.

  There is already a method that has a phoneme connection part in the form of an LPC coefficient or a speech waveform, but in this embodiment, an articulation template with feature parameter and pitch difference information is used. The Articulation part of is synthesized.

  For example, consider a case where two consecutive quarter notes of the same pitch are used to synthesize a song with the lyrics “A” and “I”. There is a transition from “A” to “I” at the boundary between two notes. Since “a” and “i” are both vowels and voiced sounds, they correspond to articulation from V (voiced sound) to V (voiced sound), and are articulated by the type 3 method described later. The template can be applied to determine the feature parameters of the transition part.

  That is, if the characteristic parameters of “A” and “I” are read from the Timbre database TDB and an articulation template from “A” to “I” is applied to them, the transition part has a natural change. A parameter is obtained.

  Here, if the time of the transition from “A” to “I” is the same as the original time of the articulation template applied to that part, the same change as the audio waveform used when creating the template Can be obtained.

  When synthesizing a voice that changes more slowly or longer than the template time, the feature parameter difference may be added after linearly extending the template length. Unlike the stationery described above, the speed of the changing part of the two phoneme questions can be consciously controlled, so that even if the template is expanded or contracted linearly, no great unnaturalness will occur.

  Next, consider a case where two consecutive quarter notes of the same pitch are used to synthesize a song with the lyrics “A” and “SU”. There is a short transition from the “A” to the “SU” consonant at the boundary between the two notes. Since this corresponds to articulation from V (voiced sound) to U (unvoiced sound), the feature parameter of the transition part can be obtained by applying the articulation template by the type 1 method described later.

  By obtaining the characteristic parameter of “A” from the Timbre database TDB and applying the articulation template from “a” to “s” to it, the characteristic parameter of the transition part having a natural change can be obtained.

  The reason for using Type 1, that is, the difference from the beginning of the template, in the articulation from V (voiced sound) to U (unvoiced sound) is simply the pitch and feature parameters in the U (unvoiced sound) part, which is the end part. This is because there is no.

  “Su” is “su” in Roman letters, and is composed of a consonant part “s” and a vowel part “u”. There is also a transition portion where “u” is pronounced while leaving the sound of “s” at this intermediate point. Since this corresponds to articulation from U to V, the articulation template is again applied in the type 1 method.

  The characteristic parameter of “u (u)” is read from the Timbre database TDB, and the articulation template from “s” to “u” is applied to the characteristic parameter, so that the characteristic parameter of the changing part from “s” to “u” Can be obtained.

  An articulation template having feature parameter difference information has the advantage of a smaller data size than a template in which feature parameters are recorded as absolute values.

  The note template database NDB includes at least a note attack template (NA template) database NADB, a note release template (NR template) database NRDB, and a note-to-note template (NN template) database NNDB.

  FIG. 7 shows an example of the NA template database NADB. The NA template includes feature parameters of the rising part of the voice and pitch change information.

  The NA template database NADB stores NA templates for all voiced phonemes using the phoneme names and representative pitches as indexes. The NA template is obtained by analyzing the rising part of the sound that is actually pronounced.

  The NR template includes feature parameters of the falling edge of the voice and pitch change information. The NR template database NRDB has the same structure as the NA template database NADB, and has NR templates for all voiced phonemes with the phoneme name and the representative pitch as indexes.

  Analyzing the rising portion (Attach) when a phoneme having a constant pitch, for example, “A”, is analyzed, it can be seen that the amplitude gradually increases and becomes stable at a constant level. Not only the amplitude value but also the formant frequency, formant bandwidth, and pitch change.

  Applying the NA template obtained by analyzing the rising part of human speech, for example “A”, to the characteristic parameter of the steady part, the natural change of the voice of the rising part Can be given.

  If an NA template is prepared for every phoneme, it becomes possible to change the attack part for any phoneme.

  In singing, there is a case where the rising speed is fastened or a song is sung in order to make a musical expression. The NA template has a certain rise time, but it can be made faster or slower than the NA template by applying it after linearly expanding and contracting the template time axis. become.

  Experiments show that even if the template is expanded or contracted, the attack does not cause unnaturalness within a range of several times. In order to be able to synthesize by specifying a wider range of attack lengths, a method such as preparing NA templates of several stages in length, selecting a template having the closest length, and expanding and contracting is used.

  As for the portion where the utterance ends, that is, the fall (Release), the amplitude, pitch, and formant change in the same manner as the rise (Attack).

  The natural change of the human voice is given to the falling part because the NR template obtained by analyzing the falling part of the voice actually uttered by the human is used as the characteristic parameter of the phoneme before the falling starts. It becomes possible by applying to it.

  FIG. 8 is an example of the NN template database NNDB. The NN template has a voice feature parameter of a portion where the pitch changes. The NN template database NNDB stores NN templates for all voiced phonemes using the phoneme name, the pitch of the start time of the template, and the pitch of the end time as indexes.

  There is a singing method in which, when two notes having different pitches are sung continuously without any gap, the pitch is smoothly changed from the pitch of the previous note to the pitch of the subsequent note. Naturally, the pitch and amplitude change, but even if the two preceding and following notes have the same pronunciation (for example, the same “A”), the frequency characteristics of the sound such as the formant frequency slightly change.

  By using the NN template that is obtained by analyzing the change of the voice sung by actually changing the pitch from the start point to the end point, it is possible to give a natural musical expression to the boundary of such notes with different pitches. it can.

  In actual melody, there are many combinations of pitch changes even if the range of 24 octaves is 2 octaves. However, in practice, even if the absolute value of the pitch is different, a template with a close pitch difference can be substituted, so it is not necessary to prepare an NN template for all combinations.

  In the selection of the NN template, as will be described later, a template having a close pitch change width is preferentially selected rather than a template having a close pitch absolute value. The selected NN template is applied by a type 3 method described later.

  At this time, the NN template having a close pitch change width is preferentially selected because there is a possibility that a large value is included in the NN template created from a portion where the pitch varies greatly. This is because when applied to a small number of parts, the shape of the change of the original NN template cannot be maintained, and the change may become unnatural.

  Note that it is possible to use a specific phoneme, for example, an NN template obtained from a voice whose pitch of “A” is changing, instead of using a pitch change of all phonemes. In an environment where there is no problem, it is possible to produce a rich synthetic speech that is less monotonous by preparing an NN template by changing the pitch of several patterns for each phoneme.

  Next, a method for applying a template recorded in the database 4 will be described. The application of the template means that the time length of the template is expanded or contracted with respect to a certain section on the input data Score, the difference between the template feature parameters is added to one or more feature parameters serving as reference points, and the score is applied. To obtain a sequence of feature parameters and pitches having the same time length as a certain interval. Specifically, there are four types of template application methods from type 1 to type 4. In the following description, the template is represented by {P (t), Pitch (t), T}.

なお、時刻ｔ＝０にテンプレート及び区間Ｋの始点があるとする。この式（Ｄ）はテンプレートの始点からの変化分を時刻ｔの特徴パラメータに加算することを意味する。 First, application of a template according to type 1 will be described. Type 1 is a template application method based on the start point designation type. The application of the template of type 1 to the section K of the length T ′ of the input data Score is to obtain the feature parameter P ′ _t at time t according to the following equation (D). Note that P _t is a feature parameter at time _t in section K.

It is assumed that there is a template and the start point of section K at time t = 0. This equation (D) means that the change from the starting point of the template is added to the feature parameter at time t.

  Type 1 is used when the template is mainly applied to the feature parameter of the note release part. This is because, at the beginning of the note release, there is a steady part of the voice, so it is necessary to maintain the continuity of the parameters at the beginning of the note release, that is, the continuity of the voice, and the end of the note release is silent. Because there is no need.

なお、時刻ｔ＝０にテンプレート及び区間Ｋの始点があるとする。この式（Ｅ）はテンプレートの終点からの変化分を時刻ｔの特徴パラメータに加算することを意味する。 Next, a template application method according to type 2 will be described. Type 2 is a template application method based on the end point designation type. The application of the template according to type 2 to the section K of the length T ′ of the input data Score is to obtain the feature parameter P ′ _t at time t according to the following equation (E). Note that P _t is a feature parameter at time _t in section K.

It is assumed that there is a template and the start point of section K at time t = 0. This equation (E) means that the change from the end point of the template is added to the feature parameter at time t.

  Type 2 is used when the template is mainly applied to the feature parameter of the note attack portion. This is because, in the rear part of the note attack, there is a steady part of the voice, so it is necessary to maintain the continuity of the parameters in the rear part of the note attack, that is, the continuity of the voice, and the start part of the note attack is silent. Because there is no need.

なお、時刻ｔ＝０にテンプレート及び区間Ｋの始点があるとする。この式（Ｆ）はテンプレートの始点と終点を結んだ直線との差を、区間Ｋの始点と終点を結んだ直線に加算することを意味する。 Next, a template application method according to type 3 will be described. Type 3 is a template application method based on the double point designation type. The application of the template according to type 3 to the section K of the length T ′ of the input data Score is to obtain the feature parameter P ′ _t at time t according to the following equation (F). Note that P _t is a feature parameter at time _t in section K.

It is assumed that there is a template and the start point of section K at time t = 0. This formula (F) means that the difference between the straight line connecting the start point and the end point of the template is added to the straight line connecting the start point and the end point of the section K.

なお、時刻ｔ＝０にテンプレート及び区間Ｋの始点があるとする。この式（Ｇ）は区間Ｋに対してテンプレートの始点からの特徴パラメータの変化分を加算することをＴ毎に繰り返すことを意味する。 Next, a template application method according to type 4 will be described. Type 4 is a method of applying a template by stationery type. The application of the template according to type 2 to the section K of the length T ′ of the input data Score is to obtain the feature parameter P ′ _t at time t according to the following equation (G). Note that P _t is a feature parameter at time _t in section K.

It is assumed that there is a template and the start point of section K at time t = 0. This equation (G) means that the addition of the change in the characteristic parameter from the starting point of the template to the section K is repeated every T.

  Type 4 is mainly used when applied to the stationary part. This type 4 has an effect of giving natural fluctuation to a stationary part of a relatively long speech.

  FIG. 9 is a flowchart showing the feature parameter generation process. With this process, a feature parameter at a certain time t is generated. By repeating this characteristic parameter generation process while increasing the time t at every certain time, it is possible to synthesize unit sounds such as phrases and songs.

  In step SA1, feature parameter generation processing is started, and the process proceeds to next step SA2.

  In step SA2, the value of each track at time t of the input data Score is acquired. Specifically, the phoneme name, articulation or stationery distinction, note attack, note-to-note distinction, note release distinction, pitch, dynamics value, and opening value at time t in the input data Score are acquired. Thereafter, the process proceeds to next Step SA3.

  In step SA3, necessary templates are read from the phoneme template database PDB and the note template database NDB based on the value of each track of the input data Score acquired in step SA2. Thereafter, the process proceeds to next Step SA4.

  The phoneme template is read in step SA3 by the following procedure, for example. If it is determined that the phoneme at time t is articulation, the articulation template database is searched to read a template whose head and subsequent phoneme names match and whose pitch is closest.

  On the other hand, if it is determined that the phoneme at time t is stationery, the stationery template database is searched, and the stationery template with the same phoneme name and the closest pitch is read.

  The note template is read as follows. For example, if it is determined that the note track at time t is a note attack, the NA template database NADB is searched, and the template with the same phoneme name and the closest pitch is read.

  Also, for example, if it is determined that the note track at time t is a note release, the NR template database NRDB is searched to read a template with the same phoneme name and the closest pitch.

ここで、

上記式（Ｈ）で求めた距離ｄに基づき、テンプレートを読み込むことにより、ピッチの絶対値が近いものよりも、ピッチの変化幅が近いテンプレートを優先的に選択するようにしている。 Further, for example, if it is determined that the note track at time t is note-to-note, the NN template database NNDB is searched, the phoneme names match, and the following formula is used based on the start point pitch and end time pitch: A template having the closest distance d obtained in (H) is read. The following formula (H) is a distance scale based on a value obtained by weighting and adding a frequency change amount and an average value.

here,

By reading a template based on the distance d obtained by the above formula (H), a template having a close pitch change width is preferentially selected over a pitch having a close absolute value.

  In step SA4, the start time and end time of an area having the same attribute as the current time t of the note track are obtained. If the phonological track is stationary, the start time or the note release is determined according to the distinction between note attack, note to note or note release. The end time or both characteristic parameters are acquired or calculated. Thereafter, the process proceeds to next Step SA5.

  If the note track at time t is a note attack, the Timbre database TDB is searched to read a feature parameter that matches the phoneme name and the pitch of the note attack end time.

  When there is no feature parameter with the same pitch, two feature parameters with the same phoneme name and sandwiching the pitch of the note attack end time are obtained, and the feature parameter of the note attack end time is calculated by interpolating these. . Details of the interpolation method will be described later.

  When the note track at time t is a note release, the Timbre database TDB is searched to read a feature parameter that matches the phoneme name and the pitch of the note attack start time.

  When there is no feature parameter having the same pitch, two feature parameters having the same phoneme name and sandwiching the pitch of the note release start time are acquired, and the feature parameter of the note release start time is calculated by interpolating these. . Details of the interpolation method will be described later.

  When the note track at time t is note-to-note, the Timbre database TDB is searched, and the feature parameter and the phoneme name that match the pitch of the phoneme name and the note-to-note start time match the note-to-note end time. Read feature parameters.

  When there is no feature parameter with the same pitch, two feature parameters with the same phoneme name and sandwiching the pitch of the note-to-note start (end) time are obtained, and note-to-note start (end) is interpolated between them. ) Calculate the time feature parameter. Details of the interpolation method will be described later.

  If the phoneme track is articulation, the feature parameters of the start time and end time are acquired or calculated. In this case, the Timbre database TDB is searched to read the feature parameters having the same phoneme name and the pitch of the articulation start time, and the feature parameters having the same phoneme name and the pitch of the articulation end time.

  When there is no feature parameter with the same pitch, two feature parameters that match the phoneme name and sandwich the pitch at the start (end) time of the articulation are obtained, and articulation start (end) is performed by interpolating these ) Calculate the time feature parameter.

  In step SA5, the template read in step SA3 is applied to the feature parameters and pitch of the start point and end time obtained in step SA4, and the pitch and dynamics at time t are obtained.

  If the note track at time t is a note attack, the NA template is applied for type 2 using the feature parameter of the end time of the note attack portion obtained in step SA4 for the note attack portion. The pitch and dynamics (EGain) at time t after applying the template are stored.

  On the other hand, if the note track at time t is a note release, the type 1 NR template is applied to the note release portion using the feature parameter of the note release start point obtained in step SA4. The pitch and dynamics (EGain) at time t after applying the template are stored.

  If the note track at time t is a note-to-note, the feature parameters at the start and end times of the note-to-note obtained in step SA4 are used for the note-to-note part, and the type 3 NN template is used for that interval. Apply. The pitch and dynamics (EGain) at time t after applying the template are stored.

  Further, when the note track at time t is neither of the above, the pitch and dynamics (EGain) of the input data Score are stored.

  When any of the above processes is performed, the process proceeds to the next step SA6.

  In step SA6, it is determined from the value of each track obtained in step SA2 whether or not the phoneme at time t is articulation. If it is articulation, the process proceeds to step SA9 indicated by a YES arrow. If it is not articulation, that is, if the phoneme at time t is stationary, the process proceeds to step SA7 indicated by a NO arrow.

  In step SA7, feature parameters are read from the Timbre database TDB and interpolated using the phoneme name at time t obtained in step SA2 and the pitch and dynamics obtained in step SA5 as indexes. The reading and interpolation methods are the same as those performed in step SA4. Thereafter, the process proceeds to Step SA8.

  In step SA8, the stationary template obtained in step SA3 is applied as type 4 to the feature parameter and pitch at time t obtained in step SA7.

  In step SA8, by applying the stationery template, the feature parameter and the pitch at time t are updated, and the sound fluctuation of the stationery template is added. Thereafter, the process proceeds to step SA10.

  In step SA9, the feature parameter and pitch at time t are obtained by applying the articulation template read in step SA3 to the feature parameter of the start time and end time of the articulation part obtained in step SA4. Thereafter, the process proceeds to step SA10.

  However, the template is applied using type 1 for a change from voiced sound (V) to unvoiced sound (U), and for a change from unvoiced sound (U) to voiced sound (V), type 2 is used. In the case of a change from a voiced sound (V) to a voiced sound (V) or from an unvoiced sound (U) to an unvoiced sound (U), the change is made with type 3.

  The reason for changing the template application method as described above is to reproduce the natural change in the voice included in the template while maintaining continuity in the voiced portion.

  In step SA10, any one of the NA template, NR template, and NN template is applied to the feature parameter obtained in step SA8 or step SA9. However, here, the template is not applied to the characteristic parameter EGain. Thereafter, the process proceeds to the next step SA11, and the feature parameter generation process is terminated.

  In the application of the template in step SA10, when the note track at time t is a note attack, the NA template obtained in step SA3 is applied by type 2 to update the feature parameter.

  When the note track at time t is a note release, the feature parameter is updated by applying the NR template obtained in step SA3 by type 1.

  If the note track at time t is note-to-note, the NN template obtained in step SA3 is applied by type 3 to update the feature parameter.

  In any of the above cases, however, no template is applied to the characteristic parameter EGain. As for the pitch, the pitch obtained in the step before step SA10 is used as it is.

  Hereinafter, the feature parameter interpolation performed in step SA4 of FIG. 9 will be described. The feature parameter interpolation includes interpolation of two feature parameters and estimation from one feature parameter.

  It is known that the vocal cord waveform (sound source waveform generated by the air from the lungs and the vocal cord vibration) changes when the pitch is changed when a human utters voice, and the formant also changes with the pitch. It has been. If the feature parameters obtained from the voice sung at a certain pitch are used as they are when synthesizing the voice at other pitches, even if the pitch is changed, a similar voice tone will be produced, which is unnatural. turn into.

  In order to avoid this, pitches of about three points are selected at approximately equal intervals on the logarithmic axis in the range of 2 to 3 octaves which is a human singing range, and feature parameters are stored in the Timbre database TDB. When synthesizing speech with a pitch other than the pitch in the Timbre database TDB, the feature parameters are obtained by interpolation of two feature parameters (linear interpolation) or estimation from one feature parameter (extrapolation).

  With this method, it is possible to simulate a change in the feature parameter of the voice when the pitch changes. Also, having about 3 feature parameters with different pitches is that even if the same phoneme and the same pitch are generated, the feature parameters vary depending on the situation. This is because the difference from the obtained case is not so meaningful.

上記式（Ｉ）では、データベースのインデックスがピッチ1個だけの場合を考えたが、一般的にインデックスがＮ個ある場合でも、目標を囲む近傍のＮ＋1個のデータをもとに、以下の式（Ｉ’）を用いて、目標のインデックスｆの代理として使用する特徴パラメータを補間して求めることが出来る。なお、Ｐ_ｉは、近傍のｉ番目の特徴パラメータであり、ｆ_ｉはそのインデックスである。

１つの特徴パラメータからの推定は、データベースに含まれるデータの音域を外れる音声の特徴パラメータを推定するときに用いる。 The interpolation of the two feature parameters is, for example, when a pair {P1, f1 [cents]} and {P2, f2 [cents]} of the two feature parameters and the respective pitches are given, the pitch f1 [ The feature parameter in [cents] is obtained by linear interpolation using the following equation (I).

In the above formula (I), the case where the index of the database has only one pitch was considered. However, even when there are generally N indexes, the following formula is used based on N + 1 pieces of data in the vicinity surrounding the target. Using (I ′), it is possible to interpolate and obtain a characteristic parameter used as a proxy for the target index f. Note that P _i is the i-th feature parameter in the vicinity, and f _i is its index.

The estimation from one feature parameter is used when estimating the feature parameter of speech that is out of the range of the data included in the database.

This is because, when synthesizing speech having a pitch higher than the range of the database, if the feature parameter with the highest pitch in the database is used as it is, the sound quality is clearly degraded.

  In addition, when synthesizing a voice having a pitch lower than the range of the database, the sound quality is similarly deteriorated when the feature parameter having the lowest pitch is used. Therefore, in this embodiment, using the rules based on the knowledge based on observation of actual audio data, the characteristic parameters are changed as follows to prevent the deterioration.

  First, the case where a voice having a pitch (target pitch) higher than the range of the database is synthesized will be described.

  First, a value PitchDiff [cents] is obtained by subtracting the highest pitch HighPitch [cents] in the database from the target pitch TargetPitch [cents].

Next, the feature parameter having the highest pitch is read from the database, and the PitchDiff [cents] is added to the excitation resonance frequency EpRFreq and the i-th formant frequency FormatFreq _i , respectively, to EpRFreq ′ and FormatFreq _i ′. The replacement is used as a target pitch feature parameter.

  Next, a description will be given of the case of synthesizing speech having a pitch (target pitch) lower than the range of the database.

  First, a value PitchDiff [cents] is obtained by subtracting the lowest pitch LowestPitch [cents] in the database from the target pitch TargetPitch [cents].

  Next, the feature parameter having the lowest pitch is read from the database, and the parameter is replaced as follows and used as the feature parameter of the target pitch.

…（Ｊ１）

…（Ｊ２）
さらに、ピッチが低くなるほどバンド幅が狭くなるように、励起レゾナンスバンド幅ＥＲＢＷ及び第１から第３フォルマントのバンド幅ＦｏｒｍａｎｔＢＷ_i（１≦ｉ≦３）をそれぞれ下記式（Ｊ３）のＥＲＢＷ’、ＦｏｒｍａｎｔＢＷ_i’に置き換える。 First, the excitation resonance frequency EpRFreq and the first to fourth formant frequencies FormatFreq (1 ≦ i ≦ 4) are replaced with EpRFreq ′ and FormatFreq _i ′ using the following formulas (J1) and (J2), respectively.

... (J1)

... (J2)
Further, the excitation resonance band width ERBW and the first to third formant bandwidths FormBW _i (1 ≦ i ≦ 3) are respectively expressed by ERBW ′ and FormatBW in the following formula (J3) so that the band width becomes narrower as the pitch becomes lower. _{Replace with i} '.

…（Ｊ３）
さらに、第１から第４フォルマントのアンプリチュードＦｏｒｍａｎｔＡｍｐ１〜ＦｏｒｍａｎｔＡｍｐ４を下記式（Ｊ５）〜（Ｊ８）に従いＰｉｔｃｈＤｉｆｆに比例させて大きくして、ＦｏｒｍａｎｔＡｍｐ１’〜ＦｏｒｍａｎｔＡｍｐ４’に置き換える。

…（Ｊ５）

…（Ｊ６）

…（Ｊ７）

…（Ｊ８）
さらに、スペクトル・エンベロープの傾きＥｓｌｏｐｅを下記式（Ｊ９）に従いＥｓｌｏｐｅ’に置き換える。

…（Ｊ９）
図４に示すような、ピッチ、ダイナミクス、オープニングをインデックスとしてＴｉｍｂｒｅデータベースＴＤＢを作成することが好ましいが、時間的、データベースサイズ的な制約がある場合には、本実施例のように、図３に示すような、ピッチのみをインデックスとしたデータベースを用いることになる。

... (J3)
Further, the first to fourth formant amplitudes FormAmp1 to FormatAmp4 are increased in proportion to PitchDift according to the following formulas (J5) to (J8), and replaced with FormatAmp1 ′ to FormatAmp4 ′.

... (J5)

... (J6)

... (J7)

... (J8)
Further, the slope Elope of the spectrum envelope is replaced with Elope 'according to the following formula (J9).

... (J9)
As shown in FIG. 4, it is preferable to create a Timbre database TDB with pitch, dynamics, and opening as indexes. However, when there are temporal and database size restrictions, as shown in FIG. As shown, a database using only the pitch as an index is used.

  In such a case, using a dynamics function or an opening function, the feature parameter with only the pitch as an index is changed, as if using a Timbre database TDB created using the pitch, dynamics, and opening as an index. The effect can be obtained in a pseudo manner.

  That is, it is possible to obtain an effect as if the voice recorded by changing the pitch, dynamics, and opening is used by using the voice recorded by changing only the pitch.

  The dynamics function and the opening function can be obtained by analyzing the correlation between the actual speech uttered by changing the dynamics and the opening, and the feature parameter. In the following, examples of dynamics functions and opening functions will be given and their application methods will be described.

  FIG. 10 is a graph showing an example of a dynamics function. 10A is a graph representing the function fEG, FIG. 10B is a graph representing the function fES, and FIG. 10C is a graph representing the function fESD.

  Using these functions fEG, fES, and fESD shown in FIGS. 10A to 10C, the dynamics values are reflected in the characteristic parameters ExcitationGain (EG), ExcitationSlope (ES), and ExcitationSlopeDepth (ESD).

なお、図１０（Ａ）〜（Ｃ）の関数ｆＥＧ、ｆＥＳ、ｆＥＳＤは、一例であり、歌唱者によって様々な関数を用意することにより、より自然性を持った音声合成を行うことが出来る。 Inputs of the functions fEG, fES, and fESD in FIGS. 10A to 10C are all dynamic values and take values from 0 to 1. Using the functions fEG, fES, and fESD with the dynamics value as dyn, the characteristic parameters EG ′, ES ′, and ESD ′ are obtained by the following formulas (K1) to (K3), and the characteristic parameters at the time of the dynamics value (dyn) are obtained. Used as

Note that the functions fEG, fES, and fESD shown in FIGS. 10A to 10C are examples, and voice synthesis with more naturalness can be performed by preparing various functions by a singer.

  FIG. 11 is a graph showing an example of the opening function. In the figure, the horizontal axis represents frequency (Hz) and the vertical axis represents amplitude (dB).

また、以下の式（Ｌ２）により、ｉ番目のフォルマント周波数ＦｏｒｍａｎｔＦｒｅｑ_ｉ’をｉ番目のフォルマント周波数ＦｏｒｍａｎｔＦｒｅｑ_ｉから求め、オープニング値（Ｏｐｅｎ）のときの特徴パラメータとして用いる。

これにより、周波数０〜５００Ｈｚにあるフォルマントのアンプリチュードをオープニング値に比例させて増減させることができ、合成音声に、唇開度による音声の変化を与えることが出来る。 The opening function is fOpen (freq), the opening value is Open, and the excitation resonance frequency ERFreq ′ is obtained from the excitation resonance frequency ERFreq by the following equation (L1), and is used as a characteristic parameter for the opening value (Open).

Further, the i-th formant frequency FormatFreq _i ′ is obtained from the i-th formant frequency FormatFreq _i by the following equation (L2) and used as a feature parameter at the opening value (Open).

As a result, the amplitude of the formant at a frequency of 0 to 500 Hz can be increased or decreased in proportion to the opening value, and a change in voice due to the lip opening can be given to the synthesized voice.

  It should be noted that the synthesized speech can be further diversified by preparing and changing functions for which the opening value is input for each singer.

  FIG. 12 is a diagram illustrating a first application example of the template according to the present embodiment. The case where the singing by the score of (a) in the figure is synthesized according to the present embodiment will be described.

  In this score, the pitch of the first half note is “So”, the strength is “Piano (weak)” and the pronunciation is “A”. The pitch of the second half note is “do”, the strength is “mesoforte” (slightly strong), and the pronunciation is “a”. Since the two half notes are connected by legato, there is no break between the sounds and they are connected smoothly.

  Here, it is assumed that the change time from “So” to “Do” is given together with the input data (music score).

  First, two pitch frequencies are obtained from the note names. Thereafter, the end point and start point of the two pitches are connected by a straight line, and the pitch of the boundary portion of the note can be obtained as shown in FIG.

  Next, in terms of dynamics, values corresponding to dynamic symbols such as “piano (weak)” and “mesoforte (slightly strong)” are stored as a table and converted into numerical values using this. Get the dynamics value corresponding to the note. By connecting the two dynamics values thus obtained with a straight line, the dynamics value at the boundary portion of the note can be obtained as shown in FIG.

  If the pitch and dynamics values obtained in this way are used as they are, the pitch and dynamics change abruptly at the note boundary. Therefore, in order to connect to the legato, the note boundary ( Apply the NN template as shown in b).

  Here, the NN template is applied only to the pitch and dynamics, and the pitch and dynamics in which the boundary portions of the notes as shown in FIG.

  Next, the characteristic parameters at each time as shown in (d) of the figure are obtained from the Timbre database TDB using the determined pitch and dynamics shown in (c) in the figure as an index and the phoneme name "A" as an index.

  The stationery template corresponding to the phoneme name “a” shown in FIG. 4C is applied to the characteristic parameters at each time obtained here, and the voice fluctuation is added to the stationary part other than the connected part of the note boundary. Thus, a characteristic parameter as shown in FIG.

  Next, the rest of the NN template (formant frequency, etc.) applied only to pitch and dynamics in (b) in the figure is applied to the feature parameters shown in (e) in the figure, and fluctuations are caused in the formant frequency, etc. at the boundary of the note. A feature parameter indicated by (f) in the given figure is obtained.

  Finally, by performing speech synthesis using the pitch and dynamics in (c) in the figure and the characteristic parameters in (f) in the figure, it is possible to synthesize a song represented by the score in (a) in the figure.

  In FIG. 12B, the time width of the portion to which the NN template is applied can be increased, for example, as shown in FIG. As shown in FIG. 13, when the time width of the portion to which the NN template is applied is increased, the NN template is extended and applied, so that it is possible to synthesize a singing voice having a slow change.

  Conversely, if the time width during which the NN template is applied is narrowed, a singing voice that changes quickly and smoothly can be synthesized. By controlling the application time of the NN template in this way, the speed of change can be controlled.

  Also, even when the pitch is changed from one height to another at the same time, there is a way of singing that is changed rapidly in the first half and slowly in the second half, and vice versa. In this way, there are various ways of changing the pitch, and the difference appears as a difference in the way of listening musically. Therefore, by creating and recording a plurality of types of NN templates from voices sung by changing the way of singing such legato, various variations can be given to the synthesized voice.

  Further, there are various ways of changing the pitch (pitch) other than the legato playing method described above, and a template may be separately created and recorded for these.

  For example, instead of changing the pitch completely continuously like legato, changing the pitch for each semitone, or changing only by the scale used by the length of the song (for example, Doremifasolaside in C major) There is a so-called glissando playing technique.

  In this case, it is possible to create an NN template from the voice actually sung by the glissando, and synthesize a song in which two notes are smoothly connected by applying the template.

  In this embodiment, the NN template is created and recorded only when the pitch changes with the same phoneme. For example, the pitch changes with a different phoneme such as “A” to “E”. You can also create a case. In this case, although the number of NN templates will increase, it can be brought closer to actual singing.

  FIG. 14 is a diagram illustrating a second application example of the template according to the present embodiment. The case where the singing by the score of (a) in the figure is synthesized according to the present embodiment will be described.

  In this score, the pitch of the first half note is “So”, the strength is “Piano (weak)” and the pronunciation is “A”. The pitch of the second half note is “do”, the strength is “mesoforte (slightly strong)”, and the pronunciation is “e”.

  Here, the articulation time from “A” to “E” is set as a fixed value for each combination of two phonemes, or given together with input data.

  First, two pitch frequencies are obtained from the note names. Thereafter, the end point and the start point of the two pitches are connected by a straight line, and the pitch of the note boundary portion (articulation portion) can be obtained as shown in FIG.

  Next, in terms of dynamics, values corresponding to dynamic symbols such as “piano (weak)” and “mesoforte (slightly strong)” are stored as a table and converted into numerical values using this. Get the dynamics value corresponding to the note. By connecting the two dynamics values thus obtained with a straight line, the dynamics value at the boundary portion of the note can be obtained as shown in FIG.

  Next, using the determined pitch and dynamics shown in (b) in the figure, and the phoneme names “a” and “e” as indexes, the time parameters as shown in (c) in the figure from the Timbre database TDB. Ask for. However, the feature parameter of the articulation part is a value obtained by linear interpolation between the end point part of the phoneme “A” and the start point part of the phoneme “E”.

  Next, as shown in (c) in the figure, each of the characteristic parameters obtained in advance for the stationery template “a”, the articulation template from “a” to “e”, and the stationery template “e” To obtain the characteristic parameters as shown in FIG.

  Finally, speech synthesis is performed using the pitch and dynamics of (b) in the figure and the characteristic parameters of (d).

  In this way, it is possible to synthesize a singing voice that naturally changes from “a” to “e”, as in the case where a person actually utters.

  As with the NN template, if the articulation template can be given the length of the boundary part (articulation part) along with the score, articulation from “a” to “e” Can be controlled by synthesizing a slowly changing sound or a rapidly changing sound by expanding and contracting one template. That is, by doing this, it is possible to control the time during which the phoneme changes.

  FIG. 15 is a diagram illustrating a third application example of the template according to the present embodiment. The case where the singing by the score of (a) in the figure is synthesized according to the present embodiment will be described.

  In this score, the intensity of all notes whose pitch is “So” and pronunciation is “A” is gradually increased from the rising edge and gradually decreased at the falling edge.

  In the case of this musical score, the pitch and dynamics are flat as shown in FIG. The NA template is applied to the beginning of these pitches and dynamics, and the NR template is applied to the end of the musical notes to obtain and determine the pitch and dynamics as shown in FIG.

  It should be noted that the lengths to which the NA template and the NR template are applied are input with the crescendo and decrescendo symbols themselves having a length.

  Next, using the determined pitch, dynamics and phoneme name “A” in the figure (c) as an index, the characteristic parameters of the normal part which is neither an attack nor a release are obtained as shown in the figure (d).

  Further, a stationary template is applied to the characteristic parameter of the normal part shown in (d) in the figure to obtain a characteristic parameter given fluctuation as shown in (e) in the figure. Based on the feature parameter (e), the feature parameters of the attack portion and the release portion are obtained.

  The characteristic parameter of the attack part is obtained by applying the NA template of the phoneme “A” to the start point of the normal part (end point of the attack part) by the above-described type 2 method.

  The characteristic parameter of the release part is obtained by applying the NR template of the phoneme “A” to the end point of the normal part (start point of the release part) by the type 1 method described above.

  In this way, the characteristic parameters of the attack part, the normal part, and the release part are obtained as shown in FIG. By synthesizing speech using this characteristic parameter and the pitch and dynamics of (c), singing speech sung by crescendo and decrescendo according to the score of (a) can be obtained.

  As described above, according to the present embodiment, since the feature parameters are changed using the phoneme template obtained by analyzing the actual human singing voice, the vowel part of the singing voice is extended long, It is possible to generate natural synthesized speech that reflects the characteristics of the part where the change occurs.

  In addition, according to the present embodiment, since the feature parameter is changed using the note template obtained by analyzing the actual human singing voice, not only the difference in volume but also the expressive power of musical strength is obtained. You can generate synthesized speech.

  Furthermore, according to the present embodiment, it is possible to interpolate and use other prepared data without preparing data with finely changed music expression such as pitch, dynamics, and opening. With a small number of samples, the database size can be reduced and the database creation time can be shortened.

  Furthermore, according to this embodiment, even if a database using only the pitch as an index is used as the music expression level, three music expressions of pitch, opening, and dynamics are simulated using the opening and dynamics functions. You can get an effect close to using a database with the degree as an index.

  In this embodiment, as shown in FIG. 2, the phoneme track PHT, the note track NT, the pitch track PIT, the dynamics track DYT, and the opening track OT are input as the input data Score, but the configuration of the input data Score is It is not limited to this.

  For example, a vibrato track may be added to the input data Score shown in FIG. In the vibrato track, vibrato values of 0 to 1 are recorded.

  In this case, the database 4 stores, as a vibrato template, a function or table that returns a time series of pitch and dynamics using a vibrato value as an argument.

  Then, by applying this vibrato template in the calculation of the pitch and dynamics in step SA5 in FIG. 4, it is possible to obtain the pitch and dynamics giving the vibrato effect.

  The vibrato template can be obtained by analyzing the actual human singing voice.

  In addition, although the present Example demonstrated centering on the singing voice synthesis | combination, it is not restricted to a singing voice, The voice of a normal conversation, an instrument sound, etc. can be synthesize | combined similarly.

  In addition, you may make it implement a present Example by the commercially available computer etc. which installed the computer program etc. corresponding to a present Example.

  In that case, the computer program or the like corresponding to the present embodiment may be provided to the user while being stored in a storage medium that can be read by the computer, such as a CD-ROM or a floppy disk.

  When the computer or the like is connected to a communication network such as a LAN, the Internet, or a telephone line, a computer program or various data may be provided to the computer or the like via the communication network.

  Although the present invention has been described with reference to the embodiments, the present invention is not limited thereto. It will be apparent to those skilled in the art that various modifications, improvements, combinations, and the like can be made.

It is a block diagram showing the structure of the speech synthesizer 1 by the Example of this invention. It is a conceptual diagram which shows an example of input data Score. It is an example of a Timbre database TDB. It is another example of the Timbre database TDB. It is an example of a stationery template database. It is an example of an articulation template database. It is an example of NA template database NADB. It is an example of NN template database NNDB. It is a flowchart showing a feature parameter generation process. It is a graph showing an example of a dynamics function. It is a graph showing an example of an opening function. It is a figure showing the 1st example of application of the template by a present Example. It is a figure showing the modification of the 1st application example of the template by a present Example. It is a figure showing the 2nd application example of the template by a present Example. It is a figure showing the 3rd application example of the template by a present Example.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Speech synthesizer, 2 ... Data input part, 3 ... Feature parameter generation part, 4 ... Database, 5 ... EpR speech synthesis engine

Claims (4)

Storage means for storing a feature amount of speech at a specific time including at least one of excitation resonance or formant as a phoneme and a pitch as an index;
Template storage means for storing a template representing a temporal change in the feature amount of the pitch and the voice as an index of the phoneme and the pitch;
Input means for inputting speech information for speech synthesis including at least pitch and phoneme;
Reading means for reading the feature amount and the template of the voice from the storage means and the template storage means, respectively, according to the inputted voice information;
Voice synthesis means for applying the read template to the pitch included in the read voice feature quantity and the input voice information, and synthesizing voice based on the voice feature quantity and pitch after the application In a speech synthesizer having
When the pitch included in the input voice information exceeds the value of the highest index in the storage means, the reading means is the highest stored in the storage means from the pitch included in the input voice information A pitch difference obtained by subtracting the pitch of the index is obtained, and a feature amount obtained by adding the pitch difference to the frequency of the excitation resonance or the formant included in the feature amount read from the storage unit by the highest pitch index is used for the speech synthesis. A speech synthesizer that outputs to the means. Storage means for storing a feature amount of speech at a specific time including at least one of excitation resonance or formant as a phoneme and a pitch as an index;
Template storage means for storing a template representing a temporal change in the feature amount of the pitch and the voice as an index of the phoneme and the pitch;
Input means for inputting speech information for speech synthesis including at least pitch and phoneme;
Reading means for reading the feature amount and the template of the voice from the storage means and the template storage means, respectively, according to the inputted voice information;
Voice synthesis means for applying the read template to the pitch included in the read voice feature quantity and the input voice information, and synthesizing voice based on the voice feature quantity and pitch after the application In a speech synthesizer having
When the pitch included in the input voice information is lower than the lowest index value in the storage means, the reading means is the lowest stored in the storage means from the pitch included in the input voice information. A pitch difference obtained by subtracting the pitch of the index is obtained, and a feature amount obtained by adding the specified ratio of the pitch difference to the frequency of the excitation resonance or the formant included in the feature amount read from the storage unit by the lowest pitch index. A speech synthesizer for outputting to the speech synthesizer. The audio feature quantity stored in the storage means includes excitation resonance,
3. The speech synthesizer according to claim 2, wherein the reading unit corrects and outputs the excitation resonance so that the bandwidth of the excitation resonance becomes narrower as the pitch included in the inputted speech information is lower. The audio feature quantity stored in the storage means includes a formant,
3. The speech synthesizer according to claim 2, wherein the reading means corrects and outputs the formant so that the amplitude of the formant increases as the pitch included in the input speech information decreases.

Images