JP5301376B2 - Speech synthesis apparatus and program - Google Patents

Description

  The present invention relates to a speech synthesizer that generates synthesized speech. In particular, the present invention relates to a speech synthesizer that generates synthesized speech using speech of another speaker different from the target speaker.

  When a synthesized speech is composed using the speech unit of the target speaker, if there is no speech unit having an appropriate voice quality of the target speaker, it is possible to convert the speech unit for use. Patent Document 1 describes conversion of voice quality.

  However, if the amount of conversion is large, the sound quality of the speech segment itself may deteriorate. If there is not enough variation of the target speaker's speech segment, if the speech unit of the other speaker can be used in the same way as the target speaker's speech unit, as long as the target speaker's voice quality does not feel uncomfortable It is considered that the lack of speech segments can be compensated and the variation of speech segments can be expanded. Patent Document 2 describes a technique for performing speech synthesis using a database of speech units of other speakers.

  On the other hand, when the speech unit of another speaker is used, the synthesized speech is formed in such a way that the speech unit of the other speaker is present in the sentence of the target speaker's speech unit. It is desirable not to make a sound. In Non-Patent Document 1, in such a case, when the time length of the speech unit of the other speaker is short, or when the fundamental frequency of the part of the speech unit of the other speaker is low and not prominent as a prosody, It is described that it is difficult to recognize that the speaker is another speaker, and natural speech can be obtained as a whole.

JP 2007-148172 A JP 2007-025042 A

Reiko Tadaka, Hiroyuki Seki, Nobumasa Kiyama, Toru Toki, “Naturalities and spectral features of partially substituted speech by different phonemes”, IEICE Technical Report. SP, Voice, March 2008, vol. 107, no. 551, p. 123-128

  When constructing synthesized speech using speech units of other speakers different from the target speaker, depending on the characteristics of the speech units used, the unnaturalness of the speech as a whole may or may not stand out. Both cases are considered to exist. However, if a speech unit of another speaker is selected by brute force trial and error, the efficiency for obtaining synthesized speech is poor. In selecting a speech unit of another speaker that can be used without a sense of incongruity, it is required that whether or not the speech unit is usable can be efficiently selected according to a predetermined condition.

  The present invention has been made in order to solve the above-described problems, and it is intended for other speakers who do not give a sense of incongruity to the voice quality of the target speaker during the synthesized speech mainly composed of the speech unit of the target speaker. An object of the present invention is to provide a speech synthesizer and a program capable of efficiently creating high-quality synthesized speech using speech segments.

[1] In order to solve the above problems, a speech synthesizer according to an aspect of the present invention corresponds to a speech database storage unit that stores speech segments of a target speaker and other speakers, and a target synthesized speech. A notation data storage unit that stores notation data, a phoneme feature value fitness estimation unit that calculates phoneme feature value suitability between the plurality of speech units based on the feature values of the plurality of speech units, and Based on the notation data acquired from the notation data storage unit, a speech unit candidate constituting the target synthesized speech is selected from the speech database storage unit, and the selected speech unit candidate is selected. the speech unit sac Chi other speakers, to determine whether to adopt the candidates of the speech unit on the basis of the phonemic feature quantity matching degree of the phonemic feature quantity matching degree estimation unit is calculated, the result Before adopted Comprising a speech unit selection unit configured to output the synthesized speech constituted by speech segment, a,
The phoneme feature value fitness estimation unit is configured to determine the phoneme feature value match between the speech unit of the other speaker and an arbitrary speech unit of the target speaker, or the speech unit of the other speaker and the speech. A phoneme feature quantity matching degree with a speech unit other than the speech unit of the other speaker among the speech unit candidates selected by the unit selection unit is calculated .
According to this configuration, based on the calculated phoneme feature value matching degree, synthesized speech that employs speech segments having a high matching degree is output.

[2] A speech synthesizer according to an aspect of the present invention is the speech synthesizer described above, wherein the type of phoneme of the speech unit and the phoneme of the speech unit before and after the speech unit in the synthesized speech A phoneme environment estimation unit that calculates a phoneme environment suitability based on at least the type, and the speech unit selection unit includes a phoneme environment for a speech unit of another speaker among the speech unit candidates The adaptability is calculated by the phoneme environment estimation unit, and it is determined whether to adopt the speech segment candidate based on the phoneme environment adaptability.
According to this configuration, the phoneme environment of the speech segment that is a candidate for adoption is determined, and the phoneme environment suitability based on the determination result is calculated. Then, synthesized speech using speech segments having a high phoneme environment suitability is output.
The phoneme environment estimation unit may further calculate a phoneme environment suitability based on phoneme type discrimination.
Further, the phoneme environment estimation unit may further calculate a phoneme environment suitability based on prosodic environment discrimination.

[3] Further, the speech synthesizer according to one aspect of the present invention is the speech synthesizer described above, wherein the phoneme feature value fitness estimator is a first-order coefficient of a spectral tilt of the speech segment or an FFT cepstrum coefficient, or Any one of the feature values representing the characteristics of the vocal cord sound source is used as the feature value.
The FFT cepstrum coefficient is a cepstrum coefficient obtained using FFT (Fast Fourier Transform). According to this configuration, synthesized speech that employs speech units with high fitness is output using the formant attenuation from the low frequency range to the high frequency range as a feature value.

[4] Further, in the speech synthesizer according to one aspect of the present invention, in the speech synthesizer described above, the phoneme feature value fitness estimation unit may calculate a spectrum centroid in a predetermined frequency band of the speech spectrum of the speech segment. The frequency is used as the feature amount.
According to this configuration, synthesized speech using a speech unit having a high degree of fitness is output using the spectral centroid in a predetermined low frequency band (for example, a low frequency band) as a feature amount.

[5] Further, in the speech synthesizer according to one aspect of the present invention, in the speech synthesizer described above, the phoneme feature amount fitness estimation unit uses the formant frequency and formant bandwidth of the speech unit as the feature amount. It is characterized by that.
According to this configuration, synthesized speech that employs speech units having high fitness is output using formant frequency and formant bandwidth as feature quantities.

[6] In addition, the speech synthesizer according to an aspect of the present invention further includes the other speaker ratio setting storage unit that stores a setting value of the ratio of the number of speech units of other speakers in the speech synthesizer described above. The speech unit selection unit is configured so that a ratio of speech units of other speakers out of the speech units constituting the synthesized speech is equal to or less than the set value read from the other speaker ratio setting storage unit. The speech unit of the other speaker with the calculated phoneme environment suitability is adopted, and the speech unit of the target speaker reselected from the speech database storage unit is used for the speech unit of the other speaker. It is characterized by being replaced with a piece.

[7] A computer program according to an aspect of the present invention includes a speech database storage unit that stores speech segments of a target speaker and other speakers, and notation data that stores notation data corresponding to a target synthesized speech. A phoneme feature quantity fitness estimation step for calculating a phoneme feature quantity suitability between the plurality of speech segments based on the feature quantities of the plurality of speech segments in a computer comprising the storage unit, and Based on the notation data acquired from the notation data storage unit, a speech unit candidate constituting the target synthesized speech is selected from the speech database storage unit, and the selected speech unit candidate is selected. the speech unit Urn Chi other speakers, to determine whether to adopt the candidates of the speech unit on the basis of the phonemic feature quantity matching degree calculated by the phonemic feature quantity matching degree estimation process, A result adopted speech unit selection step of outputting the synthesized speech constituted by the speech segment, a program for executing processing of the phoneme feature quantity matching degree estimation process, said other speakers Phoneme feature match between a speech unit of the target speaker and an arbitrary speech unit of the target speaker, or the speech unit selected by the speech unit and the speech unit selection process of the other speaker A phoneme feature amount matching degree between speech candidates other than the speech unit of the other speaker among the candidates is calculated.

  According to the present invention, in order to make up for the shortage of speech units of the target speaker and to expand variations, the speech unit of the other speaker that can be used without a sense of incongruity with the speech unit of the target speaker is selected and used. Can improve the quality of synthesized speech. Further, all or part of the process of selecting the speech unit of such other speaker can be automatically performed, and the efficiency of selecting the speech unit of the other speaker is increased.

It is the block diagram which showed the function structure of the speech synthesizer by embodiment of this invention. It is the schematic which showed the structure and example of data of the audio | voice database (target speaker audio | voice database, another speaker audio | voice database) by the embodiment. It is the block diagram which showed the schematic which showed the structure and example of data of the synthetic | combination audio | voice storage part by the embodiment. It is a graph for demonstrating the spectrum inclination which is one of the feature-values which the phoneme feature-value adaptation degree estimation part 113 by the embodiment uses. It is a graph for demonstrating the low-pass spectrum gravity center which is one of the feature-values which the phoneme feature-value adaptation degree estimation part 113 by the embodiment uses. It is the flowchart which showed the process sequence of the whole speech synthesizer by the same embodiment.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
FIG. 1 is a block diagram showing a functional configuration of the speech synthesizer according to the present embodiment. As shown in the figure, the speech synthesizer 10 includes a speech unit selection unit 110, a phoneme environment suitability estimation unit 112, a phoneme feature amount suitability estimation unit 113, an other-speaker speech unit location designation unit 120, A speech database storage unit 130, a comparative phoneme designating unit 140, a synthesized speech storage unit 150, a text storage unit 155 (notation data storage unit), an other speaker ratio setting storage unit 160, and a default setting storage unit 170 Consists of including.
The voice database storage unit 130 includes a target speaker voice database 131 and an other speaker voice database 132.

The speech unit selection unit 110, the phoneme environment suitability estimation unit 112, the phoneme feature amount suitability estimation unit 113, the other-speaker speech unit location designation unit 120, and the comparison phoneme designation unit 140 are electronic circuits. Realized as an information processing apparatus. The speech database storage unit 130, the synthesized speech storage unit 150, the text storage unit 155, the other speaker ratio setting storage unit 160, and the default setting storage unit 170 are realized using, for example, a magnetic disk device or a semiconductor memory.
The function of each part is as described below.

  The speech unit selection unit 110 is a speech unit that constitutes a desired synthesized speech based on notation data (hirakana sentence, kanji / kana mixed sentence, or phoneme label notation) stored in the text storage unit 155. Are selected from the speech database storage unit 130. If the notation data is stored in Hiragana or Kanji / Kana mixed sentences, the phoneme segment selection unit 110 converts the notation data into phoneme label notation as appropriate, and then stores it in the phonetic database storage unit 130. Select a stored speech segment candidate. In addition, the speech unit selection unit 110 determines a phoneme feature value fitness between a speech unit of another speaker and a speech unit to be compared among speech unit candidates for composing synthesized speech. The phoneme feature quantity matching degree estimation unit 113 is made to calculate, and based on the phoneme feature quantity matching degree, it is determined whether or not to adopt the speech segment candidate. Further, the speech unit selection unit 110 causes the phoneme environment estimation unit to calculate the phoneme environment suitability for the speech unit of another speaker among the speech unit candidates, and based on the phoneme environment suitability Decide whether or not to adopt speech segment candidates. Then, the speech segment selection unit 110 outputs a synthesized speech configured using the speech segment that has been adopted as a result. Further, the speech unit selection unit 110 calculates so that the ratio of speech units of other speakers out of the speech units constituting the synthesized speech is equal to or less than the set value read from the other speaker ratio setting storage unit 160 The speech unit of the other speaker with the highest phoneme environment suitability is adopted, and the speech unit of the other speaker is replaced with the speech unit of the re-selected target speaker.

The phoneme environment suitability estimation unit 112 calculates a phoneme environment suitability based at least on the phoneme type of the speech unit and the phoneme types of the speech units before and after the speech unit in the synthesized speech. Furthermore, the phoneme environment suitability estimation unit 112 may calculate the phoneme environment suitability based on the determination of the phoneme type of the phoneme and the determination of the prosodic environment.
Here, the type of phoneme is (1) whether (1) the phoneme is a vowel or consonant, (2) whether the phoneme is voiced or unvoiced, and (3) the phoneme articulation method (for example, nasal consonant). It is a kind classified by any one of-(3), or these combination.
The phoneme feature quantity matching degree estimation unit 113 calculates phoneme feature quantity suitability between the plurality of phonemes based on the feature quantities of the given phonemes. Here, the feature amount of the speech element includes a spectrum inclination, a low-frequency spectrum centroid, a formant frequency, a formant bandwidth, and the like, which will be described in detail later.

The other-speaker speech segment location specifying unit 120 specifies a location where the other-speaker speech segment is used in the synthesized speech and a range of a speech database for selecting the speech segment based on an input from the user. And do.
The speech database storage unit 130 stores speech segments of the target speaker and other speakers.
The comparison phoneme designation unit 140 designates a speech unit to be compared by the phoneme feature value fitness estimation unit, that is, a reference speech unit.

The synthesized speech storage unit 150 stores data relating to a plurality of speech units for constituting synthesized speech. When the speech unit selection unit 110 selects or rejects a speech unit, the synthesized speech storage unit 150 is appropriately rewritten.
The text storage unit 155 stores text of notation data corresponding to the target synthesized speech. This notation data is, for example, data such as Japanese hiragana. Note that data in a form other than hiragana, such as a phoneme label notation, may be used as a column of labels corresponding to phonemes.
The other speaker ratio setting storage unit 160 stores a setting value of the ratio of the number of speech units of other speakers.
The default setting storage unit 170 stores default setting values. The default setting value is, for example, a location where another speaker's speech unit is used in the synthesized speech, a range of a speech database for selecting a speech unit, or the like.

Note that a phoneme is a minimum unit of sound used for meaning discrimination in a language. For example, in Japanese, vowels such as “a”, “i”, “u”, “e”, “o”, “k”, “s”, “t”, “n”, “h”, “ Each consonant such as “m”,... corresponds to a phoneme.
A speech segment is a component for configuring synthesized speech, and is speech data in a short unit prepared in advance. The speech element may correspond to a single phoneme or may correspond to a plurality of phoneme strings.

The fundamental frequency is the frequency of the frequency component having the lowest periodicity of the audio signal.
A formant is a peak in the frequency spectrum of speech. Among these peaks, they are called first formant, second formant, third formant,... In order from the lowest frequency. The formant frequency pattern is an element that characterizes phonemes.
The target speaker is a target speaker when composing synthesized speech. The main speaker of the speech segment constituting the synthesized speech to be created is the target speaker.
The other speaker is a speaker different from the target speaker. The speech synthesizer 10 according to the present embodiment creates synthesized speech that can be recognized by a person as if the speech of the target speaker as a whole, while also mixing speech segments of other speakers.

FIG. 2 is a schematic diagram illustrating a data configuration and a data example of a voice database (target speaker voice database 131 and other speaker voice database 132) stored in the voice database storage unit 130. As shown in the figure, this speech database is tabular data, and includes speaker identification information, speech segment identification information, phoneme label notation, triphone, speech signal data, spectral features, and the like. , Each item of basic frequency information.
This row of data exists for each speech unit, and the main key of the data is speech unit identification information.
The speaker identification information is data that uniquely identifies the speaker.

The speech unit identification information is information for uniquely identifying a speech unit.
The phoneme label notation is data in which the pronunciation of the speech segment is expressed using Roman letters. Here, the capital letter “Q” represents a prompt sound, and the symbol “:” (colon) represents a long sound. Therefore, for example, the phoneme label notation “hoQkaido:” in the row of the speech unit identification information “B0001” in the table represents the pronunciation “Hokkaido”.
The triphone is a notation representing a phoneme environment. For example, the triphone “a−o + i” in the row of the speech unit identification information “B0009” in the table indicates that the phoneme preceding the phoneme “o” of the speech unit is “a” and the subsequent phoneme. Represents “i”. The notation “sil” in this triphone represents silence. That is, in the triphone “o−i + sil” in the row of the speech unit identification information “B0007” in the table, the phoneme preceding the phoneme “i” of the speech unit is “o”, and the subsequent phoneme is This means that there is no sound. Thus, for example, when comparing the line with the speech unit identification information “B0004” and the line with “B0005” in the table, the speaker identification information “A001” and the phoneme label notation “a” are common. The triphone is different. That is, the speech database storage unit 130 stores speech segments corresponding to the phoneme environment.
The audio signal data is data representing the audio signal itself of the speech unit. This audio signal data is represented, for example, as time-series sound pressure level data or as frequency spectrum data in a predetermined short period.
The spectrum feature amount is data representing the feature amount of the speech unit, and for example, MFCC (Mel-Frequency Cepstrum Coefficient) is used.
The fundamental frequency information is data representing the fundamental frequency of the speech unit, and a representative value of the fundamental frequency in the speech unit or a time-series value of the fundamental frequency is used. Alternatively, the range of the fundamental frequency may be binarized with H (High, high frequency) and L (Low, low frequency), and the time series value of “H” or “L” may be used as the fundamental frequency information.
Note that the spectral feature amount and the fundamental frequency are used when selecting a speech unit, as will be described later.

The target speaker voice database 131 and the other speaker voice database 132 may be stored in individual database tables or in a common database table. In any case, since the speaker identification information is stored in the data, comparing the speaker identification information of the target speaker with the speaker identification information in the database, It is possible to distinguish between speaker speech units.
In addition to the illustrated data items, for example, an audio file number, time information, and the like may be held as items in a table on the audio database storage unit. Here, the audio file number is a number for uniquely identifying an audio file stored outside the table and holding audio signal data or the like. The time information is time information of the target phoneme included in the audio signal data (information indicating the start point and the end point of the phoneme as relative time from the head).

  FIG. 3 is a schematic diagram illustrating a data configuration and a data example of the synthesized speech data stored in the synthesized speech storage unit 150. As shown in the figure, this speech database is tabular data, and each item of synthesized speech identification information, order, speaker identification information, speech segment identification information, phoneme label notation, and speech signal data is stored. Have. In this data, there is a row for each individual speech unit constituting the synthesized speech.

The synthesized speech identification information is data that uniquely identifies the synthesized speech.
The order is a value indicating the order of speech units within a certain synthesized speech.
The speaker identification information is data that uniquely identifies the speaker of the speech unit, and is the same as the speaker identification information stored in the speech database storage unit 130.
The speech unit identification information is data that uniquely identifies a speech unit.
The phoneme label notation is data representing the phoneme label of the speech segment, and is similar to the phoneme label notation stored in the speech database storage unit 130.
The voice signal data is data representing the voice signal itself of the voice unit, and is similar to the voice signal data stored in the voice database storage unit 130.
In the example of data shown in the figure, the speaker identification information of the first speech unit of the synthesized speech identified by the synthesized speech identification information “C0001” is “A001”, and the speech unit identification information is “A001”. “B0002”, and the phoneme label notation is “to: kyo:”. Further, the speaker identification information of the second speech unit of the synthesized speech is “A002”, the speech unit identification information is “B0777”, and the phoneme label notation is “kara”. The same applies to the third and subsequent speech segments.

  Next, estimation of the fitness of speech segments will be described. The speech synthesizer 10 uses a phoneme environment suitability and a phoneme feature value suitability as speech unit suitability.

  The phoneme environment suitability estimation unit 112 estimates the phoneme environment suitability. Therefore, the phoneme environment suitability estimation unit 112 performs phoneme type discrimination, front-and-rear phoneme environment discrimination, and prosodic environment discrimination.

In the determination of the phoneme type, the phoneme environment suitability estimation unit 112 performs (1) determination of whether it is a vowel or consonant, (2) determination of whether it is voiced or unvoiced, and (3) determination based on the articulation method.
(1) Discriminating vowels / consonants The phoneme environment suitability estimation unit 112 stores a predetermined index for each of the vowels and consonants, and the speech unit that is the target of the suitability estimate is a vowel or consonant. The index value is used to calculate the phoneme environment suitability of the speech segment according to which one of the speech units is. Note that consonants have higher phoneme environment suitability (easy to fit) than vowels.
(2) Discrimination of voiced / unvoiced sound The phoneme environment suitability estimation unit 112 stores a predetermined index for each of voiced sound and unvoiced sound, and there is a speech unit that is a target of fitness estimation. Depending on whether it is a voice sound or an unvoiced sound, the index value is used to calculate the phoneme environment suitability of the speech segment. In addition, unvoiced sound has higher phoneme environment suitability than voiced sound (it is easy to adapt).
(3) Discrimination of articulation method The articulation method is a method in which the vocal tract is closed or narrowed by a voice organ. For example, “m” and “n”, which are sounds applied to the nose, belong to a common articulation method. Also, “p”, “t”, and “k”, which are sounds that burst after closing the mouth, belong to a common articulation method. The phoneme environment suitability estimation unit 112 stores a predetermined index for each articulation method, and depending on which of them is the articulation method of the speech unit for which the suitability is to be estimated. The index value is used to calculate the phoneme environment suitability of the speech segment.

In the discrimination of the front and back phoneme environment, the phoneme environment suitability estimation unit 112 stores in advance an index of the front and back phoneme environment for each phoneme type and for each speaker. Then, the phoneme environment suitability estimation unit 112 reads the data of the speech units before and after the target speech unit based on the order data stored in the synthesized speech storage unit 150, and determines the target speech unit. Determine the type of phoneme before and after and the speaker. Then, the phoneme environment suitability estimation unit 112 uses the determined phoneme type and the index value corresponding to the speaker to calculate the phoneme environment suitability.
The fitness based on the front and back phoneme environment calculated by the above method represents the goodness of connection between the front and rear sounds.

In discrimination of the prosodic environment, the phoneme environment suitability estimation unit 112 stores in advance an index corresponding to the phoneme time length and the relative height of the fundamental frequency of the phoneme. Then, index values corresponding to the phoneme time length of the target phoneme and the relative height of the fundamental frequency of the phoneme are used to calculate the phoneme environment suitability.
Note that the shorter the phoneme duration, the higher the phoneme environment suitability (easy to adapt). When the phoneme time length is longer than a predetermined threshold, the phoneme environment suitability is extremely low (cannot be used). Moreover, the lower the fundamental frequency of the phoneme, the higher the phoneme environment suitability (easy to adapt).

Note that for a phoneme unit including a plurality of phonemes, the phoneme environment suitability estimation unit 112 calculates a phoneme environment suitability for each phoneme.
Then, the phoneme environment suitability estimation unit 112 calculates the phoneme environment suitability from the weighted sum of each index value obtained above.

  The phoneme feature value fitness estimation unit 113 estimates the phoneme feature value fitness. Therefore, the phoneme feature quantity matching degree estimation unit 113 performs a process of comparing the spectrum feature quantities. The spectral feature used here is the spectral tilt, the first-order coefficient (C1) of the FFT cepstrum coefficient, the feature representing the characteristics of the vocal cord sound source (spectral characteristics), the spectral centroid of the low frequency band of the spectrum (low spectral centroid) ), Formant frequency, formant bandwidth, etc.

FIG. 4 is a graph showing a speech spectrum envelope for explaining the spectrum inclination obtained by the phoneme feature value fitness estimation unit 113.
In the figure, the horizontal axis represents frequency (unit: hertz) and the vertical axis represents intensity (unit: decibel). Also, the peak points appearing in the illustrated speech spectrum envelope are points P ₁ , P ₂ , P ₃ ,... From the lower frequency side. In addition, the coordinates of the points P ₁ , P ₂ , and P ₃ in the graph are (f ₁ , m ₁ ), (f ₂ , m ₂ ), and (f ₃ , m ₃ ), respectively. The frequencies f ₁ , f ₂ , and f ₃ are the first, second, and third formant frequencies, respectively.
The spectrum inclination is an inclination of a straight line connecting two predetermined peak points among the plurality of peak points.
In order to calculate the spectrum inclination, the phoneme feature value fitness estimation unit 113 obtains an envelope of the frequency spectrum of the phoneme, obtains a plurality of peak points in the envelope, and obtains the first peak from the lower frequency side and 3 Calculate the slope of the straight line connecting the th peak. This is a spectral inclination, which is a feature quantity representing the degree of attenuation from a low frequency range to a high frequency range.
That is, the phoneme feature quantity matching degree estimation unit 113 calculates the spectrum inclination g by the following equation (1).

  Further, as a value approximating the spectrum inclination, a first order coefficient of the FFT cepstrum coefficient may be used as the feature amount.

  The spectral tilt is affected by the characteristics of the vocal cord sound source and the radiation characteristics when sound is emitted. Since the radiation characteristic can be considered to be almost constant, it can be said that the spectral tilt changes depending on the characteristics of the vocal cord sound source. Therefore, other feature quantities influenced by the characteristics of the vocal cord sound source may be used instead of the spectrum tilt. Specifically, the characteristic amount representing the characteristics of the vocal cord sound source includes a ratio of middle and high frequency noise components, a power (decibel) difference between the first harmonic and the second harmonic obtained from the FFT spectrum, and an FFT spectrum. Either the obtained first harmonic or the power (decibel) difference of the peak near F3 can be used.

FIG. 5 is a graph showing a speech spectrum envelope for explaining the low-frequency spectrum centroid obtained by the phoneme feature value fitness estimation unit 113.
In the same figure, the horizontal axis represents frequency (unit: hertz) and the vertical axis represents intensity (unit: decibel). L is a frequency band having a low spectrum, and the range of the band L is determined in advance.
Then, the phoneme feature value fitness estimation unit 113 calculates the low-frequency spectrum centroid f _W (frequency of the spectrum centroid) by the following equation (2).

  Note that m (f) in Equation (2) represents the intensity at the frequency f in the speech spectrum.

  The formant frequency is a frequency of a plurality of peaks (formant) in the voice spectrum. The formant band width is the band width of the above formant.

  The phoneme feature amount matching degree estimation unit 113 calculates the degree of fit M between phonemes (phoneme 1 and phoneme 2) by using the feature amount as described above and the following equation (3).

In the equation _{(3), t 1, i} is the i-th feature quantity of phonemes 1 (scalar or _{vector), t 2, i} is the i-th feature value of the phoneme 2 (scalar or vector). Further, d (t _{1, i} , t _{2, i} ) is a value (scalar) determined according to the distance between both feature amounts. Further, w _i is a weight value corresponding to the i-th feature value, and this value is determined in advance and stored in the phoneme feature value fitness estimation unit 113.
As a specific example of d (t _{1, i} , t _{2, i} ), for example, a distance between these both feature amounts may be simply used. The fitness M at this time is calculated using the following equation (4).

Note that for a phoneme unit including a plurality of phonemes, the phoneme feature value fitness estimation unit 113 calculates a phoneme feature value fitness for each phoneme.
Note that instead of calculating each feature amount described above, the phoneme feature amount matching degree estimation unit 113 calculates the feature amount of the speech segment in advance and stores the value in the speech database storage unit 130. It may be read and used for comparison.

  FIG. 6 is a flowchart showing the procedure of speech synthesis performed by the speech synthesizer 10. The processing procedure of the speech synthesizer 10 will be described below along this flowchart.

Prior to the processing of this flowchart, predetermined data is stored in the speech database 130, the text storage unit 155, the other speaker ratio setting storage unit 160, and the default setting storage unit 170.
The speech database 130 stores speech segments of a plurality of speakers in advance.
The text storage unit 155 stores a notation text corresponding to the target speech to be synthesized. For example, in the text storage unit 155, “I will go from Tokyo to Yokohama” (Hiragana sentence), “I will go from Tokyo to Yokohama” (a sentence mixed with kanji and kana) and “to: kyo kara”. Text data such as “Yokohama e Ikimasu” (phoneme label notation) is stored.
The other-speaker ratio setting storage unit 160 stores a ratio value (for example, “15%”) of the number of other-speaker speech units out of the total number of speech units in the synthesized speech as a set value. Yes.
In the default setting storage unit 170, a portion (for example, “all locations”) in which the other speaker's speech unit is used in the synthesized speech, and a range of the speech database for selecting the speech unit (speakers to be selected) (For example, “all speakers”) is stored as a set value.

First, in step S1, the other-speaker speech unit location specifying unit 120 selects a location and speech unit that uses the other-speaker speech unit in the synthesized speech based on the user's operation (input). The specification of the range of the speech database is accepted, and the information is passed to the speech segment selection unit 110. At this time, the other-speaker speech unit location designation unit 120 accepts designation in units of individual speech units for locations where the other-speaker speech units are used in the synthesized speech. For example, the other-speaker speech unit location specifying unit 120 sets the synthesized speech identification information and the order stored in the synthesized speech storage unit 150 as a set, and uses the one or more sets of data as the other-speaker speech. It accepts the designation of the location where the segment is used. In addition, as the designation of the range of the speech database for selecting speech segments, the other-speaker speech segment location designating unit 120 selects “both target speaker speech database and other-speaker speech database” or “other-speaker speech”. Information representing either “database only” is accepted. At this time, when only the speech unit of a specific speaker or a plurality of other speakers is selected, the other-speaker speech unit location specifying unit 120 specifies the speaker identification information of the target speaker. It can also be accepted.
Also, at this time, the user's operation can specify “all locations” in the synthesized speech where the other-speaker speech segment is used, and the range of the speech database for selecting speech segments (selection target) Can be designated as “all speakers”.

In this step, it is possible to specify that the default setting value is used instead of a specific specification by the user. When the use of the default setting is designated, the speech element selection unit 110 reads the setting value from the default setting storage unit 170 and uses it.
For example, in the default setting storage unit 170, “all locations” are locations where other speaker speech segments are used in the synthesized speech, and the range of the speech database for selecting speech segments (the speakers to be selected). When it is stored that “range” is “all speakers”, the speech segment selection unit 110 uses the set value.

  In addition, when the location where the other-speaker speech unit is used in the synthesized speech is set to “all locations” by the user's designation or the value set in the default setting storage unit 170, that is, When the location is not designated, the other speaker's speech unit count ratio is not made larger than the ratio stored in the other speaker ratio setting storage unit 160 by the method described later. The location where the other-speaker speech unit is used is determined in the descending order of the fitness level of the speech unit.

Next, in step S2, the speech unit selection unit 110 reads the ratio value from the other speaker ratio setting storage unit 160, and determines whether the ratio of the locations where the other speaker speech unit is used in the synthesized speech is within the set range. Confirm whether or not. Specifically, the speech unit selection unit 110 (based on the information passed from the other-speaker speech unit location specifying unit 120) (the number of speech units that use the other-speaker speech unit in the synthesized speech / The number of all speech units in the synthesized speech is calculated, and it is confirmed whether or not this value is equal to or less than the ratio value read from the other speaker ratio setting storage unit 160. If the calculated value is less than or equal to the set ratio value (step S2: YES), the process proceeds to the next step S3. When the calculated value is larger than the set ratio value (step S2: NO), the process returns to step S1 in order to receive designation based on the operation from the user again.
In addition, when the location using the other-speaker speech unit is not specifically specified in step S1, the determination result in this step is always “YES”.

  Next, in step S <b> 3, the speech unit selection unit 110 selects a necessary speech unit from the target speaker speech database 131 or the other speaker speech database 132 in the speech database storage unit 130. If only a specific location (single or plural) is specified in the previous step, the speech segment to be used is selected only for the relevant location. If a specific speaker (single or plural) is specified in the previous step, a selection is made from the speech units of the speaker in the corresponding database. The selection of the speech unit here is based on the method of the prior art, and is a selection performed by matching of the phoneme label and the like, and matching of the spectral feature quantity and the fundamental frequency held in the speech database storage unit 130. is there.

Next, in step S <b> 4, the speech unit selection unit 110 counts the number of the selected other speaker speech units, and the ratio range is set in the other speaker ratio setting storage unit 160. It is determined whether it is within. If it is within the setting range (step S4: YES), the process jumps to step S6, and if it exceeds the setting range (step S5: NO), the process proceeds to the next step S5.
Note that the determination result in this step may be “NO” because there is no specific location designation in step S 1, and the “all locations” is designated by the user or the value set in the default setting storage unit 170. "Is only specified.

In step S5, the phoneme environment suitability estimation unit 112 estimates the suitability of the phoneme environment of the other-speaker speech unit. Then, the phoneme environment suitability estimation unit 112 determines the priority order of the speech units selected above in order of suitability. As a result, for other speaker speech units whose priority is not within the ratio range set in the other speaker ratio setting storage unit 160, the other speaker speech units are rejected, The speech segment selection unit 110 reselects an alternative target speaker speech segment from the target speaker speech database 131. That is, the speech unit selection unit replaces the other speaker speech unit having the lower priority with the reselected target speaker speech unit. Note that the speech element reselection method here is based on the existing technology as described above.
That is, the phoneme environment suitability estimation unit 112 employs only the speech element having the highest suitability based on the setting value stored in the other speaker ratio setting storage unit 160.

  Next, in step S <b> 6, the phoneme feature amount matching degree estimation unit 113 compares the feature amount of the speech unit selected for the synthesized speech with the feature amount of the speech unit specified by the comparison phoneme specifying unit 140. Then, the fitness of the speech unit selected for the synthesized speech is estimated. At this time, the speech unit specified by the comparison phoneme designating unit 140 as the comparison target is an arbitrary phoneme of the target speaker or the current speech unit selected from the speech units selected by the speech unit selection unit 110. Any one other than the speech unit.

  In step S <b> 7, the speech unit selection unit 110 determines whether speech units having a low degree of matching of phoneme feature values remain in the selected speech unit. In other words, in this determination, for all the other speaker speech units to be compared, the phoneme feature quantity matching level estimation unit 113 has completed the estimation of the phoneme feature quantity matching level, and the matching level is a predetermined level. This is performed depending on whether or not there is something lower than the threshold. If a speech segment having a low phoneme feature value fitness level remains (step S7: YES), the process proceeds to step S8. If no such speech segment remains (step S7: NO), step S9 is performed. Proceed to

When the process proceeds to step S8, in this step, the speech unit selection unit 110 rejects the other-speaker speech unit having a low phoneme feature value fitness estimated by the phoneme feature value fitness estimation unit 113, and the like. Re-select the speech segment. Note that the speech element reselection method here is based on the existing technology as described above. Then, the process returns to step S6.
That is, the selection of the speech unit is repeated until the matching degree of the phoneme feature amount of all the speech units becomes higher than the threshold value.
That is, the speech element selection unit 110 employs a speech element having a high degree of matching of phoneme feature quantities.

  When the process proceeds to step S9, the speech unit selection unit 110 outputs a synthesized speech composed of the selected (adopted) speech unit, and the process of the entire flowchart ends.

  In addition, you may make it implement | achieve part or all of the speech synthesizer 10 in embodiment mentioned above with a computer. In that case, the program for realizing the control function may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read by a computer system and executed. Here, the “computer system” includes an OS and hardware such as peripheral devices. The “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. Further, the “computer-readable recording medium” dynamically holds a program for a short time, like a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line. It is also possible to include those that hold a program for a certain time, such as a volatile memory inside a computer system serving as a server or client in that case. The program may be a program for realizing a part of the functions described above, and may be a program capable of realizing the functions described above in combination with a program already recorded in a computer system.

Although the embodiment has been described above, the present invention can also be implemented in the following modified example.
For example, the speech unit selection unit does not use all of the suitability (various phoneme environment suitability and various phoneme feature suitability) described above, but only a part of them. You may make it determine adoption of a speech segment or rejection.

  The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes designs and the like that do not depart from the gist of the present invention.

  The present invention can be used for the purpose of efficiently generating high-quality synthesized speech without unnaturalness. For example, the present invention can be used for the purpose of broadcasting information such as television and radio, and providing information by voice.

DESCRIPTION OF SYMBOLS 10 ... Speech synthesizer 110 ... Speech unit selection part 112 ... Phoneme environment suitability estimation part 113 ... Phoneme feature quantity suitability estimation part 120 ... Other speaker speech element location designation part 130 ... Speech database storage part 131 ... Target talk Speaker voice database 132 ... other speaker voice database 140 ... comparative phoneme designation unit 150 ... synthesized voice storage unit 155 ... text storage unit (notation data storage unit)
160 ... Other speaker ratio setting storage unit 170 ... Default setting storage unit

Claims (7)

A speech database storage unit for storing speech segments of the target speaker and other speakers;
A notation data storage unit for storing notation data corresponding to the target synthesized speech;
A phoneme feature value fitness estimation unit that calculates a phoneme feature value suitability between the plurality of speech units based on the feature values of the plurality of speech units;
Based on the notation data acquired from the notation data storage unit, a speech unit candidate constituting the target synthesized speech is selected from the speech database storage unit, and the selected speech unit candidate is selected. the speech unit sac Chi other speakers, to determine whether to adopt the candidates of the speech unit on the basis of the phonemic feature quantity matching degree of the phonemic feature quantity matching degree estimation unit is calculated, the result A speech unit selector for outputting the synthesized speech composed of the speech units employed;
Equipped with,
The phoneme feature value fitness estimation unit is configured to determine the phoneme feature value match between the speech unit of the other speaker and an arbitrary speech unit of the target speaker, or the speech unit of the other speaker and the speech. Calculating a phoneme feature value compatibility between speech units other than the speech unit of the other speaker among the speech unit candidates selected by the unit selection unit;
A speech synthesizer characterized by the above. Further comprising a phoneme environment estimation unit that calculates a phoneme environment suitability based on at least a phoneme type of the speech unit and a phoneme type of a speech unit before and after the speech unit in the synthesized speech;
The speech unit selection unit causes the phoneme environment estimation unit to calculate a phoneme environment suitability for a speech unit of another speaker among the speech unit candidates, and based on the phoneme environment suitability Decide whether to adopt speech segment candidates,
The speech synthesizer according to claim 1. The phoneme feature quantity fitness estimation unit uses the value of either the spectral slope of the speech unit or the first order coefficient of the FFT cepstrum coefficient or the feature quantity representing the characteristics of the vocal cord sound source as the feature quantity.
The speech synthesizer according to any one of claims 1 and 2. The phoneme feature amount fitness estimation unit uses the frequency of the spectrum centroid in a predetermined frequency band in the speech spectrum of the speech unit as the feature amount.
The speech synthesizer according to any one of claims 1 to 3, wherein The phoneme feature value fitness estimator uses the formant frequency and formant bandwidth of the speech element as the feature value.
The speech synthesizer according to any one of claims 1 to 4, characterized in that: A speaker ratio setting storage unit that stores a setting value of the ratio of the number of speech units of other speakers;
The speech unit selection unit calculates so that a ratio of speech units of other speakers out of speech units constituting the synthesized speech is equal to or less than the set value read from the other speaker ratio setting storage unit. The speech unit of the other speaker having the highest phoneme environment suitability is adopted, and the speech unit of the other speaker is selected by the speech unit of the target speaker reselected from the speech database storage unit. replace,
The speech synthesizer according to claim 2. A speech database storage unit for storing speech segments of the target speaker and other speakers;
A notation data storage unit for storing notation data corresponding to the target synthesized speech;
In a computer equipped with
A phoneme feature value fitness estimation process for calculating a phoneme feature value suitability between the plurality of speech segments based on the feature values of the plurality of speech segments;
Based on the notation data acquired from the notation data storage unit, a speech unit candidate constituting the target synthesized speech is selected from the speech database storage unit, and the selected speech unit candidate is selected. the speech unit sac Chi other speakers, to determine whether to adopt the candidates of the speech unit on the basis of the phonemic feature quantity matching degree calculated by the phonemic feature quantity matching degree estimation process, as a result A speech segment selection process for outputting the synthesized speech composed of the speech segments employed;
A program for executing the processing,
The phoneme feature value fitness estimation process includes the phoneme feature value match between the speech unit of the other speaker and an arbitrary speech unit of the target speaker, or the speech unit of the other speaker and the speech. Calculating a phoneme feature value fitness between speech units other than the speech unit of the other speaker among the speech unit candidates selected by the unit selection process;
A program characterized by that.

Images