EP1005694B1 - Method for determining a representative of a speech unit of a language from a voice signal comprising phonetical sections - Google Patents

Description

The invention relates to a method for determining a Representatives for a language component of a language a speech signal comprising sound sections.

It is known to the person skilled in the art to divide a signal spoken by a person, that is to say a speech signal, into sound sections (segmentation), each sound section comprising part of the speech signal.

For its part, a language can be described as a set of modular language modules .

An affiliation function specifies the affiliation measure with which a sound section is a representative of a corresponding language module.

There are several approaches to select from Language modules from databases. An optimization takes place thereby about prosodic [1], linguistic [2] or Continuity criteria [3]. Automatically generated Databases are described in [4], [6] and [7].

Hidden Markov models (HMMs) are known from [5].

A segmentation of a speech signal can be done using "Fast Viterbi Alignment" based on the on the speech signal trained HMMs (see [4] and [6]).

It is disadvantageous that a segmentation of a Voice signal in individual sound sections carried out manually is because this requires enormous effort and experience and must be carried out individually for each speaker.

It is also a serious disadvantage that none Checking the suitability of the selected representatives done and thus by choosing a bad representative as a language building block the result of speech synthesis is correspondingly bad.

The object on which the invention is based is to specify a method for determining a representative for a language component of a language from a speech signal comprising sound sections, which avoids the disadvantages described above and ensures an improved choice of the representative.

A segmentation is evaluated using statistical evaluation of the individual sound sections, whereby a statistically "good" representative for each Phonetic section is determined as a segment.

The object of the invention is according to the features of Claim 1 solved.

According to the invention, a method for determination is specified a representative for a selected language module a language consisting of a sound section Speech signal, the sound sections of the speech signal according to belonging to the language modules in one for each selected language module belonging to the group. You thus get for Several language modules each have a group with at least one a sound section. A selection measure serves from which Speech signal for the sound sections selection values too determine and frequencies of the selection values obtained To be determined according to the section of the respective group. Based on Frequencies obtained in this way become a membership function determines the one for each sound section of a group Affiliation measure that indicates how useful this is Phonetic section as a representative (i.e. a selected one Section). The sound section is now called Representative of the group for the selected one Language module determines whose affiliation measure above of a predetermined threshold.

A great advantage of this method is that it does not any representative from the group for the selected one Language module is taken out, but a representative is determined using the selected language module sufficient quality (corresponding to a high degree of belonging) describes.

The sound sections of a group, the one language element are assigned in the speech signal, which are preferably about naturally spoken language as a long speech sample is made accessible to the computer with regard to their Usability statistically distributed. So there are "good" and "bad" sound sections to the selected one Speech module. With the invention in particular avoided a bad section of sound as a representative of the selected language module.

It is a further development of the invention, at least one apply another selection measure to the sound sections, where there is at least one additional selection value for each section of sound. For each group of Sound sections (i.e. for each selected language element) frequencies are determined for all selection values and from these frequencies, as described above, a membership function.

The representative for the selected language element from the group of Sound sections determined by each membership measure (for each selection measure results in a membership function with a membership measure) multiplicatively into a total measure comes in. Is the total dimension for the respective sound section above a predetermined total threshold, so suitable this section of the sound represents the selected one Language component and is made up of the group of sound sections, that belong to this selected language module, selected.

It is advantageous to use several selection measures for the determination of the representative, as this ensures can that no selection value is too bad. The multiplicative weighting of the membership measure in the Overall dimension corresponds to an AND operation Probability density functions. The representative can thus meet all selection measures with sufficient quality.

It is also a further development of the invention that the Phonemes, diphones, triphones, syllables, half-syllables or are words of a language. Combinations of these too mentioned sound sections are possible.

Another development is that the Sound sections of individual states of a hidden Markov model (HMM) are assigned.

a) Energie der jeweiligen Lautabschnitte;

b) Länge der jeweiligen Lautabschnitte;

c) Grundfrequenz der jeweiligen Lautabschnitte;

d) Längenmanipulation der jeweiligen Lautabschnitte;

e) statistisches Maß für ein Passen der jeweiligen Lautabschnitte.

It is also a further development to determine the selection measure by one of the variables listed below:

a) energy of the respective sound sections;

b) length of the respective sound sections;

c) fundamental frequency of the respective sound sections;

d) length manipulation of the respective sound sections;

e) statistical measure for a fit of the respective sound sections.

A special development of the invention consists in synthetic language from the identified representatives produce. The language, determined by language modules, whereby representatives according to the invention for the language modules can be determined on the basis of these representatives in completely new context. From this results a synthetic speech output by using speech modules, embodied by individual representatives (sound sections), in new sequence are issued.

It is also a further development of the invention Phonetic section to represent the selected one To determine the language module, whose degree of affiliation the has the highest value or, if several selection measures are taken into account, the overall dimension of which is the highest value having. So it will be the "best" sound section in the Group of sound sections for the selected one Language module determined.

Further developments of the invention also result from the dependent claims.

Exemplary embodiments of the Invention presented in more detail.

Fig.1

ein Blockdiagramm, das einzelne Schritte eines Verfahrens zur Bestimmung eines Repräsentanten für einen Sprachbaustein einer Sprache aus einem Lautabschnitte umfassenden Sprachsignal darstellt,

Fig.2

eine Skizze, die den Aufbau einer Sprache und deren Abbildung auf ein Sprachsignal, insbesondere einen vorgelesenen Text, darstellt,

Fig.3

eine Skizze, die ein Selektionsmaß 'Längenmanipulation' darstellt,

Fig.4

eine Skizze, die ein Selektionsmaß 'Grundfrequenz' darstellt,

Fig.5

eine Skizze, die ein Selektionsmaß 'Energie' darstellt,

Fig.6

eine Skizze, die ein Selektionsmaß 'SCORE' darstellt.

Show it

Fig. 1

1 shows a block diagram which represents individual steps of a method for determining a representative for a speech component of a speech from a speech signal comprising speech sections,

Fig. 2

a sketch which shows the structure of a language and its mapping onto a speech signal, in particular a read text,

Fig. 3

a sketch which represents a selection measure 'length manipulation',

Fig. 4

a sketch which represents a selection measure 'fundamental frequency',

Fig. 5

a sketch that represents a selection measure 'energy',

Fig. 6

a sketch that represents a selection measure 'SCORE'.

A determination of language components from a speech signal, preferably a sufficiently long speech test of one Speaker's, is for a concatenative speech synthesis, so rearranging the found language modules to new ones semantic strings of language, of importance. The more precise individual sound sections from the speech signal "cut out", the higher the quality the synthetic language.

1 shows steps of a method for determining a representative of a language component of a language from a speech signal comprising sound sections. In a step 101, the sound sections of the speech signal are combined in a group for each speech module according to their affiliation to the speech modules. This can be summarized automatically and is described, for example, in [4]. An HMM (= hidden Markov model) is preferably trained on the speech signal. The speech signal can be any speech sample with an approximate length of one hour to three hours. After step 101 has been carried out, the sound sections are combined in groups, each group comprising at least one sound section which stands for a predetermined language component of the language.

There are usually several sound sections in each of these groups included, with one representative each for speech synthesis to be determined from each group. The single ones Sound sections in a group are not all the same, but are subject to a statistical distribution. In the following, the knowledge of a distribution is used to a suitable representative under the sound sections to find and select a group.

The sound sections are accordingly one predefined selection measure, whereby for each According to the phonetic section, a selection value results for each selection measure. The individual sound sections are preferably repeated different selection measures, it results for each selection measure has its own selection value (for each Phonetic section) (see step 102).

For each group, the frequencies are determined Selection values for all sound sections in this group determined (see step 103). This corresponds to one Application in a two-dimensional representation, the The abscissa values of the selection measure include and the ordinate describes their frequencies. For all sound sections of the For each group of selections, there is such a representation, this representation being a statistical distribution of the sound sections rated according to the selection measure shows.

In a next step 104, the determined Frequencies used to perform a membership function (for each of the representations described above). The membership function preferably provides an envelope about the statistically distributed frequencies of the Selection values. This step is also carried out for every selection measure for every group. A group includes how already described above, all sound sections that one Express the given language element. From the Membership function can be for any sound section Affiliation measure can be determined, which is a measure of the Usability of the respective sound section as a representative represents the group for the respective selection measure.

Accordingly, the sound section is represented as one in one Step 105 selected, its membership measure above of a predetermined threshold. Preferably, as described above, several selection measures used, so that there are several membership measures for each section of sound result which, combined multiplicatively, result in an overall measure. Accordingly, the section of the sound becomes the representative of the Group selected, the total dimension above a predetermined total threshold.

For illustration, FIG. 2 shows the relationship between the language SPR, which comprises language modules SBSi (i = 1,2, .., n) and the speech signal SSI, the sound sections LAi-j (j = 1,2, .., m ) contains, which are summarized in groups GRi.

Link 201 indicates that the Speech module SBS1 through the sound sections LA1-1, LA1-2, LA1-3, ..., LA1-m can be expressed. This Sound sections assigned to the speech module SBS1 are grouped together in a group GR1. The individual sound sections of group GR1 are from the Speech signal obtained and all describe the speech module SBS1. The individual are according to the speech signal Sound sections, based on different selection measures, each of different grades. So it's a goal a "usable" representative from the sound sections the group GR1 to determine the language module SBS1 of synthetic language.

Similarly, the same relationships apply to the link 202. Any language module SBSn can be provided by numerous (here 'p') sound sections in a group GR2 are summarized, expressed.

The following should refer to those already mentioned above Selection measures. There are several Options for such selection measures, here one Selection is presented individually or in combination with each other or in combination with others Selection measures can be used to create an advantageous one Determination of a representative from the group of Allow sound sections.

3 shows as a selection measure the length manipulation , that is a measure of the original duration of the sound section relative to the synthesized duration of the sound section. Deviations of up to a lower threshold ℓ _UG and an upper threshold ℓ _OG are considered to be unproblematic. The membership function Z , _{_syn drops} exponentially beyond these threshold values, that is to say less than the lower threshold value ℓ _UG or greater than the upper threshold value ℓ _OG . The membership function Z _{ℓ_syn is} determined by:

The deviation becomes relative by normalizing the mean length 1 _ to 1. The membership function Z _{ℓ_syn} is also standardized to 1. ZG denotes the degree of membership.

4 shows a fundamental frequency manipulation as a selection measure. The deviation of the fundamental frequency of the sound section from a target fundamental frequency (in the synthesized speech) should be minimal. The membership function Z _{f_syn} has the following form:

For the sake of clarity, the frequency f is also normalized to the average frequency fØ. The membership function Z _{f_syn} is also standardized to 1. An upper parameter of the frequency is denoted by f _OG and a lower parameter of the frequency by f _UG .

In Fig. 5 the energy of the sound section is shown as a selection measure. The relative deviation of this energy from an average value of the energy is the criterion of the membership function Z _{E_al} :

There are E _ the mean (expected value) of the energy E, E _UG a lower threshold of the energy, E _OG an upper threshold of the energy and σ _E the variance of the energy. The membership function Z _{E_al} is standardized to 1.

If instead of the energy the length ℓ of the sound section is used as a selection measure, analogous to FIG. 5, a membership function Z _{ℓ_al results} for evaluating the relative deviation of the length change of the sound section. If there is an upper threshold ℓ _OG , a lower threshold ℓ _UG and a variance in length σ _ℓ , the membership function Z _{ℓ_al applies} :

FIG. 6 shows a selection measure SCORE, which is a measure for the fitting of a sound section as a representative, ie a sound section to be selected is a typical, characteristically articulated sound section and thus 'fits' as a representative for the corresponding language element.

A membership function Z _S (s) is assumed to be linear between the sound section with the "best" (Z _S (s _max ) = 1) and the "worst" (Z _S (s _min ) = 1-s _G ) (see corresponding course of Z _S (s) in Fig. 6). This membership function Z _S (s) can be determined as follows:

Preferably, several of the listed membership functions are taken into account when assessing whether a sound section is suitable as a representative of the corresponding language module. To ensure that a representative is selected whose value does not lie below a predefined threshold for any membership function, the individual membership measures are AND-linked. This is done by multiplying the individual membership measures to a total measure. Taking into account the membership functions listed above, the following results:

The multiplication over all states in the membership functions Z _{E_al} and Z _{ℓ_al} means the individual states within an HMM which is used to describe the sound section. Depending on the modeling, HMMs can be used with different numbers of states, all of these states separately received for each phone in the section is _saturated by the membership function Z resultant overall dimensions.

[1] Nick Campbell, Alan W. Black: "Prosody and the Selection of Source Units for Concatenative Synthesis", in Progress in Speechsynthesis, ISBN 0-387-94701-9, Springer Verlag New York, 1997, S. 279-292

[2] Andrew J. Hunt, Alan W. Black: "Unit Selection in a concatenative speech synthesis system using a large speech data base", Proc. EUROSPEECH 1995, Madrid, S. 373-376.

[3] Alistair D. Conkie, Stephen Isard: "Optimal Coupling of Diphones", in Progress in Speechsynthesis, ISBN 0-387-94701-9, Springer Verlag New York, 1997, S. 293-304.

[4] R. E. Donovan, P. C. Woodland: "Improvements in an HMM-based speech synthesiser", Proc. ICASSP 1995, Michigan, S. 573-576.

[5] G. Ruske: "Automatische Spracherkennung: Methoden der Klassifikation u. Merkmalsextraktion", Oldenbourg Verlag, München 1988, S. 160-171.

[6] R. E. Donovan, P. C. Woodland: "Automatic Speech Synthesiser Parameter Estimation using HMMs", IEEE, Proc. ICASSP 1995, S. 640-643.

[7] Patent Abstracts of Japan, Bd. 17, Nr. 477, 1993, S. 1603.

The following publications have been cited in this document:

[1] Nick Campbell, Alan W. Black: "Prosody and the Selection of Source Units for Concatenative Synthesis", in Progress in Speech Synthesis, ISBN 0-387-94701-9, Springer Verlag New York, 1997, pp. 279-292

[2] Andrew J. Hunt, Alan W. Black: "Unit Selection in a concatenative speech synthesis system using a large speech data base", Proc. EUROSPEECH 1995, Madrid, pp. 373-376.

[3] Alistair D. Conkie, Stephen Isard: "Optimal Coupling of Diphones", in Progress in Speech Synthesis, ISBN 0-387-94701-9, Springer Verlag New York, 1997, pp. 293-304.

[4] RE Donovan, PC Woodland: "Improvements in an HMM-based speech synthesizer", Proc. ICASSP 1995, Michigan, pp. 573-576.

[5] G. Ruske: "Automatic Speech Recognition: Methods of Classification and Feature Extraction", Oldenbourg Verlag, Munich 1988, pp. 160-171.

[6] RE Donovan, PC Woodland: "Automatic Speech Synthesizer Parameter Estimation using HMMs", IEEE, Proc. ICASSP 1995, pp. 640-643.

[7] Patent Abstracts of Japan, Vol. 17, No. 477, 1993, p. 1603.

Claims (8)

1. Method for determining a representative of a predetermined speech unit of a language from a speech signal comprising phonetic segments,

   a) in which the phonetic segments of the speech signal are combined in each case in a group in accordance with membership of speech units of the language,

   b) in which selection values are determined from the speech signal for the phonetic segments of the respective group in accordance with a prescribed selection measure,

   c) in which frequencies of the selection values are determined for the group,

   d) in which the frequencies are used to determine a membership function which specifies a membership measure for the usefulness of the respective phonetic segment of the respective group, and

   e) in which that phonetic segment whose membership measure lies above a prescribed threshold value is determined as representative from the group of the phonetic segments for the selected speech unit.
2. Method according to Claim 1, in which at least one further selection measure is used to determine further selection values for the phonetic segments of the group and further frequencies are determined for the further selection values, and a further membership function with a corresponding further membership measure is determined for each further frequency.
3. Method according to Claim 2, in which each membership measure features multiplicatively in an overall measure, and the representative is determined from the group of phonetic segments whose overall measure lies above a prescribed total threshold value.
4. Method according to one of the preceding claims, in which the phonetic segments are phonemes, diphones, triphones, syllables, half-syllables, words of the language, or combinations of these.
5. Method according to one of the preceding claims, in which the phonetic segments are assigned to individual states of hidden Markor models.
6. Method according to one of the preceding claims, in which the selection measure is one of the variables listed below:

   a) energy of the respective phonetic segments;

   b) length of the respective phonetic segments;

   c) fundamental frequency of the respective phonetic segments;

   d) length manipulation of the respective phonetic segments;

   e) a statistical measure for matching the respective phonetic segments.
7. Method according to one of the preceding claims, in which speech is composed from the respresentatives obtained.
8. Method according to one of the preceding claims, in which the representative of the speech unit is determined as the phonetic segment whose membership measure has the highest value or, if a plurality of selection measures are considered, whose overall measure has the highest value.

Images