EP1058235A2 - Reproduction method for voice controlled systems with text based speech synthesis - Google Patents

Abstract

The invention specifies a simple reproduction method with improved pronunciation for voice-controlled systems with text-based speech synthesis even when the stored train of characters to be synthesized does not follow the general rules of speech reproduction. According to the invention, the method of "copying" the original spoken input text into the otherwise synthesized reproduction text, which is the current state of the art, is avoided, which will significantly increase the acceptance of the user of the voice-controlled system due to the process invented. More specifically, when there is actual spoken speech input that corresponds to a stored train of characters, the converted train of characters is compared to the speech input before reproduction of the train of characters described phonetically according to general rules and converted to a purely synthetic form. When the converted train of characters is found to deviate from the speech input by a value above a threshold value, at least one variation of the converted train of characters is created. This variation is then output instead of the converted train of characters as long as this variation deviates from the speech input by a value below the threshold value.

Description

Technical field

The invention is concerned with the improvement of voice-controlled systems text-based speech synthesis, especially with the improvement of synthetic Playback of saved but determined during pronunciation Strings subject to peculiarities.

State of the art

When it comes to operating technical devices, the language is becoming increasingly important Importance. This affects both the input of data and commands as well the output of messages. Systems where communication between User and machine in both directions with the help of acoustic signals in shape done by speech are called speech dialogue systems. That of the respective System-issued utterances can either be pre-recorded natural Language or according to the subject matter of the present invention can be generated synthetically. Arrangements are also known in which the respective utterances combinations of synthetic and previously recorded are natural language.

In order to better understand the invention, some general explanations and Speech synthesis definitions sent ahead.

The subject of speech synthesis is the mechanical transformation of the symbolic Representation of an utterance in an acoustic signal emitted by one human speaker recognized as sufficiently similar to human language becomes.

1) Ein Sprachsynthesesystem produziert, ausgehend von einem Text, gesprochene Sprache.

2) Ein Sprachsynthetisator produziert, ausgehend von gewissen Kontrollparametern gesprochene Sprache. Der Sprachsynthetisator stellt damit die letzte Stufe eines Sprachsynthesesystems dar.

In the field of speech synthesis, there are two different systems to distinguish:

1) A speech synthesis system produces spoken language based on a text.

2) A speech synthesizer produces spoken language based on certain control parameters. The speech synthesizer thus represents the last stage of a speech synthesis system.

A speech synthesis technique is a technique that involves building a speech synthesizer allowed. Examples of speech synthesis techniques are direct synthesis, the Synthesis using a model and simulation of the vocal tract.

In direct synthesis, either parts of the speech signal are started from stored signal pieces (e.g. one per phoneme) to the corresponding ones Words put together or the transfer function of the vocal tract, which at People used for speech generation by energy of a signal in simulated certain frequency ranges. For example, here Voiced sounds through a quasi-periodic excitation of a certain frequency represents.

The phoneme mentioned above is the smallest meaning-distinguishing, but itself not meaningful unity of language. Two words different Meaning that only differ by a phoneme (e.g. fish - table; forest - Wild), form a minimal pair. The number of phonemes in a language is proportional small (between 20 and 60). There are 45 phonemes in German, for example.

To take into account the characteristic transitions between the phonemes, Diphones are mostly used in direct synthesis. A can be simplified Diphon as the range from the invariable part of the first phoneme to the invariable Part of the following phoneme can be defined.

Phonemes or sequences of phonemes are created with the help of the International Phonetic alphabet (IPA) noted. The implementation of a text in a sequence of Phonetic Alphabet characters is called Phonetic Transcription designated.

In the synthesis using a model, a production model is formed, which mostly on minimizing the difference between a digitized human Speech signal (original signal) and a predicted signal based.

Another method consists in the simulation of the vocal tract, in which its Shape and position of the individual articulation organs (tongue, jaw, lips) is reproduced. To do this, a mathematical model of the flow conditions generated in such a defined vocal tract and the speech signal with the help of this model.

The following are additional terms and procedures related to the Speech synthesis will be used, briefly explained.

The phonemes or diphones used in direct synthesis must first can be obtained by segmentation from natural language. Here you can two approaches can be distinguished:

With implicit segmentation, only the information contained in the voice signal itself is used for segmentation.
The explicit segmentation, however, uses additional information, such as the number of phonemes contained in the utterance.

For segmentation, features must first be extracted from the speech signal, on the basis of which it is possible to distinguish between the segments.
These characteristics are then classified into classes.
Features for feature extraction include spectral analysis, filter bank analysis or the method of linear prediction.

For the classification, for example, Hidden Markov models, artificial ones neural networks or dynamic time warping (a method for line normalization) to be used.

The Hidden Markov Model (HMM) is a two-step stochastic process. He consists of a Markov chain with a mostly small number of states, the Probabilities or probability densities are assigned. Observable are the speech signals or those described by probability densities Parameter. The continuous state sequence itself remains hidden. Have HMMs because of their performance, robustness and good trainability in the Speech recognition widely used.

With the help of the so-called Viterbi algorithm, the correspondence of several HMMs can be determined.
In more recent approaches, self-organizing feature maps (Kohonen maps) are often used for classification. This special type of artificial neural network is able to simulate the processes taking place in the human brain.

A common approach is to classify voiced / unvoiced / silent - according to the different forms of stimulation in the generation of language in Vocal tract.

Regardless of which of the above-mentioned synthesis techniques is also used, the problem with text-based synthesis arrangements remains that, even if between the pronunciation of a text that is stored or saved String has a relatively large correlation, words in any language are available, from which the spelling is not readily based on their Pronunciation can be closed. It is particularly common for proper names not possible to specify general phonetic rules for pronunciation. Have so for example the two city names Itzehoe and Laboe have the same ending, although Itzehoe is pronounced with "oe" and Laboe with "ö". Are they lying respective words intended for synthetic reproduction as a string Before, applying a general rule results in the example above both city names are pronounced either consistently with "ö" or "oe", what in the case of the "ö version" for Itzehoe and in the case of the "oe version" for Laboe would be wrong in pronunciation. If you want to take these special features into account, is it is necessary that the appropriate words of that language are used to represent a have to undergo special treatment. But at the same time this means that no purely text-based entry of those intended for later playback Words more is possible.

Since the special treatment of certain words of a language is extremely complex, it has been used in speech-controlled arrangements to form the announcement, which an arrangement is to indicate, from a mix of spoken and synthesized language. For this purpose, the desired destination is recorded for a route finder, for example, which has special pronunciations in terms of pronunciation compared to the other words of the corresponding language and which is specified by a user in the case of voice-controlled arrangements, and copied into the corresponding destination announcement. This then means that in the destination announcement " In three kilometers you reach Itzehoe", only the part in italics was synthesized and the rest of the part "Itzehoe" was taken from the destination input of the user. The same conditions also occur when setting up mailboxes, in which the user must enter his name when setting up. There the corresponding announcement text, which is reproduced when a caller is connected to the mailbox, is also made from the synthesized part "You are connected to the mailbox from" and the original part - recorded when the mailbox was set up - to avoid the expense " educated.

Apart from the fact that composite announcements of the type described above You can leave a rather unprofessional impression by the integration of the original language can lead to bugging problems. In this The relationship is only to voice input in a noisy environment pointed out. The invention is therefore based on the object of a reproduction method for voice-controlled systems with text-based speech synthesis specify at which the disadvantages given in the prior art are eliminated become.

Presentation of the invention

This object is achieved with the features specified in claim 1. Advantageous further developments of the invention are claims 2 to 9 removable.

Is according to claim 1 in the presence of an actually spoken and with a stored character string corresponding voice input in front of a actual reproduction of those described phonetically according to general rules and the character string converted into a purely synthetic form Character string compared with the spoken speech input and takes place actual rendering of the converted string only when the comparison this string with the actual spoken voice input one below shows a threshold deviation, the use of the original language superfluous when playing according to the state of the art. This is the case even if the spoken word differs from that word corresponding, converted string deviates significantly. Here must only ensure that at least of the converted string a variant is formed and that the variant formed, provided that this is in a Comparison with the original voice input one below the threshold Shows deviation instead of the originally changed character string is issued.

If the method is carried out according to claim 2, the computation and storage effort relatively low. This is due to the fact that there is always only one variant must be formed and examined.

Are formed according to claim 3 at least two variants and is made in the variants produced, the one that is the least selected Deviations from the original speech input is in contrast to the procedure according to claim 2 always one of the original voice input synthetic reproduction possible.

The procedure is simplified if, according to claim 4 Segmentation of the speech input and the converted character string or the variants formed therefrom. This segmentation allows segments in where no differences or differences below the threshold are found, of exclude further treatment.

If an identical segmentation approach is used according to claim 5, the Comparison particularly easy, since a direct assignment of the respective segments given is.

As shown in claim 6, different segmentation approaches can also be used become. This is especially true when looking at the original voice input Advantages, because the segmentation contains those contained in the speech signal information that can only be determined in a very complex step is used must, while the segmentation of strings very simply the well-known Number of phonemes contained in the utterance can be used.

The procedure becomes very efficient if, according to claim 8 Segments are eliminated in which there is a high degree of agreement exists, and only the segment of the string that corresponds to the corresponding Segment of the original voice input one above the threshold lying deviation shows, is varied by that in the segment phoneme present in the character string is replaced by a replacement phoneme.

A particularly simple procedure is achieved if according to claim 9 at least one replacement phoneme similar to this phoneme is linked to each phoneme or stored in a list.

The computing work is further reduced if according to claim 10 at a variant of a character string that is determined to be reproducible, the special features, associated with rendering the string, along with the String can be saved. In this case there is the special pronunciation the respective character string if you use it later without much effort available in the memory.

Brief presentation of the figures

Fig. 1

einen schematischen Ablauf gemäß der Erfindung

Fig. 2

einen Vergleich von segmentierten Äußerungen

Show it:

Fig. 1

a schematic flow according to the invention

Fig. 2

a comparison of segmented utterances

Ways of Carrying Out the Invention

The invention will now be explained in more detail with reference to the two figures.

In order to better demonstrate the effects of the invention, one of speech-controlled system with text-based speech synthesis. Such Systems are implemented, for example, in route finders or mailbox arrangements that because of the high prevalence of such systems, their representation is based on the Can limit things that are imperative for the implementation of the invention are.

All these systems have a memory in which a plurality of Strings are stored. These strings can be one For example, route finders are street or place names. In a mailbox application can do this as in a phone book the names of subscribers his. So that the memory easily with the appropriate information loaded or the stored information can be easily updated, the respective strings are available as text.

In Fig. 1, the schematic flow according to the inventive method shows, such a memory is designated 10. This memory 10, which for the Representation of the invention, which should contain German city names, belongs to one Route finder 11. This route finder 11 also includes an arrangement 12 with which natural voice inputs are recorded and temporarily saved can. In the present case, this is realized in such a way that the respective voice input by one Microphone 13 is detected and stored in a voice memory 14. Now becomes a The user of the route finder 11 is prompted to enter his destination, the each destination spoken by the user z. B. "Bochum" or "Itzehoe" from Microphone 13 detected and passed on to the voice memory 14. Because the route finder 11 has either been informed of his current location or has still been given it knows, he is first based on the desired destination and the current one Determine the appropriate route to the destination. If the route finder 11 not only show the corresponding route graphically, but spoken ones Deliver announcement, the textual strings of the respective announcement Described phonetically according to general rules and then for speech converted into a purely synthetic form. In that shown in Fig. 1 Exemplary embodiment is the phonetic description of the stored character strings in the converter 15 and the synthesizing arranged in the following Speech synthesizer arrangement 16.

As long as those called up via voice input and intended for playback Strings related to their respective pronunciation following the rules of phonetic Transcription of the language in which the dialogue between the user and the Routefinder 11 should follow, the respective character string, if it has passed through the converter 15 and the speech synthesizer assembly 16 as a word corresponding to the phonetic conditions of the respective language a speaker 17 to the environment and from this as such be understood. For a route finder 11 of the type described above, this means that, for example, the one consisting of a plurality of character strings, for Playback specific playback text initiated by voice input "An der turn right at the next intersection! "without any problems, i.e. according to the phonetic Conditions of the language output via the loudspeaker 17 and also understood can be because this information is not peculiar to the playback subject to.

However, for example, the user should be given the option after entering the destination will be granted to check the correctness of his destination entry, the Play route finder 11 after entering the destination approximately the following sentence: "You have Berlin chosen as the destination. If this does not meet your expectations, give it now set a new goal. "Even though this information follows general rules can be reproduced correctly, problems arise when that The goal should not be Berlin, but Laboe. If the string that the Textual representation of the destination Laboe in the converter 15 according to general rules written phonetically and then in the speech synthesizer 16 Output through speaker 17 like the rest of the information above in brought a synthetic form, that would be given over the speaker 17 Correct result only if, according to general rules, the ending "oe" is generally reproduced as "ö". The correctness of the rendering of the destination In the latter case, however, Laboe inevitably leads to incorrect reproduction, if the user chooses Itzehoe as the destination, because of the basic Pronunciation of the "oe" as "ö" would make the destination phonetically incorrect as "Itzehö" reproduced.

To avoid this, is between the speech synthesizer 16 and the Loudspeaker 17 arranged a comparison arrangement 18. This comparison arrangement 18 become the destination actually spoken by the user and the the character string corresponding to the destination after the converter 15 and the Speech synthesizer 16 has passed, fed and then compared. If the synthesized string shows a high - above a threshold lying - coincidence with the originally spoken destination, is used for the Playback uses the synthesized string. Can this match are not ascertained, a variant becomes in the speech synthesis arrangement 16 of the original string and in the comparator 18 again Comparison between the originally spoken destination and the variant formed carried out.

Is the route finder 11 designed such that as soon as a character string or a Variant has the required agreement with the original, whose Playback via the loudspeaker 17 takes place, further variant formations stopped immediately. The route finder 11 can also be modified such that a A plurality of variants are formed and then one from the variants Variant is selected that most closely matches the original shows.

How the comparison is carried out in comparator 18 is related to 2a and b shown in more detail. 2a there is a speech signal in the time domain of actually represented by Itzehoe spoken by a user. Fig. 2b also shows a speech signal in the time domain of the word Itzehoe, but in in Fig. 2b the word Itzehoe from a corresponding present String first in the converter 15 phonetically according to general rules described and then subsequently in the speech synthesizer 16 in a synthetic form. The illustration according to FIG. 2b is clear it can be seen that, when applying the general rules, the ending "oe" des Word Itzehoe is reproduced as "ö". However, this incorrect playback exclude the spoken and the synthesized form in one Comparator 18 compared with each other.

To simplify this comparison, both the spoken and the synthesized form divided into segments 19, 20 and then the comparison between corresponding segments carried out 19/20. In the in Fig. 2a and b shown embodiment shows that only in the last two Segments 19.6, 20.6 there is a large deviation during the comparison of the remaining segment pairs 19.1 / 20.1, 19.2 / 20.2 ... 19.5 / 20.5 a relatively large one Show agreement. Because of the large deviation in the pair of segments 19.6 / 20.6 the phonetic description in segment 20.6 is based on one in one Memory 21 (Fig. 1) stored list, which better fitting or similar Contains phonemes, changes. Since here the phoneme in question is "ö" and the list with similar phonemes that provide replacement phonemes "o" and "oh", the phoneme "ö" replaced by the replacement phoneme "o". For this, the deposited Character string in a converter 15 '(Fig. 1) again described phonetically in the Speech synthesizing arrangement 16 brought into a synthetic form and again with compared to the actually spoken destination input in the comparator 18.

For the sake of completeness, it should be pointed out that the converter 15 'in another - not shown - embodiment of the converter 15th can be formed.

It turns out that the correspondingly modified character string, which in the In connection with this application is also referred to as a variant, none matches the spoken word above a threshold the procedure is repeated with another replacement phoneme. Is the The degree of correspondence then above the threshold, it becomes corresponding synthesized word output through the speaker 17.

The process sequence can also be modified. It is found that a Deviation between the spoken and the original synthetic form is given, and there are a plurality of replacement phonemes in the memory 21 stored list, a plurality of variants can also be formed at the same time and be compared to the actual spoken word. Is played then the variant that most closely matches what is spoken Word shows.

Should be avoided that the multiple use of words that the can trigger the above procedure, always the correct - synthetic - pronunciation can be elaborately determined if, for example, the correct synthetic Pronunciation for example of the word Itzehoe has been identified, the corresponding Modification with reference to the string Itzehoe can be saved. This means that if the Itzehoe string is requested again at the same time for the correct pronunciation of this word from the phonetic description General rules deviating special features are taken into account, so that the Comparative step in the comparator 18 can be omitted. To make this modification visible 1, an additional memory 22 has been indicated by dashed lines in which the information indicating modifications of stored character strings be filed.

Just for the sake of completeness, it should also be noted that the additional memory 22 not just on the inclusion of information on the correct pronunciation of stored strings is limited. For example, a comparison in Comparator 18 that between the spoken and the synthesized form of a Word no deviation or below a threshold, can be stored in the additional memory 22 for this word, which in the future use of this word an elaborate comparison in Comparator 18 excludes.

It can also be seen from FIGS. 2a and b that the segments 19 according to FIGS. 2a and the segments 20 according to FIG. 2b do not have the same format. For example the segment 20.1 compared to segment 19.1 a larger width, while the segment 20.2 compared to the corresponding segment 19.2. essential is narrower. This is due to the fact that the "speech length" of the different phonemes to be compared can be of different lengths. However, since such long speaking times cannot be excluded can, the comparison arrangement 18 is designed so that different lengths Speaking times of a phoneme do not yet indicate a mutual deviation.

For the sake of completeness, it should be noted that when using different segmentation methods for the spoken and the synthesized Format also a different number of segments 19, 20 can be calculated can. If this occurs, then a certain segment 19, 20 should not only have one correspond to segment 19, 20, but also with the Predecessor and successor of the corresponding segment 19, 20. So it is also possible to replace one phoneme with two other phonemes. This procedure is also possible in the opposite direction. There is no match for a segment 19, 20, this can be excluded, or by two better fitting ones be replaced.

Claims (10)

Playback method for voice-controlled systems with text-based speech synthesis,
characterized, that if a speech input that is actually spoken and corresponds to a stored character string is present, before the reproduction of the character string described phonetically and according to general rules and converted into a purely synthetic form, the converted character string is compared with the voice input, that when a deviation of the converted character string from the speech input lying above a threshold is determined, at least one variant of the converted character string is formed and that one of the variants formed, provided that this has a deviation below the threshold when compared with the voice input, is output instead of the converted character string. Playback method according to claim 1,
characterized, that only one variant is formed in step two and that, if in step three a comparison of the variant with the speech input always shows a deviation above the threshold, step two is carried out at least once more to form a new variant. Playback method according to claim 1,
characterized, that at least two variants are formed in step two and that in the presence of variants that each have a deviation below the threshold compared to the voice input, the variant that has the least deviation from the voice input is always reproduced. Method according to one of claims 1 to 3,
characterized, that before comparing the speech input with the converted character string or the variant (s) formed from it, the voice input and the converted character string or the variant (s) formed are segmented. Playback method according to claim 4,
characterized, that the same segmentation approach is used both for segmenting the speech input and the converted character string or the variant (s) derived from it. Playback method according to claim 4,
characterized, that a different segmentation approach is used for both segmenting the speech input and the converted character string or the variant (s) derived from it. Playback method according to claim 4,
characterized, that an explicit segmentation approach is used to segment the converted character string or the variant (s) derived therefrom and an implicit segmentation approach to segment the speech input. Playback method according to one of Claims 4 to 7,
characterized by that the converted character string present in segmented form and the segmented speech input in the corresponding segments are examined for common features and that if there is a deviation lying above a threshold value in two corresponding segments, the phoneme present in the segment of the converted character string is replaced by a replacement phoneme. Playback method according to claim 8,
characterized, that at least one replacement phoneme similar to this phoneme is associated with each phoneme. Playback method according to one of Claims 1 to 9,
characterized, that as soon as a variant of a character string is determined to be reproducible, the special features associated with the rendering of the character string are stored in connection with the character string.

Images