EP0058130B1 - Method for speech synthesizing with unlimited vocabulary, and arrangement for realizing the same - Google Patents

Abstract

1. A method for the synthesis of speech with an unlimited vocabulary in the time domain from sound elements which are obtained from natural speech samples and are coded with low redundancy in digital form, stored and also reduced in length, in each case to the significant area of the relevant time signal typical of the sound, and in number, by utilizing related sounds which are mutually transformable into each other, having regard to the necessary storage space requirement, these sound elements being linked, with respect to the form, number and sequence required, into digital signal sequences on the basis of input commands and of predetermined rules of linkage for the purposes of speech synthesis, these signal sequences being used to generate, by means of digital/analog conversion and controllable amplification, sound waves which can be perceived as speech, characterized in : providing a total of about 100 sound elements, that is to say - about 50 elements for transitions sounds with an average of 240 samples each for an output frequency of 8 kHz, and - about 40 elements for phonemes with an average of 500 samples for unvoiced and 140 samples for voiced phonemes each and an output frequency of 8 kHz, and enabling the pitch to be varied for reproduction, in the case of the elements for the voiced transition sounds and phonemes, by omitting or using at least once, as a result of appropriate input commands, those samples and/or values which are preset as suitable by means of marker words at discrete positions in the time signal, depending on requirement, when the digital signal sequences are being formed.

Description

Process for the synthesis of speech with unlimited vocabulary and circuitry for carrying out the process.

The invention relates to a method for the synthesis of speech with unlimited vocabulary in the time domain from sound elements that are obtained from natural speech samples and coded, stored in digital form, with little redundancy, and also with regard to the required storage space length in each case to the significant range of relevant sound-typical time signal and the number are reduced by utilizing mutually convertible related sounds, for speech synthesis these sound elements are chained in the required form, number and order to form digital signal sequences based on input commands and predetermined linking rules, from which by means of digital-analog Conversion and controllable amplification as speech-perceptible sound waves are generated, as well as on a switching arrangement for performing the method.

Speech synthesis is understood to mean the conversion of a text present as a symbol sequence into the equivalent acoustic signal by means of technical equipment. It is of fundamental importance that between the input of the symbol sequence in the equipment and the output of the equivalent acoustic signal, all processes take place immediately, without the interposition of human mind powers. The precisely determined individual technical measures follow the planned use of predictable and controllable natural forces.

The evaluation criteria for synthetic language are intelligibility and naturalness. The standards for this are, albeit z. B. detectable in terms of intelligibility according to objective criteria, subjective nature. Nevertheless, there are circumstances that everyone can immediately use for the assessment. These are the course of the basic pitch (pitch frequency), the speaking rhythm and the course of the intensity. The individual sounds merge into one another in the course of the natural language signal. They are characterized by several sound generation frequencies (formants). These sound generation frequencies are independent of the fundamental pitch, i.e. regardless of the speech height. These facts affect more or less both the intelligibility and the naturalness. While the intelligibility of known speech synthesis systems has so far been in the foreground, efforts have recently been aimed more and more towards improvements in naturalness after sufficient intelligibility has been achieved. There are minor difficulties with the dynamics. The relative volume can be varied with controllable amplifiers. The duration of the sound, and thus the rhythm of speaking, can also be changed with relatively simple means by dynamically controlling the number of repetitions of the individual sound elements. However, mastering the melody is problematic, since the length of the fundamental speech frequency periods is fixed for the individual sounds and a simple, proportional extension or shortening of fundamental speech frequency periods means a corresponding shift in the formant frequency spectrum, i. H. leads to completely unnatural sounds.

Comprehensibility and naturalness of synthetic language also depend on the performance for which the system in question is designed. Of course, a system with a limited vocabulary can guarantee excellent language quality. Complete words or even longer phrases, perhaps also given by a trained speaker, can be saved while maintaining the natural melody or rhythm and played back on demand. On the other hand, if the goal of a speech synthesis system is to generate an unlimited vocabulary, smaller synthesis modules, e.g. be used on sounds. In any case, sentence and word dynamics as well as the melody are lost at first and have to be regenerated during the synthesis. The extent to which this is successful is essential for the naturalness of synthetic language.

ermöglicht eine erste Abschätzung des erforderlichen Aufwandes für die Realisierung: Eine Wortsynthese, sowohl im Zeitbereich als auch im Parameterbereich, benötigt mit wachsendem Umfang des auszugebenden Vokabulars auch ein wachsendes Speichervolumen. Derartige Systeme sind also mit vernünftigem Aufwand nur für Systeme mit begrenztem Wortschatz geeignet. Auf der Lautsynthese beruhende Systeme ermöglichen die Ausgabe eines unbeschränkten Vokabulars und erfordern unterschiedlichen Aufwand, der in der folgenden Tabelle grob angedeutet ist.

Here, the technical possibilities and the economic aspects play a crucial role. A classification of the synthesis systems or their subdivision according to the synthesis principle

enables a first estimate of the effort required for the implementation: word synthesis, both in the time domain and in the parameter domain, requires a growing storage volume as the volume of the vocabulary to be output increases. Such systems are therefore only suitable for systems with limited vocabulary with reasonable effort. Systems based on sound synthesis allow the output of an unlimited vocabulary and require different efforts, which are roughly indicated in the following table.

The various types of speech synthesis systems are dealt with in large numbers in the technical-scientific and patent literature. For example, from DE-OS 3006339 a method and a device for speech synthesis is known, wherein for the purpose of miniaturization an information compression technique is to be used, which can be stored in a single integrated LSI circuit with minimal loss of intelligibility and naturalness. Chip is possible. The phonemes (individual sounds) stored as synthesis building blocks are to be subjected to a change or regulation in their shape retrieved from the memory with regard to an adaptation of the pitch interval, the amplitudes and the time axis in order to approximate the quality of natural language again. The data compression technique used, which is explained in more detail using an example, means that for one word (example: «nana») one sequence fewer (in the example: five) phonemes must be saved. These facts, which are known per se, are described in detail in this prior publication. However, there is no indication as to whether there are possibilities to synthesize an unlimited vocabulary and to influence melody and rhythm as desired.

The speech synthesis system known from DE-OS 2 016 572 takes into account the problems at the transitions between successive phonemes in particular with regard to intelligibility. Since the formant frequencies - taking into account the three main formants is sufficient - can increase, decrease or remain the same at the transitions, there are nine versions for each phoneme to be stored purely mathematically. In order not to have to increase the storage capacity by practically a further power of ten, the solution in this known prior art aims to get by with a stored version and to modify this representation according to the requirements during the synthesis process. In addition, only the significant range of the individual sounds is saved, e.g. with a / s / -loud only 10% of the total duration of the sound and can be reproduced exactly enough and understandably by repeating ten times. In order to avoid abrupt transitions between two consecutive phonemes, the saved sections should start with an oscillation zero crossing. For voiced phonemes, the suitability at the transition to other phonemes must also be selected in a special way - a subjective test. With this compromise, abrupt transitions can be avoided or at least reduced to a small extent, but on the other hand, completely smooth transitions must be avoided.

The task known from DE-OS 2306816 is based on the task in the preparation of phonetic segments to create a comprehensive range of pitch periods for the synthesized sounds, which should benefit the improvement of naturalness and intelligibility. The solution given is to pick out sound waveforms from natural language for voiced sounds with defined periodicity of each pitch length and to add a waveform to each such waveform at the end area that was obtained by a rough calculation for the waveform of the respective sound. Sound waveforms of unvoiced sounds and the transitions between consonants and vowels that have an undefined periodicity should be divided into fixed lengths. The sound waveforms obtained in this way then represent the synthesis building blocks. However, changing the duration of a pitch period not only results in a corresponding change in pitch, but also - as already mentioned above and also explained in more detail below - also results in a sound shift or contamination .

In IEEE Spectrum, Volume 16, No. 8, August 1979, Pages 22, 23 and Electronics, Volume 26, Issue 9, September 1977, Pages 44 to 48, reference is made to the known “VOTRAX” system. This is a speech synthesis based on the formant vocoder principle (vocoder = voice coder), for which a few characteristic parameters are determined from all the sounds required in the analysis phase, in particular the formants, the duration of the sound and the like. The stored parameters are used in the synthesis to control a vocal tract simulated by electronic means. As already mentioned above, this known system requires little storage space for the parameter data, but considerable effort is required to regenerate the sounds.

• - etwa 50 Elemente für Übergangslaute mit je durchschnittlich 240 Abtastwerten für 8 kHz Ausgabefrequenz

und

• - etwa 40 Elemente für Einzellaute mit je durchschnittlich 500 Abtastwerten bei stimmlosen und 140 Abtastwerten bei stimmhaften Einzellauten und 8 kHz Ausgabefrequenz,

und dass die Tonhöhe für die Wiedergabe bei den Elementen für die stimmhaften Übergangs- und Einzellaute veränderbar ist, indem solche Abtastwerte, die an diskreten Stellen des Zeitsignals mittels Markierwörtern als geeignet vorgegeben sind, je nach Bedarf aufgrund entsprechender Eingangsbefehle bei der Bildung der digitalen Signalfolgen ausgelassen bzw. mindestens einmal verwendet werden.The invention is based on a prior art as known from DE-OS 2 531 006 and taken into account in the preamble of claim 1. The subsequent reduction, which is easy to understand, already led to the storage volume required for storing the voice data, uncoded, in the time range of only approx. 1 Mbit, corresponding to 125 kByte. The aim of the invention is now to further reduce the storage space requirement and, in particular with regard to the naturalness of the speech to be synthesized, to provide simple control measures for word and sentence melody variation, with which the advantages inherent in speech synthesis in the time domain with regard to comprehensibility, the synthesis algorithm and the rate of synthesis becomes significantly more important than the systems working in the parameter area. According to the invention, this is achieved in that a total of approximately 100 sound elements are provided, namely:

• - about 50 elements for transitional sounds each an average of 240 samples for 8 kHz output frequency

and

• - about 40 elements for individual sounds, each with an average of 500 samples for voiceless and 140 samples for voiced individual sounds and 8 kHz output frequency,

and that the pitch for the reproduction of the elements for the voiced transitional and individual sounds can be changed by omitting such sample values, which are specified as suitable at discrete points of the time signal by means of marker words, as required on the basis of corresponding input commands when forming the digital signal sequences or be used at least once.

Without wishing to reduce the meaning of the details given in the reduction of the speech data, the measures for the melody variation are first explained in more detail below. What is essential for this is the fact that changes in the melody of speech are accounted for by the voiced parts and that voiced sounds have a high periodicity. The significant areas to be stored thus require only relatively few true samples, in the order of 80 true samples per voiced individual sound. Within these significant areas, which represent a pitch period and contain the typical frequency mix of the formants, there are several discrete places where the formant frequency mix shows little or no changes in the time signal curve. The key finding for the invention now lies in deliberately providing possible changes at precisely these “insensitive” discrete points. This means that the pitch period can be changed, lengthened or shortened, and thus the fundamental pitch can be lowered or raised accordingly, if samples are used or omitted at these discrete locations without the sound character thereby changing. To locate these discrete locations, approximately 30 within such a significant area, special “samples” are used, the marker words, which allow these locations to be found at any time. The marker words themselves are omitted when the elements are linked to form the digital signal sequences. Accordingly, 60 samples, e.g. B. the one adjacent to a marker word, depending on whether they are used or not, a practically continuous variation of the pitch, so very many melody lines.

tert wird -z.B. um 90 wahre, d.h. aus natürlicher Sprache gewonnene, und um 30 berechnete Abtastwerte handeln. Wird - bei Tonhöhenvariation in nur einer Richtung - die tiefste Tonlage gewünscht, sind alle 120 Abtastwerte zu benutzen. Zur Erhöhung der Tonlage können zunächst berechnete Abtastwerte ausgelassen werden. Eine weitere Erhöhung ist durch Auslassen von maximal 30 wahren Abtastwerten an den unempfindlichen Stellen möglich. Hierdurch kann die Tonlage insgesamt innerhalb einer Oktave verändert werden.This should be explained in more detail using the table below in comparison with some of the intervals used in acoustics. An example is a saved element for a voiced individual sound with 120 samples, plus 30 markings for places where pitch changes can be made. The 120 samples can - as explained in more detail below in connection with particularly preferred embodiments of the invention -

For example, tert will deal with 90 true samples, ie those obtained from natural language, and 30 calculated samples. If the lowest pitch is desired - with pitch variation in only one direction - all 120 samples must be used. In order to increase the pitch, calculated sample values can first be omitted. A further increase is possible by leaving out a maximum of 30 true samples at the insensitive points. As a result, the pitch can be changed within an octave.

But it is also conceivable that even lower pitches are desired. This can be achieved in that some, at most all, of the calculated sample values are output multiple times within a period.

The table contains the two options mentioned (cf. also FIG. 9 and associated description).

• 

zueinander. Aus der Tabelle ist zu entnehmen, dass beim gewählten Beispiel in der Nähe der vorgegebenen normalen Tonlage für einen Halbtonschritt eine Änderung um 5 bzw. 7 Abtastwerte pro Periode benötigt wird, d. h. schon Tonlagenänderungen um weniger als Halbtonschritte möglich sind. Übrigens können besonders geeignete Abtastwerte bereits mehrmals zur Toniagenänderung benutzt werden, bevor auf weniger gut geeignete zurückgegriffen wird. Hierdurch lassen sich auch die Sprachgrundfrequenzverläufe an den Übergängen zu den folgenden Lauten kontinuierlich gestalten, also Stossstellen vermeiden.With the chromatic scale, the number of oscillations between two adjacent semitones is in relation

• 

to each other. From the table it can be seen that in the selected example, a change of 5 or 7 samples per period is required for a semitone step in the vicinity of the predetermined normal pitch, ie changes in pitch by less than semitone steps are possible. Incidentally, particularly suitable samples can already be used several times to change the tone before using less suitable ones. As a result, the fundamental speech frequency curves at the transitions to the following sounds can be designed continuously, thus avoiding joints.

This is also one of the reasons why only about 100 sound elements are required as synthesis building blocks. When preparing the sound elements, i.e. in the analysis phase, the natural speech samples from which the sound elements to be used are obtained are to be examined anyway, for example to determine the «insensitive» points mentioned above. These speech samples can be modified mathematically, especially in the case of transitional noises, discontinuities in the formant courses can be eliminated.

The exploitation of sound transformations, i.e. mutual transferability of related sounds has already been the subject of the prior art known from DE-OS 2 531 006, from which the invention is based. There the reduction led e.g. for the consonants from 22 to 8. Furthermore, a number of exceptions, about 150 transitions, a pitch period of voiced sounds as well as a section from the middle part of the unvoiced sounds and finally the beginning of the time function for explosive sounds had to be saved. In the invention there is a considerable reduction due to the following measures: transitions - with the exception of combinations of plosives - can be inverted in time; by lengthening or shortening the length of time, vowel conversions take place; by shortening the length of time, there are also consonant conversions. The required sound elements are made up of almost 60 elements for transitional sounds, 27 elements for voiced individual sounds and 13 elements for unvoiced individual sounds. Further details follow in connection with the description of the figures.

Particularly preferred embodiments of the invention consist in the fact that the digitally stored elements for the voiced sounds - with an average of 140 samples for individual sounds or 240 for transition sounds - contain additional samples for the purpose of pitch variation. In the worst case, which is then no longer covered by this embodiment, this measure leads to a slight increase by approximately 1000 bytes of the required storage space volume, but enables further variations in the melody courses.

In close connection with this, it is also advantageous if an additional sample value has an interpolated value lying between the adjacent true sample values. In this way, any discontinuities that would occur between the true samples that are definitely needed and used can be reduced or avoided.

As already mentioned above, "unemp _f ind Liche" sites are preferred in the timings for the measures to Melodivariation, that mark words are to be provided preferably at points of low slope of the time signal. An associated error signal has very small deflections at such locations and thus allows the desired discrete locations to be determined, localized and marked in a simple manner.

Sometimes, especially with large, desired pitch fluctuations, it may be necessary to take full advantage of the possible range of samples suitable for omissions or use. However, the cases are more frequent in which only some of the predefined sample values available are required. For this reason, it is advantageous if marking words at points with a small slope of the time signal are given a higher priority for pitch variation than those at points with a larger slope. This means that such changes always take place at the least sensitive areas, but the more sensitive areas may also be used.

Although it is also possible to manage the associated addresses separately from the stored sound element for the sample values which are suitable for pitch variation, the solution with the marker words is preferred in the embodiments of the invention. A marker word and a true or additional sample value digital Mu of the same stock. With regard to a clear distinction between the marker word and the sample value, however, marker words are to be reserved for digital words which do not occur in the sample values.

For reasons of different priorities alone, a single pattern for marker words is not sufficient. Since a software identification of the patterns does not require any special systematic for the distribution of the digital patterns, it is easily possible to use the patterns with the highest number of digits for marking words, e.g. with 8-bit words. reserve patterns 246, 247, ... 255. These patterns can be omitted in a particularly advantageous manner when digitizing the sampled values because a limitation at the upper end leads to hardly noticeable restrictions.

It is of particular importance for embodiments of the invention to be able to determine the shape of the sound elements required for the concatenation of the next word following the pauses on the basis of the input commands. This avoids discontinuities in the output of the individual words. The duration for determining the shape of the required synthesis building blocks, even for very long words, is in the range of a few milliseconds. Determining the shape is to be understood here: searching for the relevant sound element, possibly inverting in time, lengthening or shortening the duration of the sound and specifying the number of repetitions of the stored sound element.

A further essential advantage of the invention is that sequences of conventional characters entered via an alphanumeric keyboard can be automatically transcribed into a sequence of phonetic characters suitable as input commands in a method step preceding the actual synthesis process. As a result, even inexperienced or untrained users will find it much easier to use, or even opened up. Of course, there is also the option of entering phonetic characters or the appropriate input commands directly.

However, additional storage volume is required for the transcription. It is surprising that only about a third of the storage space that is to be provided for the synthesis is required for this, i.e. about a quarter of the total storage space for synthesis and transcription, if the transcription is carried out in the following way: first, lexically recorded exceptions and foreign words are processed; Otherwise, the vocabulary is subjected to a prefix processing, taking exceptions into account, an ending split-off and a suffix processing, also taking exceptions into account, and the transcription of the stems is carried out according to rules stored in a catalog. These or similar measures are familiar to linguists per se.

A circuit arrangement for carrying out the method according to the invention can be constructed with a microprocessor, to which read-only memories with a total storage capacity of 32 kbytes and a working memory for 1 kbyte are to be connected, and also has a decompanding digital-to-analog converter and a -volume-controllable low-frequency amplifier and a loudspeaker as an electro-acoustic transducer device. Such circuit elements and components are common on the market. The concept also enables extensive integration.

Decomparing before the digital-to-analog conversion naturally means that the stored data have previously been subjected to coding which reduces the data rate. The logarithmic PCM and the adaptive delta PCM are common and increasingly reducing methods in the order given. Relevant components are known from common voice transmission systems and can also be used without problems in embodiments of the invention.

• 1,5 kByte für das Transkriptionsprogramm,
• 6 kByte für die Transkriptionsgrammatik,
• 1,5 kByte für das Syntheseprogramm,
• 1 kByte für die Synthesematrix und
• 22 kByte für die Lautelemente

erfolgen kann.With regard to the complexity of circuit arrangements, the memories, more precisely their size, are still important. It is therefore important for cost estimates that, in the case of a circuit arrangement for carrying out the method according to the invention, the division of the capacity of read-only memories into:

• 1.5 kByte for the transcription program,
• 6 kByte for the transcription grammar,
• 1.5 kByte for the synthesis program,
• 1 kbyte for the synthesis matrix and
• 22 kByte for the sound elements

can be done.

Finally, for the various uses of embodiments of the invention, it is important that the input of the data, i.e. the writing or phonetic symbol sequences, as well as the output of the acoustic signals can take place both directly on the device and at remote locations. For this purpose, e.g. at the entrance a V24 interface or a low-frequency socket can be provided at the output.

The possible uses for such a speech synthesis system are extremely varied due to the possibility of generating an unlimited vocabulary. Examples include: telephone information systems; acoustic replacement or support for confusing display boards, in particular flight or timetable plans; Replacement or supplementation where the attention of people is unduly claimed through constant observation of individual numerical or text displays or warning systems, for example in aircraft on-board systems; Keypad telephones as input keyboard and telephone handset as output in data processing systems, for example for information on continuously changing data, such as inventory levels, account balances, stock exchange prices, medical diagnoses or continuous monitoring of body functions of patients in the hospital or at home; Orders of goods according to catalog numbers, theater or Concert tickets; Placing and accepting orders, redistribution, etc. the like; Remote transmission of process data; Home control systems; Language teaching; Computer aided instruction; Traffic management; Library inquiries and information; Lexicon information service, help for the disabled - speech and visual impaired - and much more.

• Fig.1: ein Blockschaltbild für ein Sprachsynthesegerät mit Transkriptionseinheit,
• Fig. 2: ein Blockschaltbild eines Sprachsynthesegerätes mit Transkriptionseinheit, auf Mikroprozessorbasis;
• Fig. 3: eine Darstellung der Lage der drei ersten Formanten für verschiedene Laute;
• Fig. 4: eine Darstellung von Formantsprüngen an den Übergängen zwischen drei Einzellauten;
• Fig. 5: eine Darstellung für die Reduktionsmöglichkeit der Länge von Elementen;
• Fig. 6a: ein Beispiel für zeitliche Invertierung von Übergangslauten;
• Fig. 6b: die Möglichkeiten für Vokalumwandlungen;
• Fig. 6c: die Möglichkeiten für Konsonantenumwandlungen;
• Fig. 7: ein Beispiel für die Veränderung des Höreindrucks durch Verschieben des Anfangspunktes;
• Fig. 8a,b,c: ein Beispiel für die rechnerische Modifizierung eines stimmhaften Einzellautes zur Variation der Tonhöhe;
• Fig. 9: ein - auszugsweises - Beispiel für die Anordnung von wahren, auslassbaren und zusätzlichen Abtastwerten sowie von Markierwörtern in einem gespeicherten Element eines stimmhaften Einzellautes;
• Fig. 10: eine Darstellung der Aufteilung und des Inhaltes des Lautelemente-Speichers,-
• Fig.11: eine Darstellung des Ablaufs einer Transkription und
• Fig. 12: eine Darstellung eines Synthesebeispiels (monoton).

Details of embodiments of the invention are shown schematically in the drawings. Show:

• 1: a block diagram for a speech synthesis device with a transcription unit,
• 2 shows a block diagram of a speech synthesis device with a transcription unit, based on a microprocessor;
• 3 shows a representation of the position of the first three formants for different sounds;
• 4: a representation of formant jumps at the transitions between three individual tones;
• 5 shows a representation of the possibility of reducing the length of elements;
• 6a: an example for temporal inversion of transition sounds;
• 6b: the possibilities for vowel conversions;
• 6c: the possibilities for consonant conversions;
• 7 shows an example of the change in the auditory impression by shifting the starting point;
• 8a, b, c: an example of the computational modification of a voiced individual sound to vary the pitch;
• 9: an excerpt of an example of the arrangement of true, skipable and additional sample values as well as marker words in a stored element of a voiced individual sound;
• 10: a representation of the division and the content of the sound element memory,
• 11: a representation of the sequence of a transcription and
• Fig. 12: a representation of a synthesis example (monotone).

As shown in FIG. 1, a speech synthesis system in embodiments according to the invention essentially consists of two details, that for the transcription and that for the synthesis itself. You have to enter either a character string, which is done via an alphanumeric keyboard or via a V24 interface can, or a sound string. Although experienced or trained users can also enter the sound strings directly using suitable keyboards, in most applications, if the transcription is not used, the synthesis unit will then receive the corresponding input signals from a remote location via a data line and the V24 interface. Of course, other interface conditions can also be complied with and implemented within the scope of professional skills. The transcription unit uses prepared rules, summarized under the term grammar, the synthesis unit essentially uses the stored sound elements. The synthesized sample sequences arrive via a digital-to-analog converter D / A and a controllable amplifier either directly via a loudspeaker or via a low-frequency socket and a voice transmission line, not shown, and at a remote location via a loudspeaker as sound waves for reproduction, better output.

The block diagram shown in FIG. 2 shows, in particular in the size comparison of the individual blocks, the storage space requirement with the proportions that are required for the synthesis and the transcription as a whole. The system is designed on the basis of a microprocessor IlP. An alphanumeric keyboard is provided for entering the character string, and a conventional electro-acoustic converter is provided for outputting the sound waves perceptible as speech. For the transcription, the microprocessor µP works with the transcription program TP and the transcription grammar TG, for speech synthesis with the synthesis program SP and the synthesis matrix SM, the required sound elements being taken from the sound element memory SE as required and into the RAM stored in the working memory from which Derived shape of the relevant sound string, chained in the relevant number and order and passed to the digital-to-analog converter (see FIG. 1, D / A). A volume control within the synthesized words and sentences takes place, also controlled by the microprocessor µP and according to commands entered therefor, in the controllable low-frequency amplifier (see FIG. 1) before the radiation of the sound waves or the transmission of the low-frequency signal.

The position of the three first formants for nine different sounds shown in FIG. 3 shows that the first and the second formants in particular are of considerable importance for the formation of sounds. Due to the linear division of the frequency scale, it should not be overlooked that the third formant also occupies the area of about half an octave.

4 shows the position of the formants for three sounds. It can be seen that there are sometimes considerable jumps at the transitions that would be perceived as extremely unpleasant. However, these are known phenomena that should not be left unmentioned only to indicate the complexity of the problems that have to be considered in a speech synthesis system.

The time signal of the word "ash" shown in FIG. 5 is intended to illustrate the possibility of reducing the length of sound elements by segmentation into quasi-stationary areas S and transition areas U. Within the quasi-stationary areas S, fundamental speech frequency periods P can be recognized, which form the significant area of a sound and are only stored in this length as an element for the synthesis need to be. Similar fundamental frequency periods can also be seen in transition areas and are also sufficient as a synthesis module.

The options given in FIGS. 6a, 6b and 6c for temporal inversion of transitions (FIG. 6a), for vowel conversions (FIG. 6b) and for consonant conversions (FIG. 6c) speak for themselves and therefore do not require any further explanation here. However, as already mentioned above, it should be pointed out that a shortening or lengthening of the duration of the sound not only brings about a shift in the pitch, but in particular causes a sound conversion. Incidentally, of the 16 sounds shown in FIG. 6c, only those given in the first place in each line need to be stored. Although these are the sounds with the most required sample values in each case, this saves storage space of a good 60% compared to storing all of these sounds.

The change in the auditory impression shown in FIG. 7 indicates that 20 test persons should determine a consonant conversion (in brackets) which - apart from two people when the starting point is shifted to 160 ms - confirm the stated auditory impression in the individual conversion forms.

8a, 8b and 8c show an example of the manner in which the pitch variation which is essential in the invention is made possible. A fundamental frequency period of the sound / a / is plotted in FIG. 8a. For modification, the associated error signal (FIG. 8b) is first generated by a prediction error filter. From this it can be seen that discrete places can be specified where modifications have to be made without changing the sound character but its pitch. 8c shows the period of the sound / a / shortened by approximately 20% compared to FIG. 8a. It can be seen in the comparison of the curve profiles of FIGS. 8a and 8c that a shortening of the period, i.e. H. an increase in pitch, the actual characteristic image does not change, the sound / a / is therefore retained as such and - as desired - sounds higher.

FIG. 9 shows an example - in excerpts - of the order (serial number) in a stored element of a voiced transitional or individual sound that can be changed in pitch, true samples WAW, skipped samples DAW, additional samples ZAW and marker words MAW follow one another. Usually, H. if no pitch variation is to take place, only the true samples WAW are used. Additional samples ZAW are used to lower the pitch, but samples DAW that can be omitted compared to the normal case are omitted for an increase. The marker words not only localize the additional ZAW or skewable samples DAW, but also advantageously determine their priority for pitch changes.

The block shown in FIG. 10 is intended to illustrate the ratio of the storage space requirement which is required for the synthesis building blocks, the elements of the individual and the transition sounds. These are primarily the true sample values WAW of the elements, but also also the marker words MAW and the computationally determined additional sample values ZAW for the voiced individual sounds or the voiced areas of transition sounds. The dashed line between the areas for the individual sound and the transition sound elements shows a distribution roughly in the ratio 4: 6 to 5: 6.

11, in which the sequence of a transcription is shown, speaks for itself, but is to be explained in more detail using an example, the transcription of the word “blurring”.

• Präfix: «ver»
• Stamm: «wisch»
• Suffix: «en»
• Endung: «d»

The lexical processing shows that it is no exception. The word analysis is done according to:

• Prefix: «ver»
• Trunk: «wipe»
• Suffix: «en»
• Extension: «d»

When transcribing the stem according to rules, it must be determined whether the pronunciation of the symbol sequence «sch» must be as a sound / sh / (as in: school) or as two separate sounds / s / and / ch /. The following rules from the catalog apply: If there are two vowels or an umlaut before “sch”, the second alternative applies first, ie two separate sounds / s / and / ch / (example: Röschen / Roeschen). However, if the second vowel is a "u", the first alternative still applies, i.e. the individual sound / sh / (example: swap).

If there are three "vowels" before "um", whereby an umlaut is again considered to be two vowels, the second alternative applies again, ie two separate sounds / s / and / ch / (example: Häuschen / Haeuschen). Exceptions to this are just two words: deceive and deceive.

wobei hier jeweils nur einige Lautbeispiele aufgeführt sind.Another example from the extensive rule catalog concerns the sound / ch /. A distinction is made between:

only a few sound examples are given here.

12 shows the signal curve - monotonous - of the synthesized word / pocket /. (A representation containing the waveform, the melody, rhythm and dynamics would undoubtedly be confusing, as far as is possible with the usual means). A shortened / s / was used for the / t /. The transition / ta / comes from the double sound / sa /. For the / a / 8 repetitions were added to a period. The transition / asch / became the double sound / sa /, in time inverted, taken. The / sh / is an unvoiced individual sound. The transition / scha / stems from the double sound / sa /. Finally, for the / a / at the end a period was repeated 6 times and then 6 times, but evaluated with a section of a sine function.

Claims (14)

1. A method for the synthesis of speech with an unlimited vocabulary in the time domain from sound elements which are obtained from natural speech samples and are coded with low redundancy in digital form, stored and also reduced in length, in each case to the significant area of the relevant time signal typical of the sound, and in number, by utilizing related sounds which are mutually transfomable into each other, having regard to the necessary storage space requirement, these sound elements being linked, with respect to the form, number and sequence required, into digital signal requences on the basis of input commands and of predetermined rules of linkage for the purposes of speech synthesis, these signal sequences being used to generate, by means of digital/analog conversion and controllable amplification, sound waves which can be perceived as speech, characterized in:
providing a total of about 100 sound elements, that is to say

- about 50 elements for transitions sounds with an average of 240 samples each for an output frequency of 8 kHz, and

- about 40 elements for phonemes with an average of 500 samples for unvoiced and 140 samples for voiced phonemes each and an output frequency of 8 kHz,

and enabling the pitch to be varied for reproduction, in the case of the elements for the voiced transition sounds and phonemes, by omitting or using at least once, as a result of appropriate input commands, those samples and/or values which are preset as suitable by means of marker words at discrete positions in the time signal, depending on requirement, when the digital signal sequences are being formed.

2. A method as claimed in claim 1, characterized in providing within the digitally stored elements for the voiced phonemes - with the average of 140 samples for phonemes and 240 samples for transistions sounds - additional values for the purpose of varying the pitch.

3. A method as claimed in claim 2, characterized in that an additional value occupies an interpolated value between adjacent true sampled values.

4. A method as claimed in one of claims 1 to 3, characterized in that marker words are provided preferably at low-slope positions in the time signal.

5. A method as claimed in claim 4, characterized in that marker words at low-slope positions in the time signal are given a higher priority for pitch variation purposes that those at places with a greater slope.

6. A method as claimed in one of claims 1 to 4, characterized in that marker words have digital patterns reserved for them which do not occur in the samples.

7. A method as claimed in claim 6, characterized in that the patterns with the highest positions, for example, with 8-bit words, patterns 246, 247, ... 255 are reserved for marker words.

8. A method as claimed in one of claims 1 to 7, characterized in that during the pauses between words, the shape of the sound elements needed for linking the next successive word is determined by means of the input commands.

9. A method as claimed in one of claims 1 to 8, characterized in that sequences or orthographical characters entered via an alphanumeric keyboard are being automatically transcribed, in an process step preceding the synthesis process proper, into a sequence of phonetic characters which are suitable for use as the input commands.

10. A method as claimed in claim 9, characterized in that firstly exceptions and foreign words, complied lexicographically, are processed, and besides this the vocabulary is subjected to prefix processing, taking into account exceptions, separating off endings and suffix processing, also taking into account exceptions, and the stems of the words are transcribed in accordance with rules stored catalog-fashion.

11. A circuit arrangement designed for carrying out the process according to one of claims 1 to 10, characterized by:

a microprocessor (pP) which is connected to read-only memories (ROM) with a storage capacity of a total of 32 kBytes an to a 1 kByte random-access memory (RAM), and by

an electro-acoustical conversion device, known in itself and consisting of a decompanding-type digital/analog converter, an audio-frequency amplifier and a loudspeaker.

12. A circuit arrangement as claimed in claim 11, characterized in that the capacity of read-only memories (ROM) is distributed in:

1.5 kBytes for the transcription program,

6 kBytes for the transcription grammar,

1.5 kBytes for the synthesis program,

1 kBytes for the synthesis matrix and

22 kBytes for the sound elements.

13. A circuit arrangement as claimed in claim 11 or 12, characterized in an input at which a V24 (RS 232) interface is provided.

14. A circuit arrangement as claimed in one of claims 11 to 13, characterized by an output at which an audio-frequency jack is provided.

Images