FI119859B - A method for producing speech synthesis - Google Patents

Description

METHOD FOR PRODUCTION OF SPEECH SYNTHESIS FIELD OF THE INVENTION

The invention relates to the production of text-to-speech TTS. In the method of the Kek-5 invention, for example, the size of the database used in speech synthesis can be reduced such that some of the sound units used in the synthesis, such as diphones, are phonetically replaced by a similar sound unit more general in the target language. In addition, in the target language, rare sound units are replaced by smaller sound units such as semifones.

BACKGROUND OF THE INVENTION

15 A speech signal is an extremely complex signal whose structure depends not only on linguistic content but also on the size and state of the speaker's vocal cords during speech production. Audio can be digitized or digitized, as is the case with CDs. 20 Recording human speech or even music in digital form is thus not a major problem, although digital speech takes up much more space than text.

Producing artificial speech is a challenging task for its si-25. Identifying words into word categories and accentuation may require complex analysis before text can be spoken. In some respects, producing artificial speech from stored human speech is advantageous, since whole sentences or weighting of sound are naturally natural in this way. In different ways, cut and glued parts can be varied in speech. Such pre-recorded announcements are used, for example, in buses and buses of some cities and buses, where each stop tells about alternatives or timetables.

2

The more the various combinations are cut and glued, the more there is also a risk of joint misalignment.

The Cut and Paste method overcomes the naturalness of the 5 frequently synthesized speeches and the more the more extensive finished units are used. Similarly, in the cut and glue method, it may be difficult to match speech to the prosody of natural speech with the rhythm, weight, and intonation of 1st speech, especially when producing completely new material. The pause and intensity of speech can be adjusted as needed within certain limits, but adjusting the tone of speech to natural sentences or replies can be challenging because it is both cumbersome and difficult to obtain a sufficiently wide range of samples from one and the same speaker for synthesis.

Previously recorded speech can also be used in a text-to-speech synthesis. In the concatenate-20 thiosynthesis, short speech segments are selected from a prerecorded database, which are linked sequentially to provide the desired expressions. The longer the speech segments are used, the less synthesized speech becomes to problematic segment junctions and other annoying discontinuities, but at the same time the need for memory increases.

In linguistics and phonetics in general, words are considered to consist of phonemes that come to be known as "phonemes". as allophones or sounds. The phoneme is an abst-30 key sonically feasible unit that cannot be divided into smaller segments, but can be considered to consist of so-called. of the discriminatory features. For a clearer idea of how words are divided into linguistic segments, 1. sound units can be found in Ku-35 bar 1, which shows the division of the word "fireplace" from letters to sounds, semifones and diphones. On the other hand, when a diphon is composed of halves of a sound, 3 is made up of a single triple and the halves of the preceding and following sounds.

However, cutting and gluing whole sounds as such does not work as desired for speech synthesis. Therefore, in the diphonicon catenation, the latter half of the sound and the first half of the subsequent sound are cut into one sample. Diphon sample segments are shown in the top 10 rows in Figure 1. In diphon collagenization, as in triphon collagenization, the junctions of the segments hit the center of each sound, thus minimizing the changes in sound and minimizing distortion of the junctions. Di-15 phones are obtained by cutting them out of any pair of phonemes and / or allophones that occur in the language being implemented. (Diphone) concatenation synthesis at its simplest does not require extensive phonetic understanding and is a very popular form of speech synthesis in 20 markets.

The problem with concatenation synthesis is the large size of the database containing the sound samples. If you want to include all possible combinations, take up a lot of memory space in the sample library. Particularly problematic is the production of speech synthesis on portable terminals where the database storage space is limited.

PURPOSE OF THE INVENTION

It is an object of the invention to provide a method for producing speech synthesis. In particular, it is an object of the invention to reduce the size of the database required for speech synthesis by replacing some of the voice units used in speech synthesis with a sound sample of another phonetically similar voice unit, which is more general in the target language.

4

SUMMARY OF THE INVENTION

It is an object of the present invention to provide a method, system and software product for generating speech synthesis from sound samples of sound units stored in a database, so that the size of the database used is as small as possible. In this application, sound units and their smaller parts are especially meant diphones and semi-phons. However, the sound unit may be any other sound portion or part thereof, such as a tryphone, byte or half-byte, suitable for the purpose.

With respect to the features of the present invention, reference is made to the claims.

The method of the invention generates 15 synthetic speech samples of voice samples stored in a database, the method of receiving a text input and dividing the received text input into sound units. The method of the invention further comprises extracting sound samples corresponding to 20 sound unit strings from the database so that they satisfy the following conditions: determining very common sound units (diphons), common sound units, and rare sound units in the target language; determining whether the sound unit is statistically classified as 25 rare sound units in the target language; if the sound is statistically reported as a rare sound unit in the target language, it is compiled from smaller sound units such as semifones; otherwise, determining if the sound unit is phonetically similar to another sound unit more common in the target language; if the sound unit is phonetically similar to another sound unit that is more general in the target language, it is replaced by the sound sample of the more general sound unit; otherwise, retrieving from the database a sample of the sound corresponding to the original sound unit; and generating a speech-35 synthesis from the audio samples extracted from the database.

Furthermore, the method of the invention further determines the extremely rare sound units in the target language and replaces these sound units, which are very rare in the target language, with a silence-silence sample.

The definition of sound units as very common, generic, rare or very rare is based on the limits of the relative likelihood of each sound unit being specific to each target language.

The present invention further relates to a system for generating speech synthesis from voice samples stored in a database comprising: a central processing unit for speech synthesis, a display, means for text input, a memory and a database containing voice samples, where ; phonetically identical sound units are replaced by another, more general, sound unit in the target language; and the very common sound units tal-20 are flown as such. In addition, the system comprises an output for transmitting speech generated in speech synthesis.

The invention further relates to a software product for generating speech synthesis from audio samples stored in a database, which software product is arranged to receive a text input and divide the received text input into sound units. The software product is further arranged to extract an audio sample corresponding to a sound unit from a database so that it satisfies the conditions: the software product is arranged to determine very common sound units, common sound units, and rare sound units in the target language. The software product is further arranged to determine whether the audio unit is statistically identified as a rare-35 audio unit in the target language; if the sound unit is spatially rare in the target language, the software product is further arranged to assemble the sound unit from smaller 6 sound units such as semifones; otherwise, the software product is arranged to determine whether the sound unit is phonetically similar to another sound unit more common in the target language; if the sound unit 5 is phonetically similar to another sound unit more general in the target language, the software product is arranged to replace it with the sound sample of the more general sound unit; otherwise, the software product is arranged to retrieve from the database an audio sample corresponding to the original audio unit. In addition, the software product is arranged to generate speech synthesis from audio samples extracted from a database.

With the present invention, the amount of memory required for a database used in concatenation synthesis, such as the diphtheritie-15, is reduced. The present invention further facilitates the implementation of speech synthesis, particularly in portable devices with limited memory capacity. Furthermore, the invention enables multilingual use of speech synthesis terminals, whereby databases containing multiple target languages require less and less memory capacity. In this case, a larger portion of the memory capacity of the portable terminal may be used for other purposes. In addition, ver-25 can transmit the desired data in plain text, regardless of the form (in plain text format or as an audio signal) the user receives the message.

LIST OF FIGURES

Figure 1 shows a segmentation of the word "fireplace" into smaller units, Figure 2 illustrates a functional block diagram for implementing speech synthesis according to the invention, Figure 3 shows an application for determining the frequency of sound units according to the invention; 5 shows a system according to the invention for generating speech synthesis in simplified form.

DETAILED DESCRIPTION OF THE INVENTION

Figure 2 illustrates an embodiment of a functional block diagram of speech synthesis according to the invention. In the method according to the invention -10, the input is received in text format 21. The text input is normalized, i.e., the numbers are converted into read words, the abbreviations are written out, etc. After that, the text is further processed e.g. converting the words into sound units (22), for example diphones, dividing the sound unit strings and their corresponding diphones into bytes, and calculating the sound durations and the pitch of the diphones. The musical flow includes speech rhythm, accents, and intonation, the characteristics of which are analyzed from natural speech. An analysis p-20 platform is used to create rules for producing similar properties to synthetic speech. Each diphtheria is analyzed separately. Analyzing a diphon determines whether the diphon in question is classified as a rare diphon in the target language. Diphon 25. 30 If the diphon is similar to another diphon that is more common in the target language, it will be replaced by this diphon 26. If the diphon is not phonetically similar to a more general diphon, the original diphon 27 will be used. its duration 29 and generating the speech signal 210 itself. For each text input audio unit, a corresponding audio sample is retrieved from 8 databases until all text input diphones have been processed.

The most common and rare diphthongs in the Finnish language can be determined, for example, in the following way. Create a text corpus, a database that contains masses of Finnish text. From this corpus of text it is possible to statistically examine which diphones are common and rare in Finnish by defining a threshold for the frequency of sound unit / diphon 10. Exceeding the threshold can be determined to be general. Figure 3 illustrates an application for the general or rare determination of diphones. The y-axis of Figure 3 represents the relative frequency of the diphones, that is, how common the diphon is, and the x-axis represents the probability that a replacement sample of the sound unit will be used for the diphon. In the case of Figure 3, diphones above the threshold 31 are defined as very common. Diphones above the threshold 32 are defined as common, with diphones below that defined as rare. Diphones below threshold 33 are extremely rare in the target language. In Figure 3, diphones above the first threshold 31 are so common that they can be stored in a database as such. Diphones above the second cut-off value 32 can be replaced by another diphon, e.g. Diphones below the second cut-off value 32, on the other hand, are defined as rare enough to be formed of smaller voices, such as half-diphones, semiphones. Very rare diphones, that is, sound units between the limit values 33 and 34, can be replaced, for example, by mere silence.

If the diphon does not exceed the threshold 32 of Figure 3 for a sufficiently generalized diphon, the diphon is formed from its half semiphones. In this case, the diphtheria is statistically classified as rare in the 9 target languages. For example, if the word contains a diphon "h-j", this rare diphon is assembled from the pairs "h-silence" and "silence-j". If the diphtheria is not classified as a very rare sound unit, it is checked whether the diphtheria is phonetically similar to another diphtheria which is more common in the language. For example, "g-ö" and "k-ö" are phonetically similar diphones. If the diphon in question is phonetically similar to another diphon, it may be replaced by 10 more general diphones. Unless the diphtheria is classified phonetically similar to another diphtheria more common in the language, the original diphon is used to generate speech synthesis.

The replacement of a diphon by a sound sample of another diphon, which is more common in the target language, is based on the phonetic similarity of the sound units, that is, the proximity of the sound units to each other when producing them. Replacing another unit of sound with a very similar type of sound sample does not produce exactly the same speech as the original sound sample, but the changes in its quality are so slight that it may not be distinguished at all by the human ear.

Substitution of diphones can also be accomplished according to the following 25 principles. All diphones above the limit 31 of Figure 3 are classified as very common diphones and all diphones above this limit are stored as such in the diphone database. The diphones 30 between the thresholds 31 and 32 are replaced by a very common diphon above the threshold 31, which is phonetically similar to the present diphon between the thresholds 31 and 32. Diphones between the boundary values 32 and 33 of Figure 3 are defined as rare diphones, whereby they are replaced by two halves of the diphon, semiphones. In addition, the diphones (very rare diphones) between the limit values 33 and 34 are replaced by the silence only, i.e. the silence-silence pair.

All possible sound units in the target language are marked in the database with the appropriate 5 name tags, 1. indexes, in order to retrieve a specific segment from the database. In the prior art database, each diphtheria is represented by a single sample of sound, whereupon the index assigned to it recognizes it directly to the corresponding sound sample in the database. However, in this case, the diphone database requires a disproportionate amount of memory.

In the database of the sound samples of the sound units of the system according to the invention, some of the diphones have been phonetically replaced by a similar di-15 phon or two semiphones. Figure 4 shows an indexing of diphones 41 according to the invention in a diphonic database 42. Each diphon 41 is given an index 43, based on which a corresponding voice sample is retrieved from a diphone database 42. Diphones 411 and 20412 are designated 42. A phonetically similar diphone 411 to diphone 411 has been replaced by a diphone 411 sound sample 413. Also, half-diphones, i.e. semiphones, can be replaced on the basis of their phonetic properties. For example, the sound _a 414 after the sound m or p is phonetically similar so that only one sample of such a sound unit is needed in the database. In this case, the index iy refers to the m_a 415 of the semiphones composed 30 of the silence 417 and of the silence 414. For example, the index ix of the diphon t_k 419, compiled as a rare sound 35 in the target language, refers to two semiphones: t_silence and silence_k.

11

The number of sound samples stored in the database can be significantly reduced by replacing some of the diphones with another phonetically similar diphon, which is more common in the target language. Of course, reducing the number of sound samples by 5 also reduces the amount of memory required by the database. However, di-phones that begin or end with a silence-sound unit are not removed from the di-phonebook because rare sound units, such as 10-tkk, are formed from t-silence-silence-k.

The system according to the invention is shown in simplified form in Figure 5. The system comprises a display 50 and an input device 54 such as a traditional 15 keyboard or mouse for entering text. The system of the invention comprises a central processing unit (CPU) 51 which controls the operation of the entire system. The Central Unit is responsible for receiving and processing the text input. It searches the Diphone Database 55 20 for a sample of voice corresponding to each diphon and forwards the generated speech to the output 53. The output may be, for example, a microphone that transmits the text-generated synthetic speech to the user. The memory 52 contains, in addition to the diphon database 55, software for the central processing unit 51 to perform the functions assigned to it.

Although the diphtheria database is mentioned as the database in the above examples, the method of the invention can also be utilized to generate speech synthesis using other sound units or smaller units of sound units, for example triphones, semiphones, bytes or half-bytes. Correspondingly, although the target language in the above examples is Finnish, the method of the invention can also be applied to other target languages.

The invention is not limited only to the above exemplary embodiments, but many other embodiments are possible within the scope of the inventive idea defined by the claims.

Claims (9)

A method of speech synthesis generation from burst sound units corresponding to sound samples stored in a database, the method comprising the steps of: a text input (21) being received; the received text input is divided into burst audio units (22); characterized in that the method further comprises the stages: from the database, a sound sample corresponding to the burst sound unit is selected so as to meet the conditions: in the milling puncture, particularly general burst sound units, general burst sound units and unusual burst sound units are defined; is defined, if the burst sound unit in the burst is statistically recorded as an unusual burst sound unit (23); if the burst sound unit is statically recorded as an unusual burst sound unit in the molar crest, it is joined by 20 smaller burst sound units (24); otherwise, it is defined whether the burst sound unit is phonetically similar to any other burst sound general burner (25); if the burst sound unit is phonetically the same as any other burst sound unit, it is replaced by the sound burner sound sample (26); otherwise, an audio sample (27) corresponding to the original burst sound unit is retrieved from the database; And speech synthesis is generated by the cracked sound samples (210) extracted from the database. 2. A method according to claim 1, characterized in that the method further comprises the stages: in the milling puncture, the particularly unusual cracking sound units are defined; the static-sounded burst units in the target slog as particularly unusual bursting sound units are replaced with a silence-silence sound sample. 3. A method according to claims 1 or 2, 5, characterized in that the definition of the explosive sound units as extremely general, general, unusual or particularly unusual sound sound units is based on pitch-drawing limit values for the density of the relative presence of the spray sound unit in each mill. A system for speech synthesis generation from sound samples stored in a database, comprising: a central unit for speech synthesis generation (51); a display (50); means (54) for providing a text entry; and a memory (52); Characterized in that the system further comprises: a database (55), in which the database when storing sound samples the bursting sound units which are statically recorded in the target seam, which are unusually replaced with smaller bursting sounding units; phonetically similar explosive sound devices are replaced by someone else, in general, a more pronounced sound sound device; and particularly general cracking sound units are saved as such; and an output (53) for conveying the speech generated in speech synthesis. 5. A system according to claim 4, characterized in that when storing sound samples in the database, the unusually unusual crack sound units are replaced by a silent-silent sound sample. 6. A system according to claim 4 or 5, characterized in that defining the burst sound unit as a very general, general, unusual or particularly unusual bursting unit is based on characteristic limit values for the density of the bursting unit's occurrence in each mill. 7. Software product for speech synthesis generation from sound samples stored in a database (55), which software product is arranged to: receive a text input (21); and dividing the received text input into crack audio units (22); characterized in that the software product is further arranged to: extract from the database a sound sample corresponding to the spray sound unit, so that it fulfills the condition: the software product is arranged to define in the target language particularly general spray sound devices, general sound devices and unusual speech audio devices; The software product is further arranged to define if the burst sound unit in the burst is statistically recorded as an unusual burst sound unit (23); if the burst sound unit is statically listed as an uncommon burst sound unit in the burst, the software product is further arranged to assemble the burst sound unit of smaller burst sound units (24); otherwise, the software product is arranged to define whether the burst sound unit is phonetically similar to any other burst sound unit 30 (25) in the mill. if the burst sound unit is phonetically similar to any other burst sound unit, the software product is adapted to replace the burst sound unit with a sound sample (26) of a more general burst sound unit; otherwise, the software product is arranged to retrieve from the database a sound sample (27) corresponding to the original burst sound unit; and the software product is arranged to generate the speech synthesis of the audio samples extracted from the database (210). 8. A software product according to claim 7, characterized in that it is arranged to: 10 define the highly explosive sound devices in the milling language; in the target slug, replace a static-sounded unit with a silent-silent sound sample as a particularly unusual burst sound unit. 9. A software product according to claim 7 or 8, characterized in that the software product is arranged to: define the spray sound unit as a particularly general, general, unusual or extremely unusual spray sound unit based on the characteristic limit values for the density of the spray sound device. in each mill.