EP1110203B1 - Device and method for digital voice processing - Google Patents

Abstract

The invention relates to a device for digital voice processing which comprises a sentence melody generating device for generating a sentence melody for a text, and an editing device for displaying and modifying the generated sentence melody.

Description

The present invention relates to an apparatus and a method for digital speech processing or language production. Current systems for digital Voice output has so far been used in environments in which a synthetic Voice is acceptable or even desired. The present invention, however, relates a system that enables natural-looking speech to be generated synthetically.

In current systems for digital speech generation, the Automatically generated information about the melody and the emphasis, such as B. described in EP 0689706. In some systems it is possible to add additional To include commands in the text stream before it reaches the language producer is handed over, e.g. in EP 0598599. These commands are e.g. as not pronounceable) special characters, such as described in EP 0598598.

The commands built into the text stream can also provide information about the Characteristics of the speaker (i.e. parameters of the speaker model) included. In EP 0762384 describes a system in which on the screen on a graphic User interface these speaker characteristics can be entered.

The speech synthesis takes place using auxiliary information which is given in a database (e.g. as a "waveform sequence" in EP 0831460). For the pronunciation of words that are not stored in the database, must but there are still rules for pronunciation in the program. The Composition of the individual sequences leads to distortions and acoustic Artifacts if no measures are taken to suppress them. This The problem (one speaks of “segmental quality”) is considered largely solved today (see e.g. Volker Kraft: Linking natural language building blocks to Speech synthesis: requirements, techniques and evaluation. Fortschr.-Ber.VDI Row 10 No. 468, VDI Verlag 1997). Nevertheless, there is also a modern one Speech synthesis systems have a number of other problems.

One problem in digital speech output is, for example Multi-language capability.

Another problem is the improvement of the prosodic Quality, i.e. the quality of the intonation, compare for example "Volker Kraft: Linking natural language building blocks for speech synthesis: requirements, Techniques and Evaluation, Progr.-Ber.VDI Reihe 10 Nr 468, VDI-Verlag 1997 ". The difficulty is due to the fact that the intonation from the Orthographic input information can only be reconstructed inadequately. It is also dependent on higher levels such as semantics and pragmatics as well Speaker situation and type of speaker.

DE-A-196 10 019 describes a method and an apparatus for digital speech processing with a sentence melody generation device for Generation of a sentence melody for a text known. It also enables display the speech signals in the time and frequency domain and by a Marking the time signal a change in the fundamental frequencies of certain Reach segments.

In general it can be said that the quality of today Narrator systems meet the requirements where the listener is one synthetic voice expected or accepted. However, the quality is often synthetic language as inadequate or perceived as unsatisfactory.

It is therefore an object of the present invention, an apparatus and to create a process for digital speech processing that enables to produce synthetic language of better quality.

It is another object of the invention to use natural language to produce synthetically. The applications range from creating simpler Texts for multimedia applications up to film settings (dubbing), Radio plays, and audio books.

Even if the synthetically generated language looks natural, are sometimes interventions to create dramaturgical effects required. A further object of the present invention is therefore that of Provision of such intervention options.

The present invention is defined in the independent claims. The dependent claims define particular embodiments of the invention.

Essentially, the object of the invention is achieved by the for Text generated sentence melody can be modified using an editor.

Special embodiments of the invention allow in addition to Editing the sentence melody editing other characteristics of the synthetic generated language.

The starting point is the written text. But one to achieve sufficient (especially prosodic) quality, as well as to achieve it Dramatic effects are preferred to the user Embodiment given extensive options for intervention. The User is in the function of the director who is the speaker on the system defined and them speaking rhythm and melody, pronunciation and emphasis pretends.

Preferably, the present invention also includes generating one Phonetic transcription for a written text, as well as the provision of the possibility Modify the phonetic transcription generated, or the phonetic transcription based on generate modifiable rules. This can be a special one, for example Accent of a speaker are generated.

In a further preferred embodiment, the invention comprises a dictionary facility in which the words of one or more languages are saved along with their pronunciation. In the latter case, this enables Multilingual capability, i.e. editing texts in different languages.

The editing of the generated phonetic transcription or Sentence melody using an easy-to-use editor, such as a graphic one User interface.

In a further preferred embodiment, the Voice processing included speaker models that are either predefined or can be defined or modified by the user. This allows Characteristics of different speakers can be realized, be it male or female Female voices, or different accents of a speaker, such as a Bavarian, Swabian or North German accent.

In a particularly preferred embodiment, the device exists from a dictionary in which all words also include pronunciation in phonetic transcription are saved (if we speak of phonetic transcription below, then this is one any phonetic spelling, such as the SAMPA notation, cf. e.g. "Multilingual speech input / output assessment, methodology and standardization, standard computer-compatible transscription, pp 29-31, in Esprit Project 2589 (SAM) Fin. Report SAM-UCC-037 ", or the international one known from language teaching aids phonetic writing, cf. e.g. "The Principles of the International Phonetic Association: Adescription of the International Phonetic Alphabet and the Manner of Using it. International Phonetic Association, Dept, Phonetics, Univ. College of London "), one Translator who converts typed texts into phonetic transcription and a sentence melody created, an editor with which texts can be entered and speakers can be assigned can and in which both the generated phonetic transcription and the sentence melody can be displayed and changed, an input module in which speaker models can be defined, a system for digital speech production, which from the Representing phonetic spelling along with the sentence melody spoken language Generates signals or data representing such signals and is capable of to process different speaker models, a system of digital filters and other devices (for reverb, echo etc.) with which special effects are created can, a sound archive, as well as a mixing device in which the generated Speech signals mixed together with sounds from the archive and can be provided with effects.

The invention can either be hybrid in software and hardware or entirely in Software can be realized. The digital voice signals generated can be via a special device for digital audio or via a PC sound card.

The present invention will hereinafter be described with reference to several Embodiments and reference to the accompanying drawings in detail described.

Figure 1 shows a block diagram of a device for digital Speech generation according to an embodiment of the present invention.

In the exemplary embodiment of the present described below Invention this consists of several individual components, which by means of or several digital computing systems can be realized, and their Operation and interaction is described in more detail below.

The dictionary 100 consists of simple tables (for each language a) in which the words of a language are stored along with their pronunciation are. The tables can be used to include additional words and their pronunciation can be expanded as required. For special purposes, e.g. for creating accents can also add additional tables with different in one language phonetic entries are generated. The different speakers will each get one Table assigned to the dictionary.

On the one hand, the translator 110 generates the phonetic script by using the Words of the entered text by their phonetic counterparts in the Dictionary replaced. If in the speaker model modifiers, the later more precisely are described, he uses them to modify the Pronunciation.

In addition, he creates the prosody using in the Speech processing known heuristics. Such heuristics are e.g. the model by Fujisaki (1992) or other acoustic methods, then the perceptual ones Models, e.g. that of d'Alessandro and Mertens (1995). These, but also older ones linguistic models are e.g. described in "Thierry Dutoit: An Introduction to Text-to-Speech Synthesis, Kluwer 1997 ". There are also procedures for the Segmentation (setting breaks), which is also generated by the translator.

The choice of procedure is of minor importance, because the translator merely creates a specification of the prosody, which is created by the user can still be changed.

With the Editor 120, the user has an instrument in his hand with which Enter pronunciation, intonation, emphasis, tempo, volume, pauses, etc. and can change.

First, he assigns a speaker model to the text sections to be processed 130 to which later explained in more detail with regard to structure and mode of operation becomes. The translator responds to this assignment by using the phonetics and if necessary, adapt the prosody to the speaker model and regenerate it. The Phonetics are displayed to the user in phonetic transcription, the prosody e.g. in one of the Music taken symbolism (musical notation). The user then has the option of to change these specifications, to listen to individual sections of text and his Improve entries again, etc.

Of course, the texts themselves can also be entered in the editor, if they are not imported directly from another word processing system can.

Speaker models 130 are, for example, parameterizations for the Speech production. In the models, the characteristics of the human Replica speech tract. The function of the vocal cords is determined by a Pulse sequence shown, of which only the frequency (pitch) can be changed. The other characteristics (oral cavity, nasal cavity) of the speech tract are included digital filters. Your parameters are stored in the speaker model. It standard models are stored (child, young lady, old man, etc.). The User can generate additional models from them by changing the parameters suitably chooses or changes and saves the model. The ones deposited here Parameters are created during language generation, which will be explained in more detail later. used with the prosody information for intonation.

Special features of the speaker such as Accents or Speech errors can be entered. These are used by the translator to modify the Pronunciation used. A simple example of such a modifier is e.g. the Rule to replace "∫t" with "st" (for the generation of the Accent of a hamburger).

A speaker model can, for example, concern the rules according to which the translator creates the phonetic transcription, different speaker models can follow different rules. However, it can also be one certain set of filter parameters correspond to the speech signals to be processed in accordance with the speaker characteristics specified thereby. Of course, any combination of these two aspects is one Speaker model conceivable.

The task of the speech generation unit 140 is to: Predefined text together with the one created by the translator and by the user edited phonetic and prosodic additional information a numerical Generate data stream that represents digital voice signals. This Data stream can then be from an output device 150, such as a digital one Audio device or a sound card in the PC, in analog sound signals text to be output.

A conventional text-to-speech can be used for speech generation Conversion procedures are used, but the pronunciation and the Sentence melody has already been created. Generally one differentiates between rule-based and chain-based synthesizers.

Rule-based synthesizers work with rules for generating the Lute and the transitions in between. These synthesizers work with up to 60 Parameters, the determination of which is very complex. You can do a lot with them good results are achieved. An overview of such systems and information for further literature can be found in "Thierry Dutoit: An Introduction to Text-to-Speech Synthesis, Kluwer 1997 ".

Chain-based synthesizers, on the other hand, are easier to use. You work with a database that stores all possible pairs of sounds. This can easily be chained, but high quality systems are high Have computing time requirements. Such systems are described in "Thierry Dutoit: An Introduction to Text-to-Speech Synthesis, Kluwer 1997 "and in" Volker Kraft: Linking natural language building blocks for speech synthesis: requirements, Techniques and evaluation. Fortschr.-Ber. VDI series 10 No. 468, VDI-Verlag 1997 ".

In principle, both types of system can be used. In the rule-based syntheses, the prosodic information flows directly into the rules a, while these are appropriately overlaid in chain-based systems becomes.

Known techniques are used to create special effects 160 used in digital signal processing, e.g. digital filters (e.g. Bandpass filter for telephone effect), Hall generators etc. These can also be used in sounds stored in an archive 170 can be used.

Archives 170 contain sounds such as Road noise, railroad, Child cries, ocean waves, background music, etc. saved. The archive can can be expanded with your own sounds. The archive can simply be one Collection of files with digitized sounds, but it can also be one Database in which the sounds are housed as blobs (binary large objects) are.

In the mixing device 180, the generated speech signals with the Assembled background noise. The volume of all signals can be used the composition are regulated. It is also possible to send each signal individually or all of them with effects.

The result of the signal generated in this way can be sent to a suitable device for digital audio 150, such as a sound card from a PC, and thus acoustically checked or issued. In addition, a (not shown) Storage device is provided to store the signal so that it later in can be appropriately transferred to the target medium.

A device that is classically implemented in hardware can be used as a mixing device can be used, or it can be implemented in software and in the entire program be involved.

Modifications to the above are readily apparent to those skilled in the art written embodiment. For example, in another Embodiment of the present invention, the output device 150 by a another computer to be replaced by a network connection to the Mixing device 180 is coupled. For example, a Computer network, such as the Internet, the voice signal generated on another Computer.

In a further embodiment, that of the Speech generator 140 generates speech signal directly to the output device 150 are transmitted without the detour via the mixing device 180. Further comparable modifications result in a casual manner for the person skilled in the art.

Claims (21)

1. A digital speech processing apparatus comprising:

   a prosody generation means for generating a prosody for a text; characterized by

   an editing means for displaying and modifying the generated prosody.
2. The apparatus of claim 1 further comprising:

   translation means for translating the text into a phonetic transcription, said translation means further comprising:

   means for displaying and modifying the generated phonetic transcription.
3. The apparatus of claim 1 or 2, wherein

   said prosody generating means and/or said translation means generates said prosody and/or said phonetic transcription based on respectively in dependence of a particular speaker's model.
4. The apparatus of one of claims 1 to 3, further comprising:

   means for displaying and/or modification of one or more speaker's models.
5. The apparatus of claim 4, wherein said speaker's model modification means comprises:

   means for modifying phonetic transcription elements for the generation of accents.
6. An apparatus for generating digital speech comprising:

   an apparatus for digital speech processing according to one of claims 1 to 4; and

   means for generating speech signals based on said phonetic transcription which may have been edited using said editing means and/or based on said prosody.
7. The apparatus of claim 6, wherein said speech signal generating means further comprises:

   a speaker's model processing means for generating said speech signals based on respectively depending on a particular speaker's model.
8. The apparatus of claim 7, wherein said speaker's model processing means comprises one or more of the following:

   a digital filter system;

   means for adopting a set of filter parameters representing a particular speaker's model.
9. The apparatus of claim 7 or 8, wherein said speaker's model processing means further comprises:

   means for selecting and/or modifying a speaker's model.
10. The apparatus of one of claims 6 to 9, further comprising:

    effect generating means for generating sound effects.
11. The apparatus of claim 10, wherein said effect generating means comprises one or more of the following:

    digital filter means for modifying the generated speech signals, and/or

    a hall generator for generating a hall effect.
12. The apparatus of one of claims 6 to 11, further comprising:

    archive means for storing sounds; and

    mixing means for mixing the generated speech signals with the sounds stored in said archive means.
13. The apparatus of one of the preceding claims, further comprising:

    a graphical user interface for editing the generated phonetic transcription and/or prosody.
14. The apparatus of one of the preceding claims, further comprising:

    means for modifying speech rhythm and/or pronunciation and/or intonation.
15. The apparatus of one of the preceding claims, further comprising:

    display means for displaying the prosody by means of a symbolic notation.
16. The apparatus of one of the preceding claims, further comprising:

    dictionary means in which the words of one or more languages are stored together with their pronunciation.
17. The apparatus of claim 16, wherein for at least one dictionary entry different phonetic entries are stored in said dictionary means.
18. The apparatus of one of claims 6 to 17, further comprising:

    means for converting said digital speech signals into acoustic signals.
19. A digital speech processing method comprising:

    generating a prosody for a text;

    displaying said generated prosody; said method being charcterized by:

    editing said generated and displayed prosody.
20. The method of claim 19, further comprising:

    using an apparatus according to one of claims 1 to 18 for generating digital speech.
21. A computer program product comprising:

    a medium, in particular a data carrier, for storing and/or transmitting digital data readable by a computer, characterized in that said stored and/or transmitted data comprise:

    a sequence of computer-executable instructions causing said computer to carry out a method according to one of claims 19 or 20.

Images