JP2007527555A - Prosodic speech text codes and their use in computerized speech systems - Google Patents

Abstract

  Disclosed is a method and system for acoustically encoding text for use in synthesizing speech from text, the method providing a desired prosody to the spoken text to convey expressive meaning. Marking the spoken text with one or more graphic symbols shown to the speaker. Marking may have grapheme / phoneme pairs, each having a visible prosodic display grapheme that can be used in written text and a corresponding digital phoneme that functions in the digital domain. The present invention is in the mind of a wide range of applications, including voicemail systems, electronically usable equipment, cars, computers, robot assistants, games, etc. in colloquial books and magazines, dramas, and other entertainment. Useful for generating appealing, human-like machine speech.

Description

  The present invention relates to a method and computer system for providing synthesized or artificial speech from normal text input using a novel prosodic speech text code.

  Synthetic, artificial, or machine speech is useful in many ways, such as in voice books and magazines, dramas, and other entertainment, such as voicemail systems, electronically available equipment, cars, computers, robot assistants, games, etc. It has various application forms. The present invention is broadly applicable to any such system implementation, as will be apparent hereinafter.

  Useful known systems for generating artificial speech are generally described as connected systems or formant systems. Concatenated artificial voice systems are used, for example, in interactive voicemail systems to produce acceptable human voice sounds using pre-recorded complete phrases or sentences. However, such a system is not suitable for converting unknown text into speech from a large number of booklets, such as magazine articles and books. A formant system that is produced by voice “on the fly” where text is read by a machine or processed by a computer system or that synthesizes a small slice of a voice-like sound is more suitable for such a wide booklet. However, until recently, the output of such formant audio systems was mechanical, monotonous and mechanical, and was not well received.

  US Pat. No. 5,748,838 by Stevens assigned to Sensimetrics Corporation (Cambridge, Mass.) Uses glottal modeling and uses mapping relationships Thus, a speech synthesis method is disclosed in which a high level parameter of 10 or less is judged and converted into 39 low level parameters. By entering these parameters into a speech synthesizer, speech can be synthesized more easily compared to prior art systems that need to input 50-60 parameters to represent any particular speech. Stevens' disclosure is useful for its intended use, but the somewhat mechanical modeling of dissecting the voice that Stevens uses results in an audio output with a human-like sound quality that appeals to the human mind. Absent. Stevens also does not provide or suggest a means to add desirable prosody or to control and modify the prosody of synthetically or artificially generated speech.

  Co-owned US Pat. No. 6,847,931 by Addison et al., Co-pending US patent application Ser. No. 10 / 334,658 (hereinafter referred to as Addison “'658”) ), And as described in WO2003 / 065349, the synthesized text may be marked using utterance training notation as a pronunciation guide to facilitate understanding . Addison “’ 658 ”provides a speech synthesis expression parsing, uses a trained speaker to generate an available speech element database, and performs speech synthesis from text. Neither the Lessac system nor other known systems provide a simple way to communicate the desired prosody to the speech synthesizer in such a way that the prosody of the output speech can be controlled.

  In “Good American Speech” (EP Dutton & Co., Inc. (1952)) (hereinafter referred to as “McLean”) by Margaret Brendagust McLean, it is monotonous or odd Describes a notation system that marks text to indicate to the reader a desired intonation pattern or pitch change in the concatenated speech that is employed to avoid defects such as dialectal intonation. McLean's intonation pattern is useful to solve today's challenges in speech synthesis in such pre-modern work where computerization is not attempted from an artistic point of view. That And entice. Furthermore, intonation pattern of McLean, without any means relative to the pitch, and therefore, different speaker, it is difficult to utilize the intonation pattern consistently.

  The above description of background art includes insight, discovery, understanding or disclosure, or association with disclosure, which is not known in the related art prior to the present invention, but is realized by the present invention. Some such contributions of the present invention are specifically pointed out herein, but other such contributions of the present invention will also be apparent from their context. The citation of a document in this specification is not an admission that a field of the document that differs significantly from the present invention is similar to one or more fields of the present invention.

  Therefore, there is a need for a simple method for communicating the desired prosody to the speech synthesizer in such a way that the prosody of the output speech can be controlled.

  To meet the above or other objectives, the present invention provides a method for acoustically encoding text used in the synthesis of speech from text, the method comprising one or more graphical symbols. Use to mark the spoken text and indicate to the speaker the desired prosody to give to the spoken text. The present invention also provides a speech synthesis method and system that includes a prosodic code or notation for marking text using expressive meanings that specify the appropriate prosody. Marking has grapheme / phoneme pairs, each of which has a visible prosodic display grapheme that can be used in the written text and a corresponding digital phoneme that functions in the digital domain.

  The prosody to be given is selected from the group consisting of tempo, intonation pattern, rhythm, musicality, amplitude, pause for emphasis and breathing, and formal and informal articulation of words and phrases. It can have the above prosodic elements.

  The method includes marking visible text using a graphical prosodic symbol or electronically marking electronic text using an electronic version of the graphical symbol, where the electronically marked text is It can be displayed or printed as text marked with a readable graphic.

  In another aspect, the present invention provides a speech synthesizer that is controlled by an acoustic coding variable input to a speech synthesizer, which provides a synthesized speech output that embodies the desired prosodic pronunciation. Corresponding to the prosodic specification used to generate the recorded person's voice with the desired prosodic pronunciation.

  In accordance with one embodiment of the present invention, a novel notation system for phonetics, structure, and name of playable and non-playable consonants, as described herein below, And so-called four “lessac” neutrals are realized. By using these, a new text marked with a graphic is synthesized.

Furthermore, the present invention provides novel procedures and systems useful for text / voice (sometimes referred to herein as “TTS”) voice or sound recognition applications. This procedure includes
Generation of prosodic speech rules in speech synthesis and their application forms;
Acoustic demonstration of prosodic speech rules,
An acoustic database library of prosodic speech elements;
Examples of software for TTS and
One or more or all of the TTS listener tests are included.

  Hereinafter, some embodiments of the present invention, the production and use of the present invention, and the best mode for carrying out the present invention will be described in detail by way of example with reference to the accompanying drawings. Note that reference characters represent similar elements throughout several figures.

  Prior to the present invention, there was no known synthesizer “code” for specifying sound, nor was there any sound display of any measurement set that the code would create. Therefore, in the present invention, a skilled voice practitioner can be said to be a “synthesizer” and pronounce a prosodically marked text to obtain an acoustic value for a desired pronunciation. Realized. These acoustic values are used to prepare a new prosodic acoustic database library that can be used for speech synthesis in accordance with the present invention. Along with the controlled database recording method described above, using the new graphical markup symbols described herein, intonation patterns, rhythms, emphasis and pauses for breathing, and formal and informal articulation of words and phrases It would be beneficial to incorporate useful prosodic elements such as into synthetic or artificial speech.

  In order to improve the unattractive mechanical sound quality typical of known synthesized speech output, the present invention uses a rule set adapted to use appropriate acoustic elements to provide a human-like speech output. Systems, methods, and novel text encoding techniques that provide controlled or standardized human speech input used to generate a database of acoustic elements that can be mechanically assembled into speech provide.

  The human voice input and rule set preferably embodies the teaching of one or more professional voice practitioners. In one embodiment of the invention, the teaching of a certified voice training instructor is used.

  Although Arthur Lessac's teachings about theater and speech are mentioned here as examples, the teachings of other voice training instructors or other groups of voice training may be used, especially It is understood that languages other than English often use significantly different speech training teachings. Such other speech training techniques, as will be apparent from the teachings herein, provide a consistent and easily understood speech output, such as musical notation, with prosodic characters that appeal to the human mind. It is desirable to have a rule set. A voice practitioner is understood to be an individual who is well educated in the relevant vocal training or instruction, for example speaking teachers, speakers, or actors, who typically use specialized vocal skills and knowledge for speaking ability. I want.

  A voice practitioner who has been trained in the Lessac method considers the voice as an orchestra sound. That is, the voice is considered to be music. The text is from Arthur Lesack's book “The Use And Training Of The Human Voice” (Mayfield Publishing Company, 3rd edition, 1997) (hereinafter “Arthur Resack”). The voice parameters are identified as a three-element interaction, described on page 61 of Part II of the book "referred to as a book". The three voice parameters identified by Lesac are consonant, tone, and structural energy. These are introduced as structural energy as the consonant “orchestra”, the tonal nature of the voice itself as music, and the interaction of structural elements to the consonant and vowels, Arthur Lesack, the three voice parameters as eNeRGy of the voice In Lessac notes that all of this is derived from the text that is pronounced: the content of the text considered as a whole, the meaning and sound of the words, their grammatical Suggests factors such as relationships, syntax used, and messages conveyed by text.

  Human speech is analog sound, and the speaker can “reproduce the voice as a continuous instrument”, but the concept of continuously variable expression speech described in Arthur Lesack's book, especially pages 149 and 170-173, is used. To teach, it is useful to take individual points in a sequence of sentences, and this text illustrates the value of “points” within a sequence and tone range.

  The Resack system provides some alpha-numeric notation to encode the desired pronunciation to facilitate understanding of individual phonetic elements, prominent phonemes, complex tones, and so-called M-ary phones. These speech elements are mainly mixed words of individual vowels and consonants, double vowels, and consonants.

  Referring now to FIG. 1, the marked text has a text line 10 of words that are marked for pronunciation according to a Lessac notation, and an alphanumeric symbol notation line 12 is a text line 10. Positioned just above. For example, an individual familiar with the Lesac system described in Arthur Lesack's book can understand the pronunciation instructions indicated by the notation in line 12, and apply them to ensure consistency among individuals. Can pronounce the text on line 10. Samples of such notation used in practicing the present invention are shown in Tables A-E below. When the text is marked well and the speaker properly performs the marking instructions, a clear and understandable voice is produced. However, even if it is clear and understandable, the sound can be somewhat monotonous or mechanical depending on the speaker or the sound source.

わかりやすい発音を示すレサック図形表記法の有用性が、２００２年１２月３１日に出願の「テキストから音声に（ＴＥＸＴ ＴＯ ＳＰＥＥＣＨ）」と題された、アディソンらによる米国特許出願第１０／３３４,６５８号明細書に記述されているが、例が記載されておらず、図１に示されているマーク付けの特定の実施形態も記述されていない。

The usefulness of the Lesack graphic notation with easy-to-understand pronunciation is the US patent application 10 / 334,658 by Addison et al. Entitled “TEXT TO SPEECH” filed on Dec. 31, 2002. Although described in the specification, no example is described, nor is the specific embodiment of the marking shown in FIG. 1 described.

  Referring now to FIG. 2, according to the present invention, the graphical symbols shown are useful and can be used to obtain the desired prosody in the spoken process, such as letters, diphthongs, syllables, or other The pitch control necessary for pronunciation of speech elements is shown.

  The prosody code used in the present invention is an array of text, an array of alphabetic characters within a word, an array of words within a sentence, a unique array of sentences, a location of sentence arrays within a paragraph, and an array of paragraphs Is a pronunciation code related to the position of the paragraph as part of. Depending on any one or more of these considerations, what is a proper prosody, what is not a proper prosody, or what is appropriate to apply to text, pitch, or It is determined whether it is a prosodic element of timing. The proper prosody may not be apparent until the sequence is complete. In accordance with the present invention, an appropriate prosody can be applied to text in view of these considerations. The chords used in the present invention are judged according to articulatory sound creation principles and context, and are corrected for expressive meaning by specifying appropriate prosody.

  An up curve 20, a down curve 22, two bent accent symbols 24 A and 24 B, and a level sustaining portion 26 are illustrated. Each graphic notation 20-26 has a left dot such as a dot 28 indicating the start pitch and a tail such as an upward tail 30 extending to the right of the dot 28.

  The shape of the tail 30 shows how the pitch changes as the audio element is pronounced clearly. The upward tail portion 30 of the rising curve 20 indicates the rising pitch. The descending curve 22 has a downward tail 32 indicating a descending pitch, and the level sustaining part 26 remains at a level indicating a lasting unchanged pitch. The circumflex symbol 24A indicates the pitch that rises to the peak and then decreases, and the circumflex symbol 24B indicates the opposite. The prosodic graphical symbols 20-26 may be placed at any convenient location adjacent to the text to be uttered, eg, adjusted to the line immediately above the text, or selectively below the text. Text can be split with text or hyphens as an adjunct phrase to the graphic phonetic symbols described herein, but can maintain the appearance of text that is normally typed, keyed, or written preferable.

  In the following figures (from FIG. 5 onwards) described below in this specification, a forward slash 36 on the letter is used to indicate that the letter is “partially prepared”, only part of it. This is because the following consonants have closely related or identical sounds. Also, using the combined U-letters and the shallow U-joint symbol, which has the form of a hammock string under and between the reference numeral 40 herein, the other alphabetic characters English letters that are separated and usually in adjacent words indicate that the connected articulation should be pronounced continuously. In the following, with reference to FIG. 9, the use of the coupling symbol 40 to mark a direct coupling will be described in more detail.

  In general, according to one embodiment of the present invention, when marking a consonant, the consonant before the vowel is uttered, but “not reproducible”. This is because when the voice flows directly into the vowel, it is only formed very short. In this case, “reproducible” means that when a speaker clearly pronounces a reproducible consonant using a pause, pause, or pitch change when the reproducible consonant is clearly pronounced, the desired prosody is obtained. It means that an effect can be made.

  In this embodiment, the silent consonant is not marked with a graphic but is left in the computer software. The last consonant before a pause for breathing or interpretation is marked as playable. The R trombone is not reproducible and will not be marked before any other consonant or at the end, before breathing or pause for interpretation. This feature is also the feature when the computer is programmed to understand.

Referring now to FIG. 3, the illustrated prosodic graphical symbol embodiment includes marking consonants as follows. That is,
Respective percussion instruments, such as timpani drum beats, D, B and G, and snare drums, bass and tom tom drum beats, one underline for marking T, P and K,
Two underlines for marking reproducible stringed instruments N, M, V, and Z, woodwinds L, NG, TH, and ZH, and (silent) acoustic effects F, S, SH, and th.

  A consonant without a mark is not playable. That is, it is not a cautionary point for creating a desired prosodic effect using pauses, pauses, or pitch changes when pronounced clearly.

  Additional rules for prosodic graphic notation according to the present invention used for consonant mixed words include that the first alphabetic character of a consonant mixed word starting from a word is not marked. A mixed word of consonants in a word can be marked as follows:

図２〜１０に例示されている本発明を実践する際に有用な図形表記法の例示的実施形態においては、弦楽器（Ｎ、Ｍ、Ｖ、及びＺ）、木管楽器（Ｌ、ＮＧ、ＴＨ、及びＺＨ）、及び（無声）音響効果（Ｆ、Ｓ、ｔｈ、及びＺＨ）として上述した英字又は英字の組合せ及び二重母音は、以下の子音が同一の子音又は同語源の語ではなく、他のすべての子音の前に現れる場合、２本の下線により「再生可能である」とマークされる。同一の子音又は同語源の語がこの後に続く場合には、最初の子音は、子音上の前方斜線を用いて「準備された」としてマークされる。

In the exemplary embodiment of the graphical notation useful in practicing the invention illustrated in FIGS. 2-10, stringed instruments (N, M, V, and Z), woodwinds (L, NG, TH, And ZH), and (silent) sound effects (F, S, th, and ZH) mentioned above as alphabets or combinations of letters and double vowels are not consonants or words of the same consonant, If it appears before all consonants, it is marked as “reproducible” by two underscores. If this is followed by the same consonant or synonym word, the first consonant is marked as “prepared” using a forward diagonal on the consonant.

図３は、様々な単語内で発生する子音のｓ−混合語中の子音のうちのどれが、再生可能であるか、つまり、韻律を高めるために、延ばされた又は強調された発音、又は楽音が与えられ得ることを示している。たとえば、「ｗｈｉｓｋｅｙ(ウィスキー）」及び「ｈｕｓｋｙ(ハスキー）」においては、Ｓは再生されるが、Ｋは再生されない。Ｋは無音ではなく、休止又は一時停止を置かずに簡単に素早く発せられる。「ｅｎｓｎａｒｅ(わなにかける）」においては、第１のＮ及びＳは再生されるが、第２のＮ及びＲは再生されない。Ｃ及びＴのための１本の下線の後の別個のＮの下に「尾部」３４を有する２本の下線は、再生可能なＮがオーボエとして再生されるが、再生されるべきドラムビートの子音、この場合は二重ドラムビート子音対が続くことを示している。「ｄｉｓｍａｎｔｌｅ（取り外す）」のＴ上でマークされた上端部に球状部３８を有する前方斜線３６は、ＴＬが、「ウッドブロックのカチッという音」としては再生され得ないが、Ｌのための２本の下線で注記されているように、再生可能な子音Ｌが後に続く子音Ｔとして再生され得ることを示している。 It is desirable that the letter NG can be underlined twice if G does not represent an oboe followed by a drum beat. If some of the words that end with the letter NG have the same meaning as the whole word, it is considered that there is no drum beat after the oboe letter, as in the following example. That is,

FIG. 3 shows which of the consonant s-consonant consonants occurring within the various words is reproducible, i.e. extended or emphasized pronunciation to enhance the prosody, Or it shows that a musical sound can be given. For example, in “whiskey” and “husky”, S is played but K is not played. K is not silent and can be emitted quickly and easily without any pause or pause. In “ensnare”, the first N and S are played, but the second N and R are not played. The two underlines with a “tail” 34 under a separate N after one underline for C and T, the reproducible N is played as an oboe, but the drum beat to be played It shows that a consonant, in this case a double drumbeat consonant pair, continues. The forward diagonal line 36 with the spherical portion 38 at the upper end marked on the T of “dismantle” cannot be reproduced as a “wood block click”, but for the L 2 As noted in the underline of the book, it shows that a reproducible consonant L can be reproduced as a subsequent consonant T.

  As shown in FIG. 4, if NG represents an oboe and a drum beat or other percussion instrument, N is underlined twice at the tail 42 to indicate that N can be played as an oboe, G is underlined once, indicating that it needs to be played as a G timpani drumbeat percussion instrument for words that should be pronounced correctly and crisply. Also in FIG. 4, the length of the word is marked with two underscores and tails for N followed by G without an underscore, where N is reproducible as an oboe, but G is crisp This indicates that the “other percussion instrument” cymbal DG for a word that should be pronounced must be pronounced crisply.

  The drum beat can be marked as reproducible with a single underline before the consonants that are independent of the drum beat and are made and felt at different contact positions of the tongue by oral anatomy. Before an identical, synonymous or mid-related consonant made to feel almost in the same position, the drum beat is marked as “prepared” with a forward diagonal on the consonant. It is useful to obtain.

  Referring now to FIG. 5, the cymbals are “reproducible using a single underline under each alphabetic character of the cymbals before all other consonants except the same word and words of the same origin. Marked as "is." Thus, for example, the DS in “heads” can be played back in “heads back” but not in “heads south”. . In FIG. 5, a direct bond is marked with a bond symbol 40 as described above. Thus, the DS in “heads back” is indicated by a joint symbol 40 coupled to B, and the TS in “beats fast” is represented by a joint symbol 40 coupled to F.

  Referring now to FIG. 6, the wood block clicks DL and TL exclude the following initial letter L (because this L is in the wood block click) and other Before every consonant, it is marked as “reproducible” with two underscores. Therefore, for example, DL in “middle” can be reproduced in “middle school”, but not in “middle life”. As described above, the hammock 40 is marked to indicate direct coupling. The “o” marked at the top of the forward slash 36 indicates a special version of the “preparation” mark, used only for wood block clicks, indicating that consonants are prepared and coupled to L. In the case of the following L, this L is directly coupled to the following L, so the end of a wood block click may not be reproduced as a sustainable consonant L.

  Referring to FIG. 7, the combination of GL, KL, BL, and PL consonants is treated as representing a clicking sound of a wood block because there is an ambiguous (unwritten) vowel uttered between them. Note that it is desirable not to break. Thus, as shown, L is reproducible, but the consonants that precede it are not reproducible.

  W, H, and Y are preferably not marked as reproducible if they occur at a position that is considered reproducible for other child music instruments. This is because they are part of a vowel or a double vowel, as shown in the examples below.

本発明による、共通の組合せＷＨ中のＷ及びＨのための共に有用な表記法は、ＷＨの上に英字「ｈｗ」をマークして、Ｈがまず発せられ、次にＷが発せられ、いずれも再生されないことを示すことである。

A notation useful together for W and H in a common combination WH according to the present invention is to mark the letter “hw” on WH, H is emitted first, then W is emitted, Is also not to be reproduced.

  Referring to FIG. 8, if “Y” or “W” occurs in another word and between words and between words, the concatenated words 50 and 52 of Y and W are Created to indicate that speech continuity is maintained from one word to the next or from one syllable to the next. Examples of symbols used in the Y and W concatenations 50 and 52 in this embodiment of the invention are with lowercase Y or W marked in, near or on U, respectively. Each has a hammock-like shallow U-looping from below Y or W to the following vowels. U indicates the continuity to be maintained and the letter Y or W indicates the sound used whether or not this letter is present in the written text input. For example, Y is emitted between E and A of “create”, and W is emitted between U and E of “cruel”.

  Referring now to FIG. 9, as described in the co-pending patent application 10 / 334,658 by Addison et al., And more particularly in the Arthur Resack book, Identify several methods that are combined when one or more additional letters or phonemes in a consonant, and a word or phrase are uttered. FIG. 9 shows some examples of how the desired pronunciation of such combined words is shown graphically in accordance with the present invention.

  Three examples of spoken word combinations utilized in the Lesac speech system, namely so-called “direct combination”, “play and combine”, and “preparation and combination” are described in FIG.

  In direct combination, the last consonant of one word is directly combined with the first vowel of the next word. For example, “far above” is pronounced as one word “farabove”.

  In “play-and-link”, there are two adjacent consonants made at different locations in the mouth, such as “k” followed by “t”, the first word, In this case, “k” is fully “reproduced” (sounded or emitted). That is, it completes before proceeding to the second consonant, in this case “t”.

  At the same location in the mouth, such as “b” followed by another “b” or “p”, such as in the case of “grab boxes” or “keep back” If there are two adjacent consonants that are made close together, preparation and combination is used. In this case, a first consonant or “drum beat” is prepared. That is, it is not completed before proceeding to the second drum beat, which is made with a slight stroke.

  An example of the prosodic graphic notation employed to show the direct combination shown in the upper row of FIG. 9 will be clearer than the example shown, but at or near the end of a word. One or more of the letters and a letter 40 at the beginning or near the beginning of the next word usually joins below and between the joined letters. Direct combination indicates that the momentum spoken should be transmitted from one combined alphabet to the next combined alphabet without any interruption or pause or rest between words.

  In the example of reproduction and combination shown in the middle row of FIG. 9, the first consonant is reproduced, but the second consonant is not reproduced. Thus, the combination symbol 40 is combined with one or two underlines of the first consonant.

  In the example of preparation and combination illustrated in the lower row in FIG. 9, the forward diagonal line on the first consonant, which is prepared and combined with the combination symbol 40 with the second consonant, is used to Indicated. In addition, as described above, reproducible consonants are indicated by underlines.

  Now referring to FIG. 10 for the two prosodic graphical symbols shown, Example 1 is relatively simple and economical to implement, and Example 2 includes colloquial books and magazines, drama and It is more sophisticated designed to facilitate the creation of high quality synthesized speech output that is suitable indefinitely for other entertainment applications. The more detailed notation of Example 2 also reduces speaker-to-speaker variability that can occur when trained speakers are used, thereby promoting output consistency.

  The notation of Example 1 can be used without limitation for industrial applications such as voice communication with appliances, vehicles, manufacturing machines, low-end games, entertainment devices, and the like. Of course, any notation can be used for other purposes if desired.

  FIG. 10 illustrates Example 1 and Example 2 applied to the same text in alternating lines for side-by-side comparison. As can be seen by comparing the first two lines in FIG. 10 in order, in some combinations of “heads” and another combined word, marked by a forward slash 36 on the DS combination D. The additional preparation that is provided provides a more subtle and more appealing sound. In each case, continuity from "heads" to the following words is maintained, but in Example 2, D is heard more clearly because it is prepared according to additional marks. In the pronunciation of example 1 and below, D may disappear.

  Referring now to FIGS. 11 and 12, the present invention is a graphical symbol set that can be used to show or provide a template for attractive prosodic audio output having one or another significantly different style. Will be understood to provide and use. FIG. 11 shows the markup for rendering a portion of the “Prosody” style Gettysburg Address, called “Report Real”, where the style shown in FIG. It is an intriguing style for a traditional person.

  The text of the address is rendered in a text line, such as text line 10, on which a percussion instrument and a sustainable tone consonant, a wood block click, as described above, in a notation line, such as notation line 12. Marked using resac structure and tonal energy vowel pronunciation notation, and using consonant energy pronunciation notation, including marks for sounds and combined symbols. The text is also a prosodic graphical symbol as described herein above, including individual alphabetic underlines, ascending curves, descending curves, hammocks, etc., and is marked below the text so as not to interfere with phonetic notation. In addition, a so-called Y-buzz line (Y-buzz) 60 is added on top of the notation line 12 above, and a further prosodic symbol is marked on the Y buzz line 60 above. Y buzz is the basis of tonal vibrations that travel through the bones of the voice of a speaker or singer, as described, for example, on page 122 et seq. Of Arthur Lesack's book.

  The desired intonation pattern is a so-called prosodic pitch chart on the Y buzz line 60 using small dots 62 and large dots 64 at the level of the dots on the Y buzz line 60 indicating the desired pitch relative to the pitch of the speaker's Y buzz line. Marked with what is called. The dot size is used to indicate the desired stress or relative amplitude of the expressed relative pitch, with a small dot 62 indicating no extra stress and a large dot 64 indicating that an additional stress is desirable. Optionally, the dot size may indicate a desired degree of stress in proportion. Although it may be possible to utter a voice at a frequency below the Y buzz line 60, the voice tone and articulation control may be insufficient for a voice to be played as a controllable instrument.

  Further, in FIGS. 11 and 12, when a text is pronounced according to marking, an English letter B is marked in a circle indicating an error made by the voice practitioner B. Errors are determined by other voice practitioners listening to the recording of the pronunciation and noting where the pronunciation deviates from the mark to follow. For example, in FIG. 12, practitioner B made two pronunciations different from those necessary for marking. The first was a failure to pronounce the drum beat consonant D at the end of the word “engaged” (text line 4, word 4). The second was a failure in playing the rising curve of E and the drum beat of consonant D at the end of the word “dedicated” (text line 5, word 5).

  Additional emotions and energies conveyed by the intriguing markings shown in FIG. 12 can be easily obtained by carefully comparing the report realistic markings of FIG. 11 with the prosodic pitch chart markings. It becomes clear. For example, many of the stress dots in FIG. 12 are larger dots 64 that require greater stress or enhancement. Also, the ascending curve 20 above “ago” on line 1 rises above the Y buzz line 60, indicating the desired higher pitch. In the lowest line, the “conceeded” V is given an extra stress dot 64 and the circumflex symbol 24B is not used. If you examine several figures, you will see other differences.

  It will be appreciated from FIGS. 2-10, and in particular from FIGS. 11 and 12, that the present invention provides a comprehensive text marking system. This makes it possible to embody advanced pronunciation and prosodic symbols with normal text as an overwriting that does not divide or interrupt normal text, and a person who can accurately and understand the text as human voice or machine voice New instructions or control documents can be generated that will appeal to you and even provide a beautifully rendered blueprint of melody.

  To achieve the objectives of the present invention, other suitable graphical symbols for marking speech text will be apparent to those skilled in the art from the disclosure herein and suitable for practicing the present invention. It will be inferred. For example, various geometric symbols or geometric symbol schemes or animated graphemes may be used. However, simple symbols as described herein are intuitive and easy to apply when marking scripts or other text, and more importantly Is easily understandable when a trained speaker reads the marked text.

  The prosodic graphical symbols shown in FIGS. 2-4 and other figures, and described herein, can be used in various ways to promote human-like sound with synthetic speech output, particularly with formant speech output. Used in For example, a notation can include one or more, preferably multiple, trained to pronounce text correctly according to the marked phonetic code described herein to create a pronounced speech database. Used by humans. The database includes pronounced speech that is illustrated to accurately follow text marked with the phonetic code described herein. As an alternative or in addition, the prosodic graphical symbols of the present invention can be rendered digitally and machine-spoken text to facilitate or guide the introduction of the digital domain of prosodic elements into the output speech. Can be used in synthesizer software for electronic marking. Recorded speech databases that support graphic notations for letters, words, phrases, sentences, paragraphs, and longer text are digitized and analyzed to support specific text with associated graphic notations Obtain algorithms and other metrics to specify unique relationships of specific voice data. This can then be used to provide input parameters to the synthesizer to recreate a sound that mimics a person's speech for a particular text that is synthesized as speech with a specified prosody.

  To simplify the conversation, each text unit and the associated phonetic code notation are considered “graphemes”. Similarly, each acoustic unit corresponding to “grapheme” is identified as “phoneme”. An extended set of hundreds or thousands (where “number” means “at least two”), or more, grapheme / phoneme pairs for associating pronunciations for pitch, amplitude, And the prosody graphic notation of the present invention is a digitally rendered synthesizer for electronic marking of text uttered by a machine to facilitate or guide the introduction of the digital domain of prosodic elements into the output speech Can be used in software.

  Those skilled in the art will appreciate that the specific prosodic graphical symbols shown in FIGS. 2-4 are merely exemplary, and prosodic graphical symbols useful for practicing the invention in accordance with the teachings herein are many other forms. It will be understood that In addition, the particular example of symbols shown is adapted for a resack speech system. If desired, other prosodic graphical symbols may be used to implement other voice guidance or training methods, or to implement resack systems to facilitate making machine speech human according to the present invention. Used. This will be understood by those skilled in the art from the teachings herein.

  In accordance with one embodiment of the present invention, guidelines such as those described in the following paragraphs can be observed in preparing prosodic markup as shown in FIGS.

Marked Script Preparation When preparing a marked script to be spoken as illustrated in FIGS. 11-12, a good page layout is beneficial to the speaker, and according to the present invention, When guiding a speaker to a consistent and attractive voice output, it is easy to simultaneously understand additional symbols as well as text, which is useful when creating a database and is also a computerized voice. It may also be useful for the synthesis of

  The text is preferably well spaced on each line, for example 3 cm or more, to accommodate the phonetic notation and prosodic graphical symbols to be added. The relatively large font is useful to accommodate a variety of notations and symbols, and facilitates accurate reading and interpretation of the mark while speaking aloud. A font such as a 14 point Lucida luminance semi-bold is an example of a suitable font.

  In one embodiment of the present invention for marking a script, each line of the script, including the last line of the page, ends with a vertical mark for a break or breath. A number is written. If the acronym is spoken as a word rather than an English letter, it is all written. Usefully, reference dictionaries for consistent pronunciation, such as the Merriam Webster's Collegiate Dictionary, 10th edition, are pointed out.

  The reference dictionary presents pronunciation choices, but the first pronunciation listed is used when the recording is not authoritative or “high”. In this case, when listed, “abbreviated” pronunciation is used.

  The intonation pattern is a general movement for each stepped pitch in the connected voice. An inflection is a so-called change in pitch of a vowel or consonant, such as an ascending curve, a level sustaining part, a descending curve, or a bent accent symbol.

  Usefully, in this embodiment of script preparation, the speaker will explore the script aloud to obtain intonations and intonations that convey the meaning of each sentence.

  Two vertical lines are marked on the script to indicate breath pauses, and a single vertical line is marked to mark the break when there is no pause affecting the handling of the last consonant of these segments. Show.

  In one useful embodiment of the invention, after these preparations, consonants are first marked and then vowels are marked for pronunciation and prosody as described herein above.

  The next step is to draw the Y buzz pitch line 60 directly above the vowel without hiding the vowel and provide a mark reference for the pitch. The pitch range to be noted is from the low Y buzz range (hereinafter Y buzz line 60), the vowels that are not deformed are mid-call, for example, # 3, R, and + Y, and optionally in # 4 and N Up to the mid-call range that can be spoken at.

  If desired, additional pitch lines (not shown) may be drawn on the Y buzz line 60, such as the lower middle range line and just the middle range line. Intonation dots 62, 64 or other suitable marks can be placed both above and between pitch lines. If desired, other methods for providing pitch ranges may be used, as described, for example, in Daniel Jones's book, “Outline of English Phonetics”. Here, a 3-line score is used to define the pitch range.

  In order to practice the illustrated embodiment described herein, the audio system practitioner should be a Lessac practitioner and then Y as a guide for his own pitch range. Buzzline 60 is used to record what is recognized as the desired intonation and intonation according to each audio system and the desired prosody. As mentioned above, a dot can be marked for every syllable. That is, small individual dots such as 62 are used for syllables that are not emphasized, and very large dots 64 are used for emphasized syllables.

  For example, for the intriguing recording illustrated in FIG. 12, voice features such as those known as “structural NRG” and “concentrated tone” are more abundantly used in the Lessac system, with larger dots Other suitable graphical displays to indicate that the call focus is used, if desired, to words that include a lessac basic call word, using a circle around or without vowel deformation Can be marked using. The terminology used in this paragraph relates to the Lessac speech or voice system and will be understood from the Arthur Lessac book.

  For example, as described in that book, the structure NRG is related to facial expression and the kinetic sensation voice energy (“NRG” by Lesac) related to the type, shape, and size of the voice resonance chamber. It is a state. The structure NRG is thought to be related to the beauty of color, body temperature, and voice tone.

  A valid word can be marked with a double accent mark, for example “before the primary stress syllable while a secondary stress syllable can take one accent mark, for example”.

  In this case, a valid word is a word in each successive phrase or other textual grouping that introduces a new concept of carrying arguments forward as the sentence progresses.

  In accordance with this aspect of the present invention, if careful and consistent voice recordings can be made in a text / speech synthesis database, each “sensory group” word in the text break and breath break split is identified. It is desirable to have valid words. In rare cases, there can be two valid words that are equally important.

  Valid words can be used in various ways, such as marking them to be uttered at higher pitches, or by making their vowels and consonants substantially longer, using the method of strength of the voice of Lesak training It can be identified by adding concentrated tone, call resonance, or a combination of these strengths.

  In one exemplary embodiment of the present invention, the introduction of arguments begins with a plain text in which all but the consecutive words are marked as having approximately the same amount of stress or emphasis. It may not be done. The pitch can rise with the first emphasis word and the rest can step down to a Y buzz range with a down curve on the last stressed syllable.

  For example, in a script that is marked for speech according to the present invention, various inflections can be used with punctuation marks. This will be described below. A period and a semicolon can take the descending curve 22 on the last highlighted word. A comma and a colon can take the ascending curve 20 or the level sustain 26. Questions that start with a question word (eg, who, what, where, when, how, or why) take a descending curve 22 over the last highlighted word, and other questions, usually “yes” or “no” The question that predicts the answer takes an ascending curve on the last highlighted word.

Other Voice Training Systems As will be appreciated by those skilled in the art, the claimed invention may be practiced in embodiments that employ rules or voice training principles or practices other than the Resack method. One such example is the method of Kristin Linklater at Columbia University Theater Department. For information on Kristin Linkator's technology and, if desired, other voice practitioner's technology in areas where the rules may be used in practicing the invention,
www. columnia. edu / cu / news / media / 00 / kLinklater / and www. kristinlindata. com
Can be found.

Prosodic speech rules and their application forms The prosodic speech rules that can be used in the present invention are for explaining the articulation and simultaneous articulation of languages and their various wordings. Examples of programming languages mentioned in this specification are American English and general well-known language. It will be appreciated that other languages may be used. Prosodic phonetic rules, at least some of which can be derived from the Lesack text, are applied to the text and are pronounced or synthesized using the new acoustic codes described herein, familiar with the Lessac system Speaker can read the text aloud with controlled pronunciation determined by the appropriate resack.

  Examples of rules include the use of random pauses described in one or more of the commonly owned applications and modified by breath pauses, rhythms, intonation patterns, word stresses, word selections, and consonant “ Prosody definitions incorporating “mixed words” are included, all of which are derived directly from the pronounced text. These prosodic phonetic rules can be adapted to other wordings and languages.

  The acoustic markup code used in the present invention may indicate how specific voice sounds are created and which voice variables are used to create these sounds. The text to be pronounced is for a speaker of a person trained to follow the code when speaking the text, along with an optional specified value or values for one or more variables of each code It can be useful as a prosodic indication. In accordance with the present invention, the computer synthesizer is instructed to pronounce text according to prosodic instructions using the same or similar such code variables, or their machine equivalents. The code that controls the creation of sounds and variables indicates quantifiable identification information related to the desired sound characteristics.

  Examples of speech variables that can be coded according to the method of the present invention include the audible frequency, amplitude, pitch, and duration of a sound element that is synthesized to represent a particular phoneme or other speech element. Some examples of specific variables that can be quantified to a desired value include the basic voice frequency, the upper and lower limits of the controllable pitch range, and the pitch expressed as a change in frequency per unit of time. There are changes, changes in amplitude per time unit, and combinations of amplitude and pitch change per time unit.

  An example of one useful relationship between a voice rule, an acoustic markup code, and one or more variable values will now be described. Other possibilities will be apparent to those skilled in the art. Both breaks between words and occurrences of commas in the sentence represent pauses in creating a sound sound. However, each type of pause has different characters that can be indicated by different acoustic codes according to one embodiment of the present invention. Pause is useful for the listener and facilitates the recognition of individual words and provides separation that aids phrase identification. Similarly, each pause has a variable time, but in general the time value or duration measured in milliseconds of relative silence between the sounds that make up the pause may vary from situation to situation. If the written text has no commas, pauses between words are part of the tempo of the speech, and rhythmic pronunciation of crisp pronunciation words that delimits each pause and is contained within a complete phrase. It may be determined by the overall speed of the sound and the rhythmic variation required for the sound.

  Thus, pauses may be determined in context by phonetic prosody, such as excited, serious, report-real, poetic, persuasive, or other prosody, corresponding to this The commas in the text indicate phrase separation, and the corresponding pause duration or the time when there is no speech when the text is spoken can vary from speaker to speaker, according to prosody and other factors. In natural human speech, this pause is not a single value but has a greater or lesser length of time variability, sometimes providing a fresh breath and and additional emphasis, or It has other time to serve as a point / opposite point for the rhythm of the entire paragraph containing the sentence and its phrase. In the case of paragraph machine voice rendering, the changing pause duration of a person's speaker is rendered as a constant millisecond value, and the resulting voice may be perceived as a machine rather than a person. This is because people tend to change the length of pauses between phrases.

Cryptographic Marking Procedure Example To create acoustic symbol sets and provide acoustic data for accurate pronunciation according to the rules for preparing examples of prosodic acoustic libraries, a team of four certified Lesack practitioners Work with 1,000 most frequently used words and 500 phrases and sentences in American English.

  The practitioner reviews and refines the prosodic speech rules used. It is desirable to be able to develop marking instructions and notation for each of the rules. A notation for prosody can also be developed. The rules are then applied to the sample words and sentences.

  In an exemplary embodiment of a script marking procedure according to the present invention, each speech practitioner uses words and sentences in a script formatted as described herein for pronunciation by prosodic speech rules. Mark Useful, the script may have at least about 1,000 words and 500 phrases that broadly represent the language of the text to be converted to speech. If desired, the words and phrases in the script are limited to a specialized subset of the language, such as a specialized subset such as medical, scientific, or local dialects. Each practitioner's markup is then examined by another team member to identify errors in applying prosodic speech rules. A final, coordinated final markup of 1,000 words and 500 phrases and sentences without errors was prepared.

  Using collated final marking, it is desirable for each practitioner to read aloud word and sentence sampling from the marked script. One or more of the other practitioners listen to the pronunciation and look for errors regarding whether they are following prosodically marked text. Using this technique, one or more speakers can be rehearsed before a recording session or other verbal presentation.

  To prepare a recording useful for preparing an acoustic database for use in speech synthesis in accordance with the present invention, word and sentence scripts that are pronounced in a studio recording session are shown in FIG. Line reports are marked with realistic prosody. Each practitioner receives a final collated copy of the report-real script along with a mark of the same sentence subset with the mark for the second prosody.

  For recording sessions, practitioners use studios with “dry room” recording environments, preferably studios that meet the exact drawn criteria for analog / digital sampling rates and sound quality. In a studio session, an acoustic CD or other analog recording of each practitioner's pronunciation is prepared along with a data CD or DVD that captures the recorded pronunciation as a WAV or other data file.

  To ensure the quality of the data, each practitioner's audio CD is provided to another practitioner who listens to the pronunciation and does not follow that marking of a copy of the complete correct markup Be aware of pronunciation errors. If an error is noticed, the pronunciation is removed from the WAV database, and it is desirable that the database contains only the correct articulation, intonation, and prosodic elements.

  By following such marking, utterance, and recording procedures, a digitized database library is provided that is relatively error-free of speech elements, including a language or language that complies with the pronunciation and prosodic rules entered. Phonemes, words, phrases, and sentences of a subset of A certain degree of consistency is possible so that a speech element library prepared by one group of practitioners can be compared to a similar library prepared by another group of similarly trained practitioners.

Prosodic Acoustic Library For prosodic speech rules that are effectively applied to computerized speech, the present invention provides specific text that each prosodic speech rule is pronounced and speech data corresponding to it when pronounced correctly. A graphic symbol set that is unambiguously connected to the object is provided. A unique prosodic acoustic library is prepared for each language and the most widely used wording. Each unique prosodic acoustic library includes a comprehensive dictionary, prosodic phonetic rules, graphic markup symbols representing the rules, and examples of phonetic data for pronunciations that follow the rules correctly, along with text for pronunciation examples It is assumed. A comprehensive prosodic acoustic library for specific languages and wordings is the basis for deriving and thus specifying formant parameter values for articulation that are uniquely associated with the applied prosodic speech rules.

Examples of prosodic acoustic library databases according to one embodiment of the present invention include: That is,
a) Selection of text words and sentences representing text to be synthesized into speech.

b) A set of rules for computerized marking of text for easy pronunciation. These may include consonants, vowels, simultaneous articulations, and pause rules.

c) Prosodic rules for the two prosody described herein, “report real” and “intriguing”. These prosody are marked, pronounced and included. These rules specify values that change over time, such as changes in pitch, volume, rhythm, speaking tempo, and word stress.

d) Coordinated manual markup copy of the prosodic phonetic rules applied to the text.

e) Pronunciation of the marked text. An example of a WAV data file of four practitioners representing all words and sentences pronounced in a “report-real” prosody, and several sentences pronounced in a “intriguing” prosody.

  The example of the prosodic sound library database structure preferably includes WAV data, text, graphics, and numerical data. Software statement examples, source code modifications, and synthesizer specification values may also be added. An example of a prosodic acoustic library database may include about 8-12 gigabytes of data. Commercially available relational databases for standard mass-produced products cannot currently combine WAV data with text, graphics, audio CDs, and numerical data. Thus, the present invention can use a temporary database structure to validate product designs for combining WAV data with text, graphics, and numerical data. Software statement examples, source code modifications, and synthesizer specification values may also be added. An example LAL database may include about 8-12 gigabytes of data. If desired, the architecture for assembling, storing, and processing database components can be improved in view of the results of using temporary structures. This can be useful for building a comprehensive database library containing text, graphics, sound, and numerical data.

Software Examples A known speech synthesizer or synthesizer engine may have: That is,
Scanner and associated software and hardware for creating text input means, for example, one or more data files, text data usable in a suitable form of system.

  Data processing unit and associated data memory for performing software and effect speech synthesis operations.

  Speech synthesis software that can be implemented by the data processing unit. The software can also be a software engine for converting text data into speech data.

  Audible output means, for example, an audio port that can provide an audio signal to a loudspeaker or headphones, and associated hardware and software for finally outputting audio data received from speech synthesis software in audio form.

It is understood that the audio can be stored, communicated, or distributed as an audio file, eg, a wav file, for playback at one or more times after synthesis, if desired. Like.

  Traditionally, such known speech synthesizers have been developed to implement a unique limited set of linguistics and synthesis rules, but are assembled from small speech components such as phonemes, words, or short phrases. If so, their output is mechanical, inhuman, and unreputable. The present invention may be added and / or used to implement the teachings herein using the new text markup symbols and prosodic markup of the present invention to provide a human-sounding, human-like audio output. A new speech synthesizer and speech synthesis software is provided that uses source code adapted to implement alternative linguistic rules. According to the present invention, the speech synthesis software can specify the sound output value of a suitable speech synthesizer that creates a sound corresponding to the phonetic notation and the pronunciation rules applied to the text indicated by the graphical symbol, It can be marked to determine the pronunciation and prosody of the output speech.

Software Examples One or more persons familiar with formant text / speech ("TTS") engine software that are suitable for practicing the invention, adapted for the purposes of the invention described herein. May be provided by a person skilled in the art, for example a technician and / or a computer linguist. Suitable additional linguistic rules and synthesizer signal specifications can be added to known speech software engines to build and test software that embodies or implements the present invention. For example, the sample prosodic acoustic library database described herein is analyzed to correspond to the phonetic markup symbols of the present invention for simultaneous articulation not currently specified in known formant TTS synthesizers and corresponding The WAV data to be isolated can be isolated and the necessary elements can be added to a known synthesizer.

  The resulting one or more speech synthesis software programs can be used for the machine generation of human speech or graceful speech from text, in addition to resack or other implementations implemented in the software. It may be useful to educate software engineers etc. to practically understand the speech training system and the new prosodic speech rules used. This may also identify items that are programmed to accommodate the desired additional linguistic, phonetic and prosodic rules and new acoustic signal parameters described herein.

  An example software may be programmed manually to mark the text and specify the associated speech values for synthesizer sound creation and write manually. Once such a sample has been created, the specific text will be used in the context of the specific word, sentence, and phrase to directly use the synthesized text as input and to specify the hybrid formant and concatenation parameters and values. Larger dictionaries can be automatically programmed using a computer system that applies the rules that require. Formant parameter values are specified for text marking that operates according to marked specific pronunciation and / or prosodic rules and according to the voice identification information characteristics, fundamental frequencies, harmony, etc. to be output. This is the value required to produce pronunciation and prosody.

Test by Listener The present invention contemplates using the test of synthesized speech output by the listener in accordance with the present invention to provide feedback to improve the product. For listeners, to recognize increased clarity and comprehension of messages, and whether the sample sounds better than conventional comparison products such as Sensimetics HLSYN® or SENSYN® formant synthesizers A vote can be made to determine whether. For recognition, comprehension, and preference measurements, it is desirable to use validated experimental design and data collection techniques that may be known in the industry.

  As will be apparent from the foregoing description, the spoken text will show a graphical symbol indicating the pitch control required for pronunciation of English letters, diphthongs, syllables or other speech elements according to the desired prosody, and an increasing pitch. A rising curve, a falling curve indicating a descending pitch, a bent accent symbol indicating a rising and descending pitch or a descending and rising pitch, a level continuation indicating an invariant pitch, and the first alphabetic character are prepared The first oblique letter followed by a closely related or identical sound consonant, and the letters separated from each other by other letters are pronounced consecutively without any pauses Shallow U-shaped liaison hammock with timpani drum beats, D, B, and G, and snare drum, bass, and Tom Tom drums, respectively A single underline marking as a reproducible percussion instrument including notes, T, P, and K. Unmarked consonants are not reproducible and reproducible stringed instruments N, M, V, and Z , Woodwind instruments, L, NG, TH, and ZH, and two underlines marked as (silent) acoustic effects F, S, SH, and th, and uncontained consonants are not reproducible; The letter combination “hw” marked on or adjacent to the letter combination WH in the spoken text, indicating that H should be uttered first, then followed by W and neither will be played. And "Y" or "W" occurs before another vowel, the continuity of speech from one word to the next or from one syllable to the next should be maintained This is a concatenated word of Y and W indicating that Each of the words has a hammock-like shallow U looping from below Y or W to the following vowel, respectively, with a small Y or W letter marked in, near or on U, respectively. May be marked with one, two or more, or all of the prosodic graphical symbols selected from the group consisting of: Here, the consonant before the vowel has a rule that it is uttered but not marked as reproducible.

  As an alternative or in addition, the spoken text should have the momentum spoken proceed from one combined alphabet to the next with no interruption or pause or break between words A direct combination with a liaison hammock-like line under and between the combined English letters, indicating that the first consonant is played and the second consonant is not played. Playback and combination with a liaison hammock combined with one or two underscores, and a liaison hammock with a second consonant prepared and indicative of a coupling between the first and second consonants One of the combined prosody graphical symbols selected from the group consisting of a preparation and combination having a forward diagonal line on the first consonant that is a consonant, the reproducible consonant being underlined Two or more, or may be marked using all.

  Several possible embodiments of marking instructions are described herein that may be used in the present invention, if desired, included in a prosodic acoustic library database. In accordance with the present invention, the novel acoustic value codes or graphical symbol sets and notations described herein are apparent from the disclosure herein, but may be used for the purposes of the present invention or It should be understood that this is merely an example of code that can be devised.

  Further, although examples of acoustic value codes are described in the context of an English language, the present invention uses consistently modified or adapted phonetic rules as needed for another language. It will be appreciated that certain pronunciation markings encompass other coding systems designed for the specific needs of other languages that embody the general principles herein. Thus, the method of the present invention is, for example, English, American English, French, Spanish, German, Japanese, Russian, Chinese, Arabic, Hindi, and graphic symbols. A language selected from the group consisting of a sentence and colloquial language having a grammar based on sets and rules, and any one wording and specialized subset of the aforementioned language or other languages, subsets, or wordings Can be implemented. This will be apparent to those skilled in the art from the teachings herein.

  Lessac or other phonetic training rules are particularly effective in pronunciation to facilitate understanding of individual English letters and relatively small phonetic elements having combinations of two or three English letters and are described herein. Prosodic rules that are commonly used to play, pause, stress, and other prosodic techniques in such a letter or combination of letters in the context of a larger speech element that has an entire word, phrase, sentence, or paragraph It will be appreciated that it is useful to apply

  In summary, the present invention provides a clear template for clear and appealing speech output that is clear and appealing to the human mind when applied to text uttered by a skilled speech practitioner. A graphically displayable global rule set is provided that has pronunciation rules for ease of understanding and prosodic rules for rhythms and melodies.

Incorporated disclosure of each and every U.S. patent and patent application, each foreign and international patent application, each other published and unpublished patent application mentioned in this specification or elsewhere in this patent application. The entire disclosure is incorporated herein by reference.

  Although exemplary embodiments of the present invention have been described, it should be understood that many and various modifications will be apparent to those skilled in the art and may become apparent as development progresses in the art. Such modifications are intended to be included within the spirit and scope of the invention disclosed herein.

FIG. 6 shows some words and phrases marked using a Lesack phonetic notation for structured NRG vowels. FIG. 5 illustrates a sample prosodic graphical symbol useful for showing a desired pitch change in text, eg, a continuous tone pitch change pattern in a prosodic intonation pattern related to text, according to an embodiment of the present invention. FIG. 4 shows a sample prosodic graphical symbol useful for showing a desired pronunciation of a consonant in a mixed word of consonant s, according to an embodiment of the present invention. FIG. 6 illustrates a sample prosodic graphical symbol useful for showing a desired pronunciation of a consonant having a mixed consonant word that is “oboe” followed by “percussion instrument” in accordance with an embodiment of the present invention. FIG. 5 shows a sample prosodic graphical symbol useful for showing the desired pronunciation of a consonant combination of a percussion instrument having “cymbals”. FIG. 6 shows a sample prosodic graphical symbol useful for showing a desired pronunciation of a consonant combination having a “wood block click”. FIG. 5 shows a sample prosodic graphical symbol useful for showing a desired pronunciation of a consonant combination with ambiguous vowels between consonants. FIG. 6 shows a sample prosodic graphical symbol useful for showing a desired pronunciation of a consonant combination having a concatenated word of Y and W. FIG. 5 is a diagram showing sampling of prosodic graphical symbols useful for articulating a point of articulation when combining words in order, in this case based on a short phrase, useful for indicating a desired pronunciation. FIG. 6 shows the use of two examples of prosodic graphic notation according to the present invention using word stress and intonation patterns for a specified prosody, in this case a “reportial” prosody. FIG. 11 is a diagram illustrating one sample of markup using both the Lesack phonetic notation and the prosodic figure notation illustrated in FIGS. 2-10 in a report-real style. FIG. 11 is a diagram illustrating another example of marking using both the Lesack phonetic notation and the prosodic graphic notation illustrated in FIGS. 2-10, which is a style that is intriguing to people.

Claims (17)

A method of marking the text used when synthesizing speech from text,
Marking the text to be spoken with one or more graphical symbols that indicate to the speaker the desired speech characteristics to be used in uttering the text,
A method of using as a graphic symbol acoustic code indicating a desired prosody given to the text uttered by the speaker.   The assigned prosody is selected from the group consisting of tempo, intonation pattern, rhythm, musicality, amplitude, pause for emphasis and breathing, and formal and informal articulation of words and phrases; The method of claim 1, comprising one or more prosodic elements. Marking the visible text with a graphic prosodic symbol or electronically marking the electronic text with an electronic version of the graphical symbol,
The electronically marked text can be displayed or printed as human-readable graphic marked text and communicates the desired prosody to the speech synthesizer in a manner that can control the prosody of the output speech 3. The method of claim 2, wherein the method is effective to do so.   The text to be uttered includes a graphical symbol that indicates the pitch control required to pronounce a letter, diphthong, syllable, or other speech element in the desired prosody, a rising curve that indicates a rising pitch, and a falling pitch. Closely related, showing the down curve shown, the curved accent symbol showing the pitch that goes up and down or the pitch that goes down and then rises, the level duration that shows the unchanged pitch, and the first letter is prepared Shallow to indicate that the front diagonal line on the first alphabet followed by a consonant of the same or the same sound and the letters separated from each other by other letters are continuously pronounced without any pause between the letters U-shaped liaison hammock, timpani drum beats, D, B, and G, and snare drum, bass, and Tom Tom drum beats, T, P, and A single underline for marking a reproducible percussion instrument including a non-marked consonant cannot be displayed, and reproducible stringed instruments N, M, V, and Z, a woodwind instrument, L, NG , TH, and ZH, and (silent) acoustic effects F, S, SH, and two underscores marked as th, unmarked consonants cannot be displayed and H should be emitted first. A letter combination "hw" and "Y" marked on or adjacent to the letter combination WH in the text to be spoken, which is followed by W and none will be played Or if “W” occurs before another vowel, Y indicates that speech continuity should be maintained from one word to the next or from one syllable to the next The connected word of W and the connected word of Y and W are respectively With a small hammock-like U looping from below Y or W to the following vowels, respectively, with a small Y or W letter marked in, near, or on U, respectively. With one, two or more or all of the prosodic graphical symbols selected from the group having the rule that consonants before the vowel are spoken but not marked as reproducible The method according to claim 1, 2, or 3.   Marking the text to be spoken means that the momentum to be spoken should go from one combined alphabet to the next without any interruption or pause or break between words 1 of the first consonant indicating that the first consonant is reproduced and the second consonant is not reproduced, and a direct combination having a liaison hammock-like line under and between the combined alphabetic characters indicating A liaison hammock with a liaison hammock combined with one or two underlines, and a liaison hammock prepared and indicating a coupling between the first consonant and the second consonant Using one, two or more or all of the prosodic graphical symbols selected from the group consisting of preparations and combinations having forward diagonal lines on the first consonant that are consonants Is to be click method according to claim 1, 2, 3, or 4, characterized in that the renewable consonants are indicated with an underline.   Placing the prosodic graphical symbol adjacent to the text to be uttered; adjusting the prosodic graphical symbol in a line immediately above the text; selectively placing the prosodic graphical symbol below the text; 6. A method according to claim 1, 2, 3, 4, or 5, characterized in that the prosodic graphical symbol is placed above and below the text.   Marking the text to be spoken by rendering the text in a line; marking the phonetic notation to facilitate understanding on the text; and prosodic graphical symbols below the text Mark a pitch reference line on the phonetic notation line to facilitate understanding, and mark additional prosodic symbols on the pitch reference line to indicate the desired pitch change and enhancement 6. The method according to claim 1, 2, 3, 4, or 5.   Using the pitch reference line to have a Y buzz pitch line and using smaller dots that show less stress and larger dots that show greater strength, the desired intonation pattern is placed on the Y buzz pitch line. 8. The method of claim 7, wherein the dots are positioned at a level above the Y buzz pitch line that indicates a desired pitch relative to the speaker's Y buzz pitch line. A method for automatically applying prosodic marks to text,
6. The method of claim 1, 2, 3, 4, or 5, comprising identifying, marking, and indicating a desired prosodic pronunciation using at least one computational linguistic algorithm. The method described.   Generate values for acoustic variables that can be used to specify input to a speech synthesizer to output the marked text as synthesized speech using chord variables corresponding to the desired pronunciation sound The method of claim 9, comprising:   The method of claim 1, further comprising using an acoustic library having audio elements recorded in a digital manner, which are audio elements uttered using the prosody indicated by the graphic symbol mark. Or the method according to 5.   English, American English, French, Spanish, German, Japanese, Russian, Chinese, Arabic, Hindi, sentence language with grammar based on graphic symbol set and rules, and 12. The method of claim 11, wherein the method is implemented in a language selected from the group consisting of colloquial language and any one wording and specialized subset of said language.   Using the prosodic graphical symbol to mark the text using the prosodic graphical symbol to create a database of pronounced speech, including speech that is correctly pronounced according to the text marked with a speech code The method of claim 1, 2, 3, comprising promoting synthetic speech output, optionally human-like sound in formant speech output, by one or more trained people to pronounce the text correctly. The method according to 4, or 5.   Rendering the prosodic graphical symbols digitally and using the graphical symbols in synthesizer software for electronic markup of machine-spoken text, thereby providing a digital domain of prosodic elements to the output speech 14. Method according to claim 13, characterized in that the introduction is facilitated or guided.   Recorded speech corresponding to the graphic notation for text having one or more of letters, words, phrases, sentences, paragraphs, and longer text is digitized into a database and the unique text and The method of claim 14, wherein the method is analyzed to provide an algorithm or metric for specifying the relationship of specific audio data corresponding to the graphical notation related to text.   Using one or more of the provided algorithms or metrics, input parameters to the speech synthesizer to recreate a sound that mimics human speech for specific text synthesized as speech using a specified prosody 16. The method of claim 15, wherein: A speech synthesizer controlled by an acoustic coding variable input to the speech synthesizer,
The acoustic coding variable corresponds to a prosodic specification used to generate a recorded person's voice having a desired prosodic pronunciation, and uses the recorded person's voice to realize the desired prosodic pronunciation A speech synthesizer characterized in that a synthesized speech output is provided.

Images