JPH11344990A - Method and device utilizing decision trees generating plural pronunciations with respect to spelled word and evaluating the same - Google Patents

Abstract

PROBLEM TO BE SOLVED: To automatically generate pronunciation of a spelled word by generating the second set of the voice phoneme provided with a score indicating the voice phoneme of an input sequence. SOLUTION: In the second stage of a pronunciation system, a mixed tree score evaluating part 20 assesses the posibility of continued existence of respective pronunciations of a list 18. A score evaluating part functions by successively investigating respective characters being in the input sequence together with phonems assigned to individual characters by a sequence generating part 16. Then, the evaluating part 20 rescores respective pronunciations being in the list 18 on the basis of questions of the mixed tree and by utilizing probability data being in leaf nodes of the mixed tree. A selection part module 24 may access a list 22 in order to draw out one or more pronunciations. Tipically, the selection part 24 draws out a pronunciation having the highest score as an output pronunciation 26.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to general language processing. The invention particularly relates to a system for generating spelling pronunciations. The present invention provides voice recognition,
It can be used in a variety of applications, including speech synthesis and dictionary editing.

[0002]

BACKGROUND OF THE INVENTION Spelled words with pronunciation occur in various situations in the field of language processing. In speech recognition, before use, each term in the dictionary must be converted to speech to learn (educate) the recognition unit. Traditionally, the switch to speech has been
It is manually generated by a dictionary editor who is skilled in the minute differences in the phonetic pronunciation of a specific language of interest. Converting each word in the dictionary to good speech is time consuming and requires a great deal of skill. If there is a reliable system that can convert words to speech based on the spelling of words, most of this effort and specialized expertise may be unnecessary. Such a system can extend the current recognition system to recognize words that are not currently found in existing dictionaries, such as geographical locations and surnames.

[0003] Spelling often appears in the field of speech synthesis. Today's speech synthesizers convert text to speech by searching digitally sampled phonemes from a dictionary and connecting these phonemes to form a sentence.

As the above example shows, both the speech recognition and speech synthesis fields of linguistic processing benefit from being able to generate accurate pronunciation from spelled words. However, the need for this technology is not limited to the language processing field. Dictionary editors today have completed fairly large and accurate pronunciation dictionaries for many of the major world languages. However, there are still hundreds of regional languages without good phonetic transcription. Until now, the work of creating high-quality phonetic transcriptions was manual, so it would take years for local languages to be represented. The notation process can be greatly enhanced if there are good computer-appropriate techniques for assessing the accuracy of the notation. Such a rating system uses an existing linguistic notation collection that identifies uncertain heading entries in the notation prototype. This greatly increases the speed of generating high quality notation.

Many attempts to convert spelled words to phonetic notation have relied solely on the characters themselves. These techniques have many problems. For example, it is very difficult for a generator that generates a pronunciation from only characters to properly pronounce the word “Bible”. Based on a character-only sequence, pronunciation-only-from-letter systems tend to pronounce the word "Bib-l", much like many first-year students learn to read. A disadvantage of conventional systems is the inherent ambiguity that many language pronunciation rules impose. For example, English has hundreds of different pronunciation rules, making it difficult to approach problems on a verbatim basis, and using a computer would be costly.

[0006]

SUMMARY OF THE INVENTION The present invention looks at the problem from different angles. The present invention utilizes a specially constructed mixed decision tree that includes both character sequence decision making rules and phoneme sequence decision making rules. In particular, a mixed decision tree includes a series of yes / no questions located at internal nodes of the tree. These questions may include questions about the characters in the spelling sequence and their proximate characters, and may also include questions about the phonemes in the word sequence and its proximate phonemes. The internal node ultimately leads to a leaf node that contains the probability data around which a given character is most likely to be phonetically appropriate when pronouncing the word defined by the character sequence.

The pronunciation generation unit of the present invention uses this mixed decision tree to score various pronunciation candidates, and selects the best candidate as the best pronunciation for a given spelled word in the tree. Let Generating the best pronunciation is preferably a two-stage process in which a tree of letters only is used in the first stage of generating a plurality of pronunciation candidates. These candidates are then scored using a second stage mixed decision tree that selects the best candidates.

While the mixed decision tree is used with advantage in the two-stage pronunciation generator, the mixed tree is useful in solving problems that do not require the processing of the first stage of only characters. . For example, a mixed decision tree may be used by linguists to score pronunciations generated using manual techniques.

For a more complete understanding of the present invention, objects, advantages and references of the invention may be set forth in the following specification and accompanying drawings.

[0010]

DESCRIPTION OF THE PREFERRED EMBODIMENTS To illustrate the principles of the present invention, the exemplary embodiment of FIG. 1 shows a spelled character to pronunciation generator. As will be described more fully below, the mixed decision tree of the present invention can be used in a variety of different applications, not just the pronunciation generator shown here. The pronunciation generator is chosen for explanation because it emphasizes many aspects and advantages of the mixed decision tree structure.

The sound generation unit uses two stages. The first stage uses a set of character-only decision trees 10 and the second stage uses a set of mixed decision trees 12. An input sequence 14, such as the sequence of characters “BIBLE”, is provided to a dynamic programming phoneme sequence generator 16. The sequence generation unit uses the character-only tree 10 to generate the pronunciation list 1
8 is generated to indicate possible pronunciation candidates for the spelled word input sequence.

The sequence generator sequentially examines each character in the sequence and utilizes the decision tree associated with that character to determine the character for that character based on data that may be contained in the character-only tree. Select phonemic pronunciation.

Preferably, the set of character-only decision trees includes a decision tree for each character in the alphabet. FIG. 2 shows an example of a character-only decision tree for character E. The decision tree includes a plurality of internal nodes (indicated by an oval in the figure) and a plurality of leaf nodes (indicated by a rectangle in the figure). A yes / no question is located at each internal node. The yes-no question is
A question that can be answered with yes or no. In a character-only tree, these questions are addressed to the given character (in this case, the letter E) and to the characters in proximity to the given character in the input sequence. In FIG. 2, each internal node branches left or right depending on whether the answer to the relevant question is yes or no.

In FIG. 2, the abbreviations are used as follows. A number in the question, such as "+1" or "-1", indicates the relative position in the spelling for the current character. For example,
"+ 1L == 'R'?" Means "(in the present case, the character E
Is the letter after the current letter R? The abbreviations CONS and VOW indicate the character type, ie, consonants and vowels. Lack of adjacent characters, ie, null character
letter) is indicated by a symbol "-", which is used as a filler or a placeholder when arranging characters and corresponding phoneme pronunciations. The symbol # indicates a word boundary.

[0015] Probability data is associated with the leaf nodes to associate a possible phoneme pronunciation with a numerical value representing the probability that a particular phoneme indicates the proper pronunciation of a given character.
For example, the notation “iy = 0.51” means “the probability of the phoneme 'iy' of this leaf is 0.51”. Empty phonemes, ie, silence, are represented by the symbol '-'.

The sequence generator 16 (FIG. 1) utilizes a character-only decision tree to construct one or more virtual pronunciations stored in the list 18 in this manner.
Preferably, each pronunciation is associated with a numerical score obtained by combining the probability scores of the individual phonemes selected using the decision tree 10. The pronunciation of words can be scored by building a matrix of possible combinations and utilizing dynamic programming to select the n best candidates. Alternatively, the n best candidates may be selected by first identifying the best possible word candidates and then generating additional candidates through iterative permutation as follows.

The pronunciation with the highest probability score is multiplied by the score of each of the highest scoring phonemes (identified by examining the leaf nodes), and this selection is used as the highest probability or first word candidate. By being chosen first. Additional (n most likely) candidates are examined by again examining the phoneme data in the leaf nodes and identifying the phonemes that have not been previously selected and are the least different from the first selected phoneme, To be elected. This minimal difference phoneme then replaces the first selected phoneme, thereby generating the second highest candidate. The above process may be iteratively repeated until a predetermined number of the n best candidates is selected. The list 18 is sorted in descending order of the score, so that the pronunciation determined to have the highest numerical value by character-only analysis appears first in the list.

As noted above, character-only analysis often yields poor results. this is,
This is because in character-only analysis, there is no way to determine for each character what phonemes are generated by subsequent characters. Thus, character-only analysis may produce a high-scored pronunciation that does not actually occur with natural speech. For example, the proper noun Achilles
Is ah-k-ih-l-l-
This tends to result in i-z pronunciation.
In natural speech, the second l does not actually sound,
−k-ih-l-iy-z. Sequence generators that use a tree of letters only have no mechanism for sifting out pronunciations of words that never occur in natural speech.

The second stage of the pronunciation system addresses the above problem. Mixed tree score evaluation unit 20
Utilizes a set of mixed decision trees 12 to assess the viability of each pronunciation in list 18. The score evaluation unit
It works by sequentially examining each character in the input sequence along with the phonemes assigned to each character by the sequence generator 16.

Like the set of character-only trees, the set of mixed trees contains a mixed tree for each letter of the alphabet. An example mixing tree is shown in FIG. Like a character-only tree, a mixed tree has internal nodes and leaf nodes. In FIG. 3, the internal nodes are indicated by an oval, and the leaf nodes are indicated by a rectangle. A yes / no question is arranged in each of the internal nodes, and probability data is arranged in each of the leaf nodes. The tree structure of a mixed tree is similar to that of a character-only tree, with one important difference. An internal node of a mixed tree can contain two different types of questions. The internal node may include a question about a given character in the sequence and about nearby characters, or may include a question about the phoneme associated with that character and about the closest phoneme corresponding to the sequence. . The decision tree is thus mixed in that it contains mixed types of questions.

The abbreviations used in FIG. 3 are the same as those used in FIG. 2, although some abbreviations (shapes) are added. The symbol L represents a question about a character and its adjacent characters. The symbol P represents a question about a phoneme and its neighboring phonemes. For example, the question "+ 1L == 'D'?" Means "is the character at position +1 'D'?" The abbreviations CONS and SYL are phoneme types, ie consonants and syllables. For example, “+ 1P == CONS?”
The question is, "Is the phoneme at position +1 a consonant?" The numbers at the leaf nodes give the phoneme probabilities, as in a tree of letters only.

The mixed tree score evaluator rescores each of the pronunciations in list 18 based on the mixed tree query and utilizing the probability data in the leaf nodes of the mixed tree. The pronunciation list may be stored as a list 22 in association with each score. Listing 22 shows the first
May be sorted in descending power order so that the pronunciations listed in the list have the highest score.

In many instances, the pronunciation occupying the highest score position in list 22 is different from the pronunciation occupying the highest score position in list 18. This is because the mixed tree score evaluator uses the mixed tree 12 to sift through the mixed tree 12 pronunciations that do not contain self-consistent phoneme sequences or that otherwise would not occur in natural speech. Occurs.

The selector module 24 may access the list 22 to retrieve the pronunciation of one or more lists. Typically, the selection unit 24 extracts the pronunciation with the highest score and gives it as the output pronunciation 26.

As noted above, the pronunciation generator shown in FIG. 1 represents only one possible embodiment utilizing the mixed tree of the present invention. As shown in another embodiment,
The dynamic programming phoneme sequence generator 16 and its associated character-only decision tree 10 may be omitted in applications where one or more pronunciations for a given spelling word sequence are already available. This situation can occur if a previously formed pronunciation dictionary is available. In such cases,
The mixed tree score evaluator 20 scores associated heading items in the pronunciation dictionary, identifies heading items that have low scores, and thus flags suspicious pronunciations in the dictionary being constructed. 12 may be used. Such a system may for example be incorporated into a tool for the creation of a dictionary editor.

The output pronunciations, ie, the pronunciations selected from the list 22, can be used to form a pronunciation dictionary for both speech recognition and speech synthesis applications. In the context of speech recognition, pronunciation dictionaries can be used during the recognizer training phase by providing pronunciation for words not yet found in the recognizer vocabulary. In the context of synthesis, pronunciation dictionaries can be used to generate phoneme sounds for linked playback. The system is, for example, email,
It can be used to augment features of readers or other text-to-speech applications.

The mixed tree scoring system of the present invention can be used in a variety of applications where only one or a list of possible pronunciations are required. For example, in a dynamic online dictionary, when a user types a word, the system will provide a list of possible pronunciations in order of probability. The scoring system is
It can also be used as a user feedback tool for language acquisition systems. A language acquisition system with speech recognition capabilities can be used to display spelling and analyze speaker attempts to pronounce the word in a new language. The system will then tell the user how good or bad the pronunciation of the user is for the word.

<< Generation of Judgment Tree >> FIG. 4 shows a tree-only generation system and a mixed tree generation system. The center of the decision tree generation system is the tree generation unit 40.
The tree generator utilizes a tree generation algorithm that operates on a predetermined set of training data 42 provided by the system developer. Typically, the training data includes arranged letter-phoneme pairs corresponding to well-known unique word pronunciations. Training data may be generated through the array processing shown in FIG. FIG. 5 shows an example of the arrangement processing performed on the word BIBLE. The spelling 44 and its pronunciation 46 are provided to a dynamic programming arrangement module 48 which arranges the characters of the spelling and the phonemes of the corresponding pronunciation. The last E in the example shown does not sound. The character phoneme pairs are then stored as data 42.

Returning to FIG. 4, the tree generator works in conjunction with three additional elements: a set of possible yes / no questions 50, and to select the most relevant question for an individual node. Or a set 52 of rules for determining whether a node should be a leaf node;
This is a pruning method 53 for avoiding overtraining.

The set of possible yes / no questions is:
Depending on whether a character-only tree or a mixed tree is being developed, it may include a character question 54 and a phoneme question 56. If you develop a character-only tree, you can ask questions 5
4 is used, and if a mixed tree is to be developed, both character questions 54 and phoneme questions 56 are used.

The rules for selecting the best question to place at an individual node in the currently preferred embodiment are designed to follow the Gini criteria. Other split criteria can be used instead. For more information on partitioning criteria, see Breiman, Friedman et al. (Breiman, Freidman et al)
Classification and Regression Trees (Classificat
ion and Regression Tree
s) ". In essence, Gin
i) The criteria is a set 50 of possible yes / no questions
Used to select questions from and to use stop rules to determine when a node is a leaf node. Gini standards are "impure (impu)
(Righty) ". Impurity is always non-negative. A node that contains an equal proportion of all possible categories has maximum impurity and only one of the possible categories is assigned Impurity is applied to nodes, such that the containing node has zero impureness (minimum possible value), and there are several features that satisfy the above situation, such as the counting of individual categories within the node. Gini impurity is defined as follows: If C is the set of classes to which the data item can belong and T is the current tree node, then f
It is assumed that (1 | T) is the ratio of the training data items belonging to class 1 and f (2 | T) is the ratio of the items belonging to class 2 in the node T. Then,

(Equation 1) Becomes

To illustrate, assume that the system is forming a tree of the letter "E". At a given node T in the tree, for example, the system has ten examples of how to pronounce "E" in a word. In five of these examples, "E" is "iy"
(The sound of "ee" of "cheeze"). In three examples, "E" is pronounced as "eh" (the "e" sound of "bed"). And in the remaining two cases,
"E" is "-" (that is, silence like "e" of "maple").

Assume that the system considers two possible yes-no questions, Q ₁ and Q ₂ , applicable to the ten cases. Items that answer "yes" to the Q ₁ is,
Includes four examples of “iy” and one example of “-” (the other five items are Q
Answer "no" to ₁ ). Item you answer "yes" to the Q ₂ is, "iy" including three cases of three patients with "eh" of (the other four items answer "no" to the Q _2.). FIG.
Compares these two cases graphically.

The Gini criterion answers to this node which question the system should choose, Q ₁ or Q ₂ . Choosing the Right Questions Gini
i) The criterion is to find the question that has the greatest reduction in impurity when going from parent node to child node. This impurity reduction ΔT is represented by ΔI = i (T) −P _yes ^* i (ye
s) is defined as -P _no ^* i _(no). Where P
_yes is the percentage of items that go to the “yes” child node,
P _no is the percentage of items that go to the “no” child node.

The Gini criterion applies to the above example.

(Equation 2) ΔI for Q ₁ is therefore:

## EQU3 ## i (T) -P _yes (Q ₁ ) = 1-0.8 ² -0.
^{2 2 = 0.32 i (T)} -P no (Q 1) = 1-0.2 2 -0.6 2 = 0.
From 56,

ΔI (Q ₁ ) = 0.62−0.5 ^* 0.32
0.5 ^* 0.56 = 0.18. For the Q _2,

I (yes, Q ₂ ) = 1−0.5 ² −0.5 ² = 0.5 i (no, Q ₂ ) = (same as above) = 0.5

ΔI (Q ₂ ) = 0.6− (0.6) ^* (0.5)
-(0.4) ^* (0.5) = 0.12.

In this case, Q ₁ gives the greatest drop in impurity. Therefore, Q ₁ instead of Q ₂ is selected.

The ruleset 52 declares that the most relevant question for a node is the one that causes the greatest reduction in impurity as it proceeds from the parent node to the child nodes.

The tree generator applies a rule 52 for generating a decision tree for a yes / no question selected from the set 50. The generator continues to generate the tree until an optimally sized tree is generated. The rule 52 includes a stop rule set that terminates the generation of the tree if the tree is generated to a predetermined size. In the preferred embodiment, the tree grows to a larger size than is ultimately required. The pruning method 53 is then used to cut the tree back to the desired size. The pruning method implements the Breiman technique described in the above reference.

As described above, the tree generator determines whether the set 50 of possible yes / no questions includes only text-only questions or is combined with phoneme questions. Generate a set of trees with only the characters shown, or a set of mixed trees, shown schematically at 70. The training data collection 42 includes character phoneme pairs, as described above. When generating a character-only tree, only the character portion of these pairs is used in the arrangement of internal nodes. Conversely, when generating a mixture tree, both the character and phoneme elements of the training data pair are used to locate internal nodes. In both examples, the phoneme portion of the pair is used to place leaf nodes. Probability data associated with the phoneme data in the leaf nodes is generated by counting the number of occurrences of a given phoneme aligning with a given character throughout the training data collection.

The tree generated by the above method for determining pronunciation from characters can be stored in a memory for use in various different language processing applications. These applications are many and varied, but here are some examples to better emphasize the performance and strengths of these trees.

FIG. 6 illustrates the use of both a character-only tree and a mixed tree to generate a pronunciation from a spelled word character sequence. Although the illustrated embodiment utilizes both character-only and mixed tree elements,
Other applications may use only one element and not the other. In the illustrated embodiment, the set of character-only trees is stored in 80 memories, and the mixed tree is stored in 82 memories. In many applications, there is one tree for each letter of the alphabet. Dynamic programming sequence generator 8
4 operates in response to an input sequence 86 to generate a pronunciation at 88 based on the character-only tree 80. In essence, the individual characters of the input sequence are considered individually and a tree of only the appropriate characters is used to select the most appropriate pronunciation for the character. As explained earlier, a character-only tree asks a series of yes-no queries for a given character in the sequence and its neighbors. After considering all the characters in the sequence, the resulting pronunciation is generated by connecting the phonemes selected by the sequence generator.

To improve pronunciation, a mixed tree set 82 can be used. While a character-only tree asks only questions about letters, a mixed tree can ask questions about letters and questions about phonemes. The scorer 90 may receive phoneme information from the output of the sequence generator 84. In this regard, the sequence generation unit 84
Can utilize the character-only tree 80 to generate a plurality of different pronunciations and classify those pronunciations based on their individual probability scores. The classified pronunciation list can be stored in 92 in response to access by the scorer 90.

The scorer 90 receives as input the same input sequence supplied to the sequence generator 84. The scorer 90 applies the questions of the mixed tree 82 to the character sequence, and responds to phoneme questions using data from the store 92 when asking. The resulting output at 94 is typically better pronounced than the output provided at 88. The reason for this is that mixed trees tend to filter out pronunciations that do not occur in natural speech. For example, the proper noun Achill
es is an ah-k-ih-l- phonetic transcription of both l's.
This tends to result in a pronunciation of l-iy-z. In natural speech, the second l does not actually sound,
ah-k-ih-l-iy-z.

The scorer generator 90 may generate a classification list of n possible pronunciations at 96. The score associated with each pronunciation represents a composite number of individual probability scores assigned to each phoneme being pronounced. These scores can themselves be used in applications that need to identify suspicious pronunciations. For example, a speech transcript provided by a team of dictionary editors could use a mixing tree to quickly identify and check for suspicious pronunciations.

<< Character Voice Pronunciation Generator >> In order to show the principle of the present invention, the example shown in FIG. 8 shows a spelled character pronunciation generator comprising two stages. As described more fully below, the mixed decision tree approach of the present invention can be utilized in a variety of different applications, not just the pronunciation generator shown here. The two stage pronunciation generator was chosen for illustration because it emphasizes many aspects and advantages of the mixed decision tree structure.

The two-stage pronunciation generator includes a first stage 116 which preferably utilizes a set of character / syntax / context / dialect (dialect) decision trees 110 and an input sequence 114 at phoneme level. A second stage 120 that utilizes a set of phoneme mixture decision trees 112 to perform. The character-syntax-context-dialect decision tree examines questions that include a character in a spelling sequence and the nearest character (ie, a character-related question). Another question that is investigated is what is the word that precedes or follows the particular word (ie, a context-related question).
Still other questions that are investigated are what parts of the word comprise the speech inside the sentence and what syntax the other words comprise within the sentence (ie, syntax related questions). is there. Yet another question that is investigated is what dialect is preferred to be spoken. The user preferably selects which dialect is spoken by dialect selection device 150.

Another embodiment of the present invention involves utilizing character-related questions and at least one of the language-level characteristics (ie, syntax-related questions or context-related questions). For example, one embodiment is:
The first stage utilizes a set of character syntax decision trees. Another embodiment utilizes a set of character / context / dialect decision trees that does not examine the syntax of the input sequence.

It is to be understood that the present invention is not limited to words that occur in a single sentence, but also includes other linguistic structures that indicate syntax, such as fragmentary sentences or phrases. Should.

An input sequence 114, such as a one-sentence character sequence, is provided to a text-based pronunciation generator 116. For example, the input sequence 114 may be the following sentence. “Did you know where
ad the autobiography? "

The syntax data 115 is an input to the text-based pronunciation generation unit 116. This input provides information that the text-based pronunciation generator 116 appropriately flows into the character / syntax / context / dialect decision tree 110. Syntax data 115 deals with what elements of the language each word in input sequence 114 comprises. For example, the word "read" in the above example input sequence is marked by a verb (rather than a noun or adjective) by the syntax tag software module 129. Syntax Taga Software Technology is project "Xtag"
University of Pennsylvania (University)
Available from institutions such as Pennsylvania). Additionally, the following references discuss Syntax Taga Software Technology: George Foster, "Statistical Dictionary Editing Disambiguation (Statistical Le
xical Disambiguation ”, a master's thesis in computer science (Master
Thesis in Computer Science
e), McGill University of Montreal, Canada (Mc
Gill University, Montral, C
anada), November 11, 1991 (Novemb
er11, 1991).

The text-based pronunciation generator 116 uses the decision tree 110 to generate a pronunciation list 118 and presents possible pronunciation candidates for the spelling input sequence. Individual pronunciations of list 118 (eg, pronunciation A)
Preferably indicates the pronunciation of the input sequence 114 including how to stress the individual words. Further, in a preferred embodiment, the rate at which individual words are spoken is determined.

Sentence speed calculation unit software module 1
52 is used by the text-based pronunciation generator 116 to determine how fast individual words should be spoken. For example, the sentence speed calculation unit 152 examines the context of the sentence and determines whether a particular word in the sentence should be spoken faster or slower than usual. For example, a sentence with an exclamation point at the end of the sentence should state that a predetermined number of words before the end of the sentence should have a shorter duration than usual to better communicate the impact of the exclamation sentence. Generate suggested speed data.

The text-based pronunciation generation unit 116 converts each character or word in the sequence into a probability data included in the decision tree using a decision tree related to the syntax (or the context of the word) of the character or word. The phoneme pronunciation for the character is selected on the basis of, and investigated in order. The set of decision trees 110 preferably includes a decision tree for each letter of the alphabet and the syntax of the associated language.

FIG. 9 shows an example of a character / syntax / context / dialect decision tree 140 applicable to the character "E" in the word "READ". The decision tree consists of multiple internal nodes (shown as ovals in the figure) and multiple leaf nodes (shown as rectangles in the figure).
Contains nodes. A yes-no question is located at each internal node. A yes-no question is a question that can be answered with yes or no. In the character / syntax / context / dialect decision tree 140, these questions are directed to: Given a character in the input sequence (eg, the letter “E” in the case of an individual) and its immediate neighbors, or the syntax of a word in the sentence (eg, noun, verb, etc.), or the context of the sentence Dialect. In FIG. 9, each internal node branches left or right depending on whether the answer to the related question is yes or no.

[0055] The first internal node preferably inquires about the dialect to be spoken. Internal node 138 represents such a query. If the southern dialect is spoken,
A southern dialect decision tree 13 that finally generates more characteristic phoneme values for the southern dialect at the node
Data is passed through 9.

The abbreviations used in FIG. 9 are as follows. The number in the question, such as "+1" or "-1", is the spelling position relative to the current character. The symbol L indicates that the question is related to a character and a character in the vicinity thereof. For example, the question "-1L == 'R'or'L'?" Means that the character before the current character (which is "E") is "R".
Or 'L'. The abbreviations 'CONS' and 'VOW' are character classes, ie consonants and vowels. The symbol '#' indicates a word boundary. 'ta
The term g (i) ′ indicates that the question is about the syntax indicator of the word at position i, where i = 0 is the current word, i = −1 is the previous word, and i = + 1 is the current word. The next word, and so on. Therefore, “tag (0) =
= PRES? "Is the current word a present verb? "That's what it means.

The leaf node is populated with probability data that associates a potential phoneme pronunciation with a numerical value that means the probability that a particular phoneme represents the proper pronunciation of a given character.
A phoneme, or silence, is represented by the symbol '-'.

For example, the present verbs “READ” and “LE”
"E" in "AD" is assigned an appropriate pronunciation with a probability of 1.0 at the leaf node 142 by the decision tree.The past tense of "read" (eg, "Whor")
“E” in “ad a book”) is assigned a pronunciation of “eh” with a probability of 0.9 at the leaf node 144.

The decision tree 110 (of FIG. 8) preferably contains context-related questions. For example,
The internal node context related question is "y
We may check for the word "did" before the word "ou." In such a context, the "y" in "you" is typically Pronounced "ja".

The present invention also generates prosody display data and conveys stress, pitch, suppression, or pause phases when speaking a sentence. Syntax-related questions are how phonemes are stressed or pitched or suppressed.
Will help you decide. For example, the internal node 141 (of FIG. 9) indicates that the first word of the sentence is the example sentence "who read".
a book? In this example, a leaf node 144 with phoneme stress is selected because the first word in this example is a question pronoun. Leaf node 146
Indicates other options that do not stress the phonemes.

As another example, in a question sentence, a pitch mark is attached to the phoneme of the last syllable of the last word of the sentence so as to convey the question phase of the sentence more naturally. Yet another example includes the present invention that can accommodate natural poses when speaking a sentence. The present invention includes details of such poses by giving questions about interruptions, such as commas and periods.

(FIG. 8) Text-based pronunciation generator 116
Thus constructs one or more pronunciation hypotheses stored in the list 118, using the decision tree 110
Use Preferably, each pronunciation is associated with a numerical score obtained by combining the probability scores of the individual phonemes selected using the decision tree 110. The word pronunciation is
A score may be provided by using dynamic programming to build a matrix of possible connections and select the n best candidates.

On the other hand, the n most suitable candidates are:
It can be selected using a permutation technique that first identifies the most relevant word candidates and then generates additional candidates through iterative permutation, such as: The pronunciation with the highest probability score is multiplied by the score of each of the highest-scoring phonemes (identified by examining the leaf nodes), and this selection is matched to the best candidate, ie, the first word candidate. By being used as the first choice. An additional (n best match) candidate examines the phoneme data in the leaf node again and finds not the phoneme selected earlier, but the phoneme that is least different from the phoneme selected first.
Selected by identification. This minimal difference phoneme then replaces the first selected phoneme, thereby generating the second highest candidate. The above process
Iterative iteration may be performed until a predetermined number of the n best candidates are selected. The list 118 may be sorted in descending power order of the score, so that the pronunciations that are determined to be most appropriate by character-only analysis appear first in the list.

Often, the decision tree 110 will only provide some success. This is because in these decision trees, there is no way to determine for each character what phoneme is generated by the following character. Thus, the decision tree 110 may generate a high-scored pronunciation that does not actually occur with natural speech. For example, the proper noun Achille
s is ah-k-ih-l-l which phonetically represents both l's
-Iy-z. In natural speech, the second l does not actually sound,
ah-k-ih-l-iy-z. Decision tree 11
The pronunciation generator using 0 has no mechanism to sift through pronunciations of words that never occur in natural speech.

The second stage 12 of the pronunciation system 108
0 addresses the above problem. The phoneme mixture tree score evaluator 120 uses the set of phoneme mixture decision trees 112 to assess the viability of each pronunciation in the list 118. The score evaluator 120 functions by sequentially examining each character in the input sequence 114 along with the phonemes assigned to each character by the text-based pronunciation generator 116.

The phoneme-mixed tree score evaluator 120 performs the list 11 based on the phoneme-mixed tree query 112 and using the probability data in the leaf nodes of the mixed tree.
Re-score for each of the pronunciations in 8. The pronunciation list may be stored as a list 122 in association with individual scores. The list 122 may be sorted in descending power order so that the first listed pronunciation has the highest score.

In many instances, the pronunciation occupying the highest score position in list 122 is different from the pronunciation occupying the highest score position in list 118. This is because the phoneme mixture tree score evaluator 120 uses the phoneme mixture tree 112 to sift through the phoneme mixture tree 112 a pronunciation that does not include a self-consistent phoneme sequence or a pronunciation that would otherwise not occur in natural speech. It happens to multiply.

In the preferred embodiment, the mixed phoneme tree score evaluator 120 utilizes the sentence speed calculator 152 to determine speed data for pronunciations in the list 122. In addition, the evaluator 120 uses a phoneme mixture tree that causes questions about dialects (dialects) to be investigated and stresses and other prosodic aspects to be determined at the leaf nodes in a manner similar to the approach described above.

The selection unit module 124 includes a list 122
The list 122 may be accessed to retrieve one or more pronunciations therein. Typically, the selection unit 12
4 draws the pronunciation with the highest score and outputs it.
Give to 26.

As described above, the pronunciation generator shown in FIG. 8 represents only one possible embodiment utilizing the mixed tree approach of the present invention. In another embodiment, the output pronunciations, ie, the pronunciations selected from the list 122, can be used to form a pronunciation dictionary for both speech recognition and speech synthesis applications. In the context of speech recognition, pronunciation dictionaries can be used during the recognizer training phase by providing pronunciation for words not yet found in the recognizer vocabulary. In the context of synthesis, pronunciation dictionaries can be used to generate phoneme sounds for linked playback. The system can be used, for example, to augment features of email readers or other text-to-speech applications.

The mixed tree scoring system of the present invention (ie, letters, syntax, context, and phonemes)
Can be used in various applications where only one or a list of possible pronunciations are required. For example, in a dynamic online language acquisition system, when a user types a sentence, the system provides a list of possible pronunciations for that sentence, in order of probability. The scoring system can also be used as a user feedback tool for a language acquisition system. A language acquisition system with speech recognition capabilities can be used to display spellings and analyze speaker attempts to pronounce the sentences in a new language. The system will indicate to the user how good or bad the pronunciation of the user is for the sentence.

Although the present invention has been described in its appropriate form, it will be appreciated that there are numerous applications for mixed tree pronunciation systems. Therefore, the present invention
Certain modifications and changes are possible without departing from the spirit of the invention as set forth in the appended claims.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating the elements and steps of the present invention.

FIG. 2 is a tree diagram showing a tree of characters only.

FIG. 3 is a tree diagram showing a mixed tree according to the present invention.

FIG. 4 is a block diagram illustrating an existing preferred system for generating a mixing tree according to the present invention.

FIG. 5 is a flowchart illustrating a method of generating training data through an alignment process.

FIG. 6 is a block diagram illustrating the use of a decision tree in an exemplary pronunciation generator.

FIG. 7 illustrates the application of the Gini criterion to assess which questions should be used in placing a node.

FIG. 8 is a block diagram of a character-to-speech pronunciation generation unit according to the present invention.

FIG. 9 is a tree diagram showing a character / syntax / context / dialect mixed decision tree.

[Explanation of symbols]

Reference Signs List 10: Character-only decision tree 12: Mixed decision tree 14: Input sequence 16: Dynamic programming phoneme sequence generation unit 18: Pronunciation list 20: Mixed tree score evaluation unit 24 ..Selection module 40... Tree generation module 42... Training data accumulation 48... Dynamic programming array module 50... Yes / no question set 52... Rule set 53. ··· Memory of tree of characters only 82 ··· Memory of mixed tree set 84 ··· Dynamic programming sequence generation unit 86 ··· Input sequence 90 ··· Scorer generation unit 110 ··· Characters / Syntax / Context dialect decision tree 112 ... phoneme mixture decision tree 114 ... input sequence 115: Syntax data 116: Text-based pronunciation generation unit 120: Phoneme mixture tree score evaluation unit 124: Selection module 129: Syntax tag software module 138・ Internal node 140 ・ ・ ・ Character ・ Syntax ・ Context ・ Dialect judgment tree 141 ・ ・ ・ Internal node 144 ・ ・ ・ Leaf node 150 ・ ・ ・ Dialect selection device 152 ・ ・ ・ Sentence speed calculation unit software module

 ──────────────────────────────────────────────────続 き Continued on the front page (72) Inventor Matteo Contorini 93109 California, USA Santa Clara Drive, 821 Cliff Drive, Number B-1

Claims (23)

[Claims] 1. An apparatus for generating at least one phonetic pronunciation for an input character sequence selected from a predetermined alphabet, comprising: a memory for storing a decision tree of only a plurality of characters corresponding to the alphabet; The character-only decision tree with internal nodes representing yes-no questions for a given character in the sequence and its neighboring characters; and a memory further storing a plurality of mixed decision trees corresponding to the alphabet; A first plurality of internal nodes representing a yes-no question for a given character of the given sequence and its immediate neighbors, and a second plurality of internal nodes representing a yes-no question for the given sequence of phonemes and their immediate neighbors. A plurality of internal nodes, wherein the mixed decision tree comprises: Further comprising a leaf node indicating the probability data associated with the input character sequence, the input character sequence being processed to correspond to the input character sequence, being combined with the character-only decision tree and the mixed decision tree; A phoneme sequence generator for generating a first set of phonetic pronunciations; and a scored phoneme associated with the mixed decision tree, processing the first set and indicating at least one phoneme in the input sequence. Generate a second set,
And a score evaluation unit. 2. The method of claim 2, wherein the second set comprises a plurality of pronunciations each having an associated score derived from the probability data, and further wherein the body accepts the second set and based on the associated score. The apparatus of claim 1, further comprising a sound selection unit operable to select one sound from the two sets. 3. The apparatus according to claim 1, wherein said phoneme sequence generation unit generates a predetermined number of different sounds corresponding to a given input sequence. 4. The apparatus of claim 1, wherein said phoneme sequence generator generates a predetermined number of different pronunciations corresponding to a given input sequence, and indicates the n most suitable pronunciations according to said probability data. . 5. The apparatus of claim 4, wherein said score evaluator re-scores said n most suitable pronunciations based on said mixed decision tree. 6. The apparatus according to claim 1, wherein the sequence generation unit constructs a matrix indicating various pronunciations, the matrix relating to a possible combination of phonemes. 7. The apparatus of claim 6, wherein the sequence generator selects n most suitable phoneme combinations from the matrix using dynamic programming. 8. The apparatus of claim 6, wherein the sequence generator selects n most suitable phoneme combinations from the matrix by iterative permutation. 9. A speech recognition system comprising a pronunciation dictionary utilized for recognizer training, wherein at least a portion of the second set provides a pronunciation of words based on spelling of words. Place dictionaries,
The device of claim 1. 10. The apparatus of claim 1, further comprising an audio system that accepts at least a portion of the second set to generate an audible synthetic pronunciation of the words based on the spelling of the words. 11. The voice synthesis system according to claim 1, wherein
11. The device of claim 10, wherein the device is incorporated in a reader. 12. The apparatus of claim 10, wherein said speech synthesis system is incorporated into a dictionary to provide a list of possible pronunciations in stochastic order. 13. Displaying a spelled word and analyzing a speaker's attempt to pronounce the word using at least one of the character-only decision tree and the mixed decision tree, Further includes a language acquisition system that shows the speaker how appropriate the pronunciation of the word is to the word,
The device of claim 1. 14. A method of processing spelling-to-pronunciation data, the method comprising: providing a first set of yes-no questions regarding characters in an input sequence and their relationship to nearby characters. Providing a second set of yes-no questions regarding the phonemes within and their relation to neighboring phonemes; a plurality of different sets of pairs each comprising a character sequence and a phoneme sequence selected from the alphabet. Using the first and second sets and the training data to form a decision tree, each comprising a plurality of internal nodes and a plurality of leaf nodes, in the alphabet of the alphabet. Generating for at least a portion; and providing the first and second Locating a question selected from a set of letters, and locating, at the leaf node, probability data relating the portion of the alphabet to a plurality of phonemic pronunciations based on the training data. The method. 15. The method of claim 14, further comprising the step of providing said collection of training data as pairs of ordered character sequence phoneme sequences. 16. The step of providing training data collection comprises: providing a plurality of input sequences including a phoneme sequence representing the pronunciation of a word formed by the character sequence; and selecting a selected one of the phonemes. Arranging with selected ones of the characters to define an arrayed character phoneme pair. 17. The method of claim 14, further comprising providing at least one associated phoneme pronunciation to an input string and using the decision tree to score the pronunciation based on the probability data. Method. 18. The method according to claim 18, further comprising the step of providing a plurality of related phoneme pronunciations to the input string and using the decision tree to select one of the plurality of pronunciations based on the probability data. 14 methods. 19. The method according to claim 19, further comprising the step of providing a plurality of associated phonemic pronunciations to an input character string indicative of a word, and using the decision tree to generate a speech transcript of the word based on the probability data. Item 14. The method according to Item 14. 20. The method of claim 19, further comprising using the speech transcription to place a dictionary associated with a speech recognizer. 21. The method of claim 14, further comprising the step of providing a plurality of associated phoneme pronunciations to an input string representing a word, and assigning a numerical score to each of said plurality of pronunciations using said decision tree. the method of. 22. An apparatus for generating at least one phonetic pronunciation for an input character sequence that forms a word selected from a predetermined alphabet and adheres firmly to a predetermined syntax. An input device receiving syntax data indicative of the syntax of the words in the sequence; a computer storage device storing a plurality of text-based decision trees comprising questions indicative of predetermined characteristics of the input sequence; The predetermined characteristic, comprising a character-related question relating to the sequence, and further including a characteristic selected from a group consisting of a syntax-related question, a context-related question, a dialect-related question or a combination thereof; Predetermined input sequence The text-based decision tree comprising internal nodes representing gender questions; the text-based decision tree further comprising leaf nodes indicating probability data associating each of the characters with a plurality of phoneme pronunciations; and processing the input character sequence. And a text-based pronunciation generator for generating a first set of phonetic pronunciations corresponding to the input character sequence based on the text-based decision tree. Characteristic speech pronunciation generation device. 23. A phoneme coupled to the text-based pronunciation generator for processing the first set, generating a second set of scored phonetic pronunciations indicative of at least one phonetic pronunciation of the input sequence. 23. The apparatus of claim 22, further comprising a mixed tree score evaluator.