JPS62246097A - Word base form synthesizer for voice recognition - Google Patents

Abstract

(57) [Summary] This bulletin contains application data before electronic filing, so abstract data is not recorded.

Description

[Detailed description of the invention] This invention will be explained as follows.

A. Industrial field of application B. Prior art C1 Problems to be solved by the invention Means for solving the problems E. Embodiment El. Environment of the speech recognition system. General description El. 2. Construction of phonetic base form E13. Construction of feeneem base form E14. Training of word model E2. SYNTHESIS OF BASE FORMS OF UNVOICED WORDS F0 EFFECTS OF THE INVENTION A. INDUSTRIAL APPLICATION FIELD OF THE INVENTION The present invention relates generally to speech recognition, and specifically relates to the synthesis of a given word model from other known word models. It's about things.

B. Prior Art In some speech recognition methods, words in common words are represented by word models. This word model is sometimes called the base form of the ejective word. For example, at IBM Corporation, each word is represented as a sequence of Markov models in an experimental speech recognition device. Note that each sequence is itself a Markov model.

By using the word model in conjunction with the eight forces generated in response to the speech input, the speech input is matched to the words in the word capsule.

In one method a set of models is defined. The base form of every word in the word cloud is constructed from multiple models selected from a predefined set of models.

However, another method of representing words in multiple base forms also seems to have some basis.

In this case, each base form is constructed from a model selected from a corresponding set of models. That is, a base form constructed from models included in the first model group is used in a speech recognition device for one purpose, and a base form constructed from models included in the second model group is used for another purpose. be able to. Furthermore, several base forms can also be used together in the process of performing acoustic matching and other purposes.

Most large word (e.g. 500+ words) speech recognition systems use the base form of a word.

Modify the speech recognition device to suit each user. That is, the user speaks a (training) text of known words in order to determine the values of certain variables associated with each base form.

Typically, each word-based form has its variables set directly from the data generated during training. If a word is represented by multiple base forms (constructed from models in each model family), it will yield enough data to "train" all the word base forms, i.e. to set their variable values. However, a long training period was required.

It is undesirable for the training period to take a long time. Therefore, the need to generate enough data to construct multiple base forms for all words in a word constellation has been considered a problem to be overcome.

Furthermore, in some cases, a base form constructed from a model included in the second model group.

The base form may already exist or may be easily formed compared to the base form constructed from the models of the first model group. moreover.

The base form required in the speech recognition system may not be the base form of the second model group, but the base form of a model selected from the first model group.

Conventionally, training data is required to construct all base forms from a first set of models, regardless of whether the base forms of the second set are known or not.

Problems to be Solved by the C0 Invention Therefore, it is desirable to provide a method for synthesizing base forms of words that were not uttered during the training period.
This is the object of the present invention.

Means for Solving Problem D1 Specifically, in the present invention, certain known words are
It is assumed that they are each represented by (a) a base form constructed from word models included in the first model group, and (b) a base form constructed from word models included in the second model group. It is also assumed that the two base forms can be aligned with each other for known words. Furthermore, it is assumed that other words are originally represented or can be readily represented in base forms constructed from models included in the second set of models. The present invention teaches techniques for synthesizing base forms (constructed from models selected from a first set of models) for such other words after a training period.

That is, from the base form generated during training, a correlation is made between the models of the first model group and each given model of the second model group in a given context. When a given model in a given context appears in a "new" word that was not uttered during training, the corresponding "
A new word intercept appears in the correlated models in the first set of models. By representing each intercept of "new" words with a correlated model of the first set of models and concatenating the correlated models for successive intercepts of "new" words,
A base form constructed from models included in the first model group is synthesized.

The present invention thus achieves the objective of synthesizing a certain base form constructed from the models included in the first model group based on the known base form without the need for further training. achieve.

The present invention also achieves the objective of generating different base forms for the same word without training each base form independently.

The present invention also provides that, in some cases, base forms for models that are preferred for use in speech recognition calculations are derived from base forms that may be easy to generate or form, but are likely to be computationally inefficient. Realize your purpose.

Furthermore, for words in common words, the base form of the second model group based on phonetics is already known or can be readily determined, but using the base form of the model related to the output of the acoustic processing device, If the accuracy and/or speed of recognition is to be improved, the present invention provides a method for synthesizing a base form of a model related to the output of certain words without the need for training.

E. Embodiments El Environment of Speech Recognition System Ell General Description FIG. 1 shows a general block diagram of a speech recognition system 100o. System 1000 includes stack decoder 1
002 and the sound processing device 1004 connected to it
．． It includes an acoustic matching element 1006 (preferably an array processing unit), and a language model processing unit 1010. The language model processor 1010 determines the likelihood of words based on some preferably stylistic criteria. Acoustic matching techniques and language models are described in various papers. For example, the following papers discuss various aspects of speech recognition and modeling methods and are cited here: S,RoBahl,F,Je
linek, R.L., Marcer, “conti
neousSpeech Recognition b
y 5 statistical methods”, Pr.
oceedings of the IEEE, Vo
l, 64, PP, 532-556, (1976);
“A Maximum Likelihood App
reach to continuous speech
hRecognition”, IEEE Tran
actions on patternAnalyses
is and Machine Intelli
gence, Vol. PAMI-5, No. 2.
March 1983.

Sound treatment bag! i! 1004 is designed to convert audio waveforms into output strings. Speech is characterized by a vector “space” with vector components corresponding to features selected by processing unit 1004 .

Traditionally, such characteristics have included energy amplitudes at various frequencies of the audio spectrum. Sound processing device 1
004 stores a plurality of prototype vectors. Each archetype vector is.

An audio signal input, with predetermined values for each component, enters the audio processing unit 1004, preferably divided into two successive time intervals, each time interval having a predetermined value for each component. Assign the output vector based on the value of . The output vector for each time interval is compared to each prototype vector and a distance measurement is made for each prototype vector. Distance measurements can be performed using one conventional vector distance measurement method.

Each time interval is then associated with a particular prototype vector or some other output-related function.

For example, each prototype vector can be identified with a label or symbol. We call it “fenem”
a) Call it J. This name originates from the minute sound system obtained by the front end processor (FE). In such a case, the sound processing device 1004 outputs fineim at each time interval.

Therefore, for a certain audio input, the sound processing device 11
004 generates a string of feeneem. There are about 200 different feeneems in a given alphabet (
or a label) is preferable. In that case, for each time interval one of the 200 finems is selected.

A specific type of sound processing device is disclosed in Japanese Patent Application Laid-Open No. 61-126600.
It is stated in the No. In the invention of this publication, the features selected for the vector components are derived from a unique model of one ear.

Each vector component corresponds to an estimated neural firing rate for each frequency band.

The feeneem exiting the sound processing device 1004 enters the stack decoder 1002. stack decoder 10
02 defines one or more possible word paths and extends each possible word path with a possible next word. The probable word path and the probable next last word are, in part, the acoustic processing bag [! It is determined based on the label generated in step 1004. A new type of stack decoder is disclosed in Japanese Patent Application No. 61-32049.

In determining the likely next word, or more specifically, the list of candidate words that are relatively likely to come next on a path, the feeneem from the acoustic processing unit 1004 is applied to the acoustic matching element 1006. Sent.

Acoustic matching of various models was published in patent application 1986-2552.
It is described in the specification of No. 05.

Acoustic matching element 1006 operates based on word models. Specifically, as described in the immediately cited patent application, acoustic matching is performed by characterizing words as sequences of stochastic finite state machines.

This finite state machine is also called a Markov model.

Generally, there is a set of Markov models, each Markov model corresponding to a certain speech category. For example, each Markov model.

It can be made to correspond to certain elements of the international phonetic alphabet. The phonetic letter AAO will have a corresponding phonetic Markov model, and so will AEO and AEI, and hereafter ZX
It continues in the same way until.

When using a phonetic Markov model, each word is first defined by a sequence of phonetic elements.

A phonetic base form of a word is constructed by concatenating the phonetic models corresponding to the phonetic elements of the word.

Each phonetic Markov model is preferably represented with a structure as shown in FIG. 2. Specifically, the phonetic Markov model in FIG.
7; (b) 13 transitions tr1 to tr13; (c) Probability of each transition P (tri) to P (tr13) (only P(tri) is shown in the figure); (d) Transition tr
Contains the label output probabilities for i to trio. Each label output probability corresponds to the likelihood that a given label is generated at a given transition of the phonetic Markov model. This likelihood is determined during the training period. For example, the likelihood that label 1 will be generated at transition tri for a particular phonetic model (for example, the AAO phonetic model) is determined based on known text utterances that correspond to a specified phonetic element sequence. , Pi (1). The likelihood that label 200 is generated at transition tr3 is also determined,
P3 (200).

Similarly, based on the training data, the label output probability of each label in each transition tri-trio is determined and identified for each phonetic model.

Transitions trll to tr13 are empty transitions. No labels are generated for blank transitions. Therefore, it is not assigned a label output probability.

The transition probabilities for all transitions tri to tr13 are also derived from the data generated during training by applying a well-known forward-backward algorithm.

As a simple explanation, FIG. 2 shows an AAO sound element and shows how the production of an AAO sound can take various paths from state S1 to state S7.

If the transition trll is followed, the AAO speech element will not generate a label. Alternatively, a path from state S1 towards state S2 or state S4 can be followed.

If either of these routes is taken, a label will be generated. These alternative routes are shown in FIG.

In Figure 3, the horizontal axis represents the time interval over which labels are generated. Solid lines indicate possible transitions within the model during the label interval. Dotted lines indicate empty transitions that can be followed.

FIG. 4 shows a grid depicting the Markov phonetic model at successive label intervals starting at start time 10. The time to corresponds to the time when the first label in the string is generated by the acoustic processing bag [71004. As an example, various paths leading to the final state S7 are illustrated, assuming that to corresponds to the state S1. In a certain route, state S1
to state S2, and from there to state jlls3. That is, from the state S3 to the state S7, which follow two non-empty transitions, there is a route for which no label is generated and a route that follows the non-empty transition. It is recognized that for a given label sequence, there are various paths along the transition of one or more phonetic models.

A phonetic Markov word model is shown in FIG.

In FIG. 5(a), the word "T HE" is shown as three phonetic elements arranged in order based on a certain pronunciation thereof. The audio elements are DH, UHI, and Xx.

In Fig. 5(b), the phonetic Markov models for DH, UHI, and XX are concatenated to form one word "T HE".
” to form a phonetic word-based form.

A grid such as that in FIG. 4 can be expanded to include all labels generated in response to a certain phonetic term (eg, the word "THE"). It should be noted that in the extended lattice, transitions between states are assigned probabilities, and label output probabilities are also assigned to transitions.

In the process of evaluating the likelihood of a word, which word model is selected at time to, t1, etc.
004) includes determining whether the level has the maximum likelihood of being generated. A detailed description of how acoustic matching elements are used to determine the likelihood of star words can be found in the aforementioned patent applications relating to acoustic matching elements.

In addition to the phonetic models used to construct phonetic base forms, Fee-Neem Markov models have also been used for acoustic matching elements. Specifically,
Instead of a relatively complex speech model such as that shown in Figure 2, a set of Markov models based on Feeneem has been used. Figure 6 shows the Feenim-type Markov model. The Feenim-type Markov model has two states fis
It is recognized that the structure is simple, including 1°82 and three transitions. One non-empty transition extends from Sl to 82 and the second
The non-empty transition of is a self-loop returning from state s1 to itself. The empty transition extends from state s1 to state S2. For these three transitions.

Each is assigned a probability, and each of the two non-empty transitions has a label output probability derived from the data generated during the training period. A lattice based on the Feeneem model is shown in FIG.

In Figure 8, multiple feeneem-type Markov forms are formed, as in the case of forming a feeneem-type base form of a word.
Models are connected.

The notation in FIG. 8 will be briefly considered. FP200 consists of 20,000 Feeneem Alphabets (Fineem Sets) that typically contain 200 different Feeneems.
Refers to the feeneem-type voice corresponding to the th eenim.

Similarly, FPIO is 1 of the feeneem alphabet.
Corresponds to the 0th feeneem. FP200. FPI
O, etc., when concatenated, give the word's fee-neem base form.6 Each fine-me usually lasts 0.01 seconds, and the length of a typical pronounced word averages 80 to 100 fine-me numbers. Further, since each Fee-Neem model produces on average about one Fee-Neem, a typical Fee-Neem base form is 80-100 Fee-Neem models long. Let P (tr2) denote the probability of the second transition of FP200.

The probability that the FP200 model generates label 1 in its second transition is P' F200 (1)i? represent. FP20
The 0 model can actually be skewed to generate the 20000th enemy. However, due to variations in pronunciation, there is a possibility that the FP200 model will produce other feeneems.

In Section 2 below, we outline methods for constructing word-based forms from phonetic Markov models and fee-neem Markov models, respectively. Considering these two base forms, the phonetic base form contains fewer connected models, but the phonetic model requires more calculations than the fee-neem model. It is recognized that there are significantly more cases. Furthermore, while the phonetic base form is defined by the hands of a phonetician, the feeneem basic form is automatically constructed without the phonetician's intervention, as described in the patent application cited in Section E13. It has been.

E12. Construction of a phonetic base formFor each word,
There are sequences of phonetic sounds, each of which has a corresponding phonetic model (also called a phonetic ``phonetic machine''). Preferably, at each non-empty transition, there is some probability attached to the generation of each feeneem (the feeneem alphabet is shown in Table 1). Transition probabilities and feeneem probabilities in various phonetic phone machines are determined during training by recording the feeneme strings produced when a known sound is uttered at least once and using the well-known forward-backward algorithm. By applying.

It is determined.

No. 4 001 AAII 029 BI2-05
7 EH (002AA12 030 BI3-
058 EHI003 AA13 031
BI3-059 EHI004 AA14
032 BI5- 060 EHI005 A
A15 033 BI6- 061 EHI0
06 AEII 034 BI7- 062
EHI007 AE12 035 BI8-
126 RXI008 AE13 036
BI9- 127 5HI009 AE14
037 DHI-128SH; 010 AF15
038 DH2-1295XI011 Awll
039 DQI-1305XS012 Av
12 040 BO2-1315X3013 A
V13 041 BO3-132SX<014
AXII 042 BO4-1335XE015
AX12 043 DXI-1345XE01
6 AX13 044 DX2- 135
SX'r017 AX14 045 II! E0
1 136 Tl(]018AX15046EEO
2137THko019 AX16 047 EE
II 138 TH;020 AX17 0
48 EH11139TH71021BQI-049
EH11140Tl (3022BO2-050EH11
141TQI023 BO3-051EH11142
TQ:024 BO2-052EH11143TX;
025 BXI-053EH11144TXI026
BXIO054EH11145TX:2 148
TX5- 176 XXl11 149
TX6- 177 XX122 150
tlHOl 178 XX133 151
UHO2179XX144 152 [81118
0XX155 153 UH12181XX16-
154 UH13182XX17- 155
LIH14183XX18- 156 UUII
184 XX19- 157 UU12
185 XX2-- 158 UXGI
186 XX20- 159 UXG2 18
7 XX2l-160υXll 188 XX
22- 161 UX12 189 XX23
- 162 UX13 190 XX24-
163 VXI-191XX3-- 164
VX2- 192 XX4-- 165 VX
3- 193 XX5-- 166 VX4-
194 XX6-- 167 WXI-1
95XX7-- 168 RiX2- 19
6 XX8-- 169 WX3- 19
7 XX9-, +170 WX4-
198 2X1-- 171 WX5- 199
XX2-- 172 RiX6- 200 XX
3-'-173WX7- As an example, a sample of the statistics of sound DH is shown in Table 2. Approximately, the transitions tri, tr2 . t
The label output probability distribution for r8 is represented by one distribution, and transitions tr3, tr4 . tr5. tr9 is represented by one distribution, and transitions tr6, tr7 . trlo is represented by one distribution. Add this to the arcs (i.e. transitions) in each column 4.
．． 5. or 6 as shown in Table 2. Second
The table shows the probability of each transition and the probability that a certain label (i.e. feeneem) is produced at the beginning, middle or end of a phonetic element (i.e. 'sound J) DH. For example, in the sound DH: The probability of transition from state S1 to state S2 is counted as 0.07243. State S
The probability of transitioning from 1 to state S4 is 0.92757 (in this case, these are the only two possible transitions from the initial state, and their sum is 1). Regarding the label output probability, the probability that the sound DH generates the feeneem AE13 (see Table 1) at its final part, ie, No. 611i in Table 2, is 0°091. Also in Table 2, each node (or state) has a count associated with it. The node count indicates the number of times a sound entered its corresponding state during training. Each phonetic model,
Alternatively, statistics such as those shown in Table 2 can be found for each phonetic phone machine.

The process of arranging phonetic phone machines into sequences of word-based forms is usually performed by a phonetician and is usually not done automatically.

; I Guri1de 〇Gideζ) PO -o u3 RoE13. Construction of the Fee-Neem Base Form The probabilities associated with each transition and the probabilities associated with each label at the transition for a given Fee-Neem model, such as the one shown in Figure 6, are determined by the phonetic base form during the training period. is determined in a similar way to when training a standard model.

A fee-neem-type word base form is constructed by connecting fee-neem-type single sounds. One method is 198
U.S. Patent Application S, N, 697174, filed February 1, 2015
listed in the number. Preferably, the feeneem base form of a word is grown from multiple utterances of the word.

This is reflected in U.S. patent application S.N. 06/738933.
listed in the number. This disclosure is hereby incorporated by reference to the extent necessary to fully disclose the invention. Briefly, one method for growing the basic form of a word from multiple utterances involves the following steps.

(a) Transform each of the multiple utterances of a word segment into a feeneem string.

(b) Define a Fee-Neem-type Markov model single-tone machine of -ffi.

(c) Determine the best single note machine P1 to generate the multiple feeneem string.

(d) Determine the best diphonic baseform of the form PIF2 or P2P1 to generate multiple feeneem strings.

(e) Align the best diphonic baseform for each feeneem string.

(f) Divide each fineme string into a left part corresponding to the first single note machine in the two-note baseform and a right part corresponding to the second note machine in the two-note baseform.

(g) Name each left part a left substring and each right part a right substring.

(h) treat a set of left substrings in the same way as a set of feeneem strings corresponding to multiple utterances, but with the addition that the monophonic base form has a lower predetermined substring than the best diphonic base form; , when the probability of generating the substring is high, prohibiting repartition of the substring.

(j) Process a set of right substrings in the same way as a set of feeneem strings corresponding to multiple utterances, but with the addition that the monophonic base form has a higher value for a given substring than the best diphonic base form. , when the probability of generating the substring is high, prohibiting repartition of the substring.

(k) concatenate unsplit single notes in the same order as their corresponding feeneem substrings;

The baseform model is constructed by inputting known utterances aloud into an acoustic processing device, which in turn generates a string of labels accordingly.
Based on the known utterances that are trained (or filled with statistics) and the generated labels, the statistics of the one-word model are derived by the forward-backward algorithm discussed in the papers cited above.

FIG. 7 shows a grid corresponding to a feeneem type single note.

This grid is much simpler than the grid of Figure 4, which relates to the phonetic model system.

Both phonetic base forms and feeneem base forms can be used in acoustic matching elements and for other speech recognition purposes.

E14. Training the word model A good training method is L, R, Bahl,
P.F.

Brown, P.V., Desouza, and R.L.
, invented by Mercer.

IBM Corp.). ``Improvements in the training of Markov models used in speech recognition systems (Imp
roving the Training of Ma
rkovModelsUsed in a 5pee
ch Recognition System), the disclosure of which is incorporated herein by reference.

Training includes determining base form statistics for each word in a manner that improves the probability of a correct word relative to the probabilities associated with other words. Rather than maximizing the probability that the label is given the script as in other methods, we maximize the difference between the probability that the label output is given the correct script of the uttered word and the probability of other (incorrect) scripts. The idea is to do so.

According to such a method, (each word in the word common is represented by a base four 11 with at least one stochastic finite state model, and each stochastic finite state model has a transition probability term and an output probability term, (in a system for decoding a word in a common word from an output selected from an alphabet of outputs in response to a communicated speech input) in which the output produced in response to the communication of a known word is The value of the stored probability term is reduced such that the likelihood of being generated by the base form of a word is high compared to the likelihood that the output generated is generated by at least one other base form of the word. A method is provided for determining the value of a probability term, including biasing a portion of the probability term.

Each word (or distinct pronunciation of the word, referred to as a "word verb") is preferably represented by one or more stochastic finite state machines (or models) in sequence.

Each machine corresponds to a certain speech of the set of sounds. Each sound is defined by a phonetic element, a label (or feeme), or some other predefined feature that can specify a Markov model or similar model. Correlates with the characteristics of the voice.

A training script usually consists of a series of known words.

According to the training method described herein, probability values associated with probability items are evaluated as follows.

An estimated value of 0' is set for each probability item.

Given an estimate of 0' and the labels generated during training, a value called 'single count' is determined. It concerns the (expected) number of times an event will occur. One particular definition of "single count" is (a
) a string of labels Y, (b) a defined estimate O′, and (c) a particular time t, the above single count can be calculated using one well-known forward-backward algorithm, or Baum-Wuerch
Apply an algorithm to decide.

According to the above definition, a single count can be expressed as:

Pr(S,,t,lY,o,τ)l If we sum up the single counts of J for a particular S,,τ,,Y,0′ at each time t, then for the corresponding transition probability term,′ Since the transition cumulative count is a sum of probabilities, its value can exceed 1. For each transition probability, it is preferable to store a respective transition probability term. The current probability value for each transition probability term is determined by dividing this cumulative count of transitions by the sum of the cumulative counts of all possible transitions from state S. It is preferable to store it in association with the relevant transition probability item.

For the label output probability terms, sum the single counts again. For each of these probability terms, sum the single count for all O' for which the corresponding generated label in that string is the label corresponding to the label output probability term. The sum in this case is the 'label output cumulative count'.

This cumulative count, preferably stored in association with its corresponding label output probability item, is divided by the sum of the single counts over the label time to determine the current probability value for each label output probability item. .

According to the method of the patent application cited above, a training script of known words uttered, an initial probability value for each probability item, and a list of candidate words for each word uttered during training are defined.

The list of candidate words is defined by a procedure such as rapid approximate acoustic matching. For every known word pronounced, there is a correct known word and an incorrect word (the incorrect word is preferably the one that has the highest likelihood of being erroneously decoded as a known word). .

The current probability value of a probability item is determined by first calculating the "plus count value" and the minus count value of each probability item based on the base form of the correct word or the base form of the incorrect word.

This plus count value is added to the cumulative value of the corresponding probability item (for each probability item) and firstly, the negative count value is subtracted from the cumulative value.

The plus count value is calculated for each probability term in the base form of the correct (i.e., known) word by applying one well-known forward-backward algorithm and preferably scaling the statistics obtained therefrom. Adding a positive count value biases the count value (and the probability term derived from it) toward the string Y, making Y appear to be the output with a higher likelihood of being a relatively correct word model. .

The negative Callan 1-value of a given probability term is computed by applying a forward-backward algorithm, such as when an incorrect word is pronounced to produce a string of labels. The negative count value derived from a single pronunciation of a known word is subtracted from the most recent value of its corresponding cumulative count (before or after addition with the positive count value). This subtraction biases the cumulative count used to calculate the probability term on the base form of the incorrect word away from string Y.

Based on these adjusted cumulative counts, probability values and probability values for the counts are established to increase decoding accuracy.

According to the above steps for each word in the word range, the stored values and probability values for the counts are adjusted to increase the decoding accuracy.

The methods discussed above serve to improve count values determined by other methods in order to improve the accuracy in decoding speech into recognized words in words.

E2. Synthesis of Base Forms of Unspoken Words In FIG. 9, the general method of the present invention is illustrated. At step 2002, words in the training text are represented in a phonetic base form. to be specific,
Each word uttered during the training period is characterized, usually by a phonetician, as a sequence of phonetic elements defined in the International Phonetic Alphabet. Each phonetic element is represented by its corresponding phonetic model. therefore.

For each word, there is a sequence of phonetic models corresponding to it as described above in stage E12. This column represents the phonetic base form.

As previously explained in paragraph E13, a word can also be represented in a feeneem base form constructed from a series of feeneem models. In step 20o4, words in the training text are represented in feeneem base form.

It is recognized that a feeneem is 'output related', i.e. a feeneem is an output produced by a sound processing device, e.g. be. In this regard, it should be noted that models related to other outputs can also be used instead.

For example, an 'output-related model' may be defined based on a simple output vector or other selectable characteristic output of the sound that the sound processing device provides as output.

The phonetic models that occur in the training text occur in the context of various phonetic models. In the currently described embodiment, the "context of a phonetic model" is defined by the phonetic model that immediately precedes and immediately follows the subject phonetic model, i.e.
For a certain sequence of sounds, place 1! ! The context of the phonetic model of the subject at pi is located at positions P(i-1) and P(i+1
) is determined by the phonetic model in A phonetic model of a particular subject can occur in any of multiple contexts. - assuming that there are 70 phonetic elements in the set of phonetic elements (in the discussion of this application, including one element corresponding to silence);

Any (non-silent) phonetic model can be preceded by any of the 70 phonetic models, and any of the 70 phonetic models can come after it. it is conceivable that.

Therefore, for a given phonetic model, 70X7
0=4900 contexts are possible.

According to one embodiment of the invention, each of a number of possible contexts for each phonetic model is assigned a location in the memory.

However, in the preferred embodiment discussed below, only selected contexts enter storage. In either case, the −
mth phonetic model n of the set of phonetic models
Multiple contexts can be identified for m. In storage, the phonetic model and its context are stored in step 2006 as n
Recorded as m, c.

Preferably, for every word spoken in the training text, there is a feeneem word base form and a phonetic word base form. At step 2008, the well-known Viterbi registration procedure is applied. That is, each successive phonetic model according to the phonetic base form of a given word is correlated with a sequence of five corresponding fee-neem models according to the fee-neem base form of the given word. The Viterbi alignment procedure is described in detail in the paper by F. JeLinek cited above.

If a phonetic model in a given context is uttered only once, then one row of feeneem models is aligned to it.6 However, as we have chosen in this example, for a given If the phonetic model in context is uttered several times during the training period, it is likely that sequences of different feeneem models will be aligned to the same phonetic model. Different sequences correspond to utterances of the same phonetic model in the same context because one pronunciation is different. That is,
The pronunciations are interpreted differently by the acoustic processor 1004 (of FIG. 1) to produce different label outputs (i.e., fee-neems), and thus different fee-neem strings.

To compensate for different feeneem strings resulting from multiple utterances, an average or composite feeneem baseform is constructed.

The method of constructing a composite feeneem baseform from multiple utterances can be readily understood by looking at paragraph 814 and the later patent applications cited therein.

Whether the contextualized phonetic model (nrn, c) is uttered once or several times, the string of each fee-neem model is IIm.

Associated with C. The process of associating the feeneem string with the corresponding phonetic model (rIm,c) in the text occurs in step 2010.

As pointed out above, 4900 items would be stored for each phonetic model if each phonetic model were uttered in every possible context. For 70 phonetic models, 4900 x 70 = 34 in storage
This will result in 3,000 items. As pointed out below, this large number of items increases the time required for training, which is not desirable in typical speech recognition environments.

Therefore, rather than providing each possible context and its associated Feeneem-type model string, the preferred mode is to yield Feeneem-type models for only a portion of the 343,000 possible combinations.

Only the selected context is uttered during the training period, and only the Fee-Neem model sequence for it is associated with it. The feeneem model string associated with the contextualized phonetic model (rIm,c) is stored as a table entry (see step 2012).

To construct feeneem base forms for "new" words that were not uttered during the training period, perform step 2014.2016.2018. Step 2
At 014, new words are represented as strings of phonetic models, each in a defined context. The phonetic model of each new word (n' m, c) is then correlated with the contextualized stored phonetic model (rIm, c).

If items are memorized for all 343,000 context combinations, there is a one-to-one correlation. If only selected items are stored, the correlation of step 2014 is a close matching process, as discussed in more detail below.

In step 2016, a phonetic model (n'
m,c) is represented by a feeneem string associated with a correlated phonetic model nm,c.

The above steps are carried out for each new word phonetic model (rIm, c), and the resulting various feeneem
The strings are concatenated in step 2018 to give the feeneem base form of the new word.

A specific method for decomposing the feeneem baseform into pieces of phonetic element size (as required in each step of FIG. 9) is shown in FIG.

In Figure 10, the first word (I←1) is picked up (
Step 2100). The phonetic base form FBI of the first word is known and during training one or more feeneem base forms FBI are generated. For each feeneem baseform of the first word, the well-known Viterbi alignment procedure is applied to the phonetic baseform (step 2102). In step 2104, the first feeneem element j←1 is picked. If there are multiple feeneem base forms for the word, then in step 2106 it is necessary to determine a single representative feeneem string for the jth phonetic element (step 210
8). The feeneem string F ( Pj) are associated. F(Pj) is 1,
Preferably, it is represented by a number or other identifier that corresponds to the feeneem model string in question. this is,
This is executed in step 211o. At step 2112
The value of 6j is incremented if 6j exceeds the number of phonetic elements in the phonetic base form (step 2114).

The next word is selected based on step 2116 and the procedure resumes at step 2102. If j does not exceed the number of phonetic elements, then steps 2106-2114 for the next phonetic element in the phonetic base form.
is repeated.

FIG. 11 shows the sample word "CAT" represented as a string of phonetic elements. this is.

In this disclosure, the word “C” is a computer-readable representation of the symbols included in the standard international phonetic alphabet.
Assume that AT'' is not uttered during the training period and that we are exploring the feeneem base form of C T''. In the lower part, we consider how the feeneem base form of the word "'CAT IT" is synthesized according to the present invention.

For each phonetic element in the word “CAT”.

There are corresponding phonetic models, such as the set of models shown in FIG. The probabilities assigned to the various transitions and label outputs are derived from statistics generated during the training period, as outlined at step 814.

FIG. 12 shows a portion of a storage table containing four columns. The first column is the phonetic element of the theme named ■m. However, m is from 1 to 70 (in the alphabet of 70 phonetic elements). There are multiple identified contexts for each phonetic element. In this example, the context of the phonetic element at position 1 Pi is the phonetic element at the previous position P(i-1) and the phonetic element at the next position [P(i+1)
) is based on the phonetic elements found in The second column is.

It is a memorized phonetic element that precedes the phonetic element of the subject. The third column is the remembered phonetic element that comes after the subject phonetic element.

Taking the phonetic element AEI of “CAT” as an example.

The first context can be named AAO-AE 1-AAO. In this case, feeneem is used before and after AEl.
Here comes the first phonetic element of the alphabet. The second context is denoted as AAO-AEl-AEO. AEI
The first phonetic element comes before the AEI, and the second phonetic element comes after the AEI. After listing various contexts that include AAO as a preceding phonetic element, we list contexts that include AEO as a subsequent phonetic element.

This list contains various three-element combinations with AEl as the (intermediate) phonetic element of the theme.

Considering the list of phonetic elements corresponding to AEI, a certain context nm,c (encircled by a dashed line) has a phonetic element that precedes it as KQ and a phonetic element that follows as TX.
It is. This context matches the context found in the word "CAT". Based on the data obtained during training, a feeneem string named f is associated with the context of KQ-AEI-TX. As pointed out above, the string f is KQ during training.
-AEl-TX context may be the result of being uttered once or several times.

The string f corresponds to the phonetic element AEI occurring in the KQ-AE 1-TX context.

In forming the feeneem baseform of the word "CAT", the f-string is associated with the intercept corresponding to the phonetic element AEI of the word "CAT".

For other phonetic elements in the word "CAT".

A corresponding feeneem model string is derived. That is, S I LENCE-KQ-A
The feeneem model string associated with the EI is recorded. Also, the finem model string for TX narrowed to AEl and TQ is recorded, and so on. The various feeneem model strings derived for the phonetic elements in the word PACAT'' are concatenated in the order in which each phonetic element occurs in that word.The concatenated feeneem models - The string becomes the composite feeneem base form for the word "CAT".

In this good example, the feeneem associated with it is
Certain phonetic elements may occur in the context of "new words" for which no model string has been memorized. To enable the use of an abbreviated correspondence list between a phonetic model string and a three-element phonetic context, the method of Figure 13 is used.

According to FIG. 13, each "new" word is represented as a string of phonetic elements, each phonetic element representing a slice of the "new" word. The phonetic element corresponding to the segment of the guest word is then identified in its context IT'm,C. Starting from the first word intercept i←1 in step 2400, 2
In step 2402, H'm,c at the 1st position Pi is
A determination is made whether it fully corresponds to a contextualized phonetic element (rIm,c) with an associated feeneem model string. If yes, the entire associated feeneem model string is
Included in Feeneem base form in 4o4.

The notation used in step 2404 (and each step below) is for ease of explanation. Double perpendicular lines represent concatenation operators. “Intercept” on the right side
is added as a tag to that part of the previously constructed base form. The "intercept" to the right of the concatenation operator contains three parameters. The leftmost parameter indicates the decision currently being made. The next parameter is.

Indicates the first feeneem model in the associated feeneem string. The last parameter indicates the last fineme model among the related fineme models to be included in the concatenation, thus the “intercept(gl, 1
, Q(gi))'' refers to the fee-neem model from start to finish that is relevant to the g1 decision (of step 2402), ie.

If the judgment of gl is "yes", the phonetic element n' m, c of the subject of the new word matches the memorized rIm, c (which has the same three-element phonetic context), and associated there is a feeneem model string (starting with model 1 and ending with model Q (gl)). If gl is yes in step 2404, then the entire feeneem model string is “
The new word is added as a tag to the base form constructed for the previous segment. After step 2404, the next word segment is examined in step 2406.

If the phonetic element ITm,c in a "new" word does not map to some remembered phonetic element that has the same three-element phonetic context, then there is a similar two-element phonetic context. A decision will be made whether or not. At step 1410, the phonetic component of the "new" word and its previous phonetic component are taken up in decision g2. Similar Preceding Elements - If the context of the subject element is included in the context of any three elements in the stored list, then the feeneem model string is retrieved. Next, in step 2412, the first half of the Feeneem model string is extracted and concatenated into the Feeneem base form (BSF) being constructed.

A similar test is performed to determine whether the subject phonetic element and the phonetic element that follows have a memorized corresponding context.

This is called a g4 decision and is performed in step 2414. This judgment requires that there be three elements in the list whose last two phonetic elements are the same as the subject phonetic element and the subsequent phonetic element in the segment of the "new" word being featured. Indicates whether the context is included. If so, the second half of the feeneem model string (the first phonemic model is omitted 118) is added as a tag to the base form (bsf) being constructed (see step 2416). Otherwise, in step 2418, the second half of the feeneem string determined based on step 2420 is concatenated to the base form (bsf) under construction.

At step 2420, a determination is made whether a context for a phonetic element with the same phonetic element P (i-1) Pi as in the 36 new word segment being featured is not stored. If not, any memorized phonetic context containing the phonetic element of the subject (i.e., the phonetic element in position Pi of the “new” word segment being featured) (If multiple strings are recorded, one string can be arbitrarily selected). In step 2422, half of the recorded feeneem model strings are Step 2422 is followed by step 2
Proceed to 414.

After the feeneem model string is added to the previously constructed portion of the base form in steps 2404, 2416, the next intercept is picked up until all intercepts of the ``1 new word have been picked up. , are performed in steps 2406 and 2424. The phonetic models derived for each segment are concatenated into the base form of the new word's feeneem model.

According to the invention, synthesis of feeneem-type word-based forms based on the context of phonetic elements is provided.

Can be used for all words or parts of words for which the feeneem base form has not been trained. (each with a known feeneem base form) 2
When two words are combined to form a single word,
Each base form is combined into a composite base form for that single word. For example, the word HOUSE
and BOAT are combined to form a single word HOUSEBOAT. The feeneem base form of the single word HOUSEBOAT is formed simply by combining the feeneem base form of the single word HOUSEBOAT with the feeneem base form of the word BOAT. Therefore, phonetic context methods may, but need not, be used for such words.

Although the invention has been described in terms of preferred embodiments thereof, those skilled in the art will recognize that various changes may be made in form and detail without departing from the scope of the invention. For example, the phonetic context to rely on may not be the three-element context described above.

Instead of two adjacent elements, the highest context can also include any number n (1≦n) of adjacent phonetic elements. Furthermore, the phonetic elements in one context do not need to be adjacent in position, and may be separated by one or more phonetic elements.

Furthermore, we have discussed the phonetic Markov model and the finem Markov model.

The present invention also contemplates the use of other types of models.

That is, one aspect of the present invention is that a word can be represented by a base form of a model included in a first model group and a base form of a model included in a second model group, and that the two base forms can be aligned. Where possible, it is planned to be applied generally.

In addition, "words" as used in this patent application are used in a broad sense, and include dictionary words, lexical elements (i.e., specific pronunciations of dictionary words as mentioned above), and (syllables, etc.) to be recognized. It should be noted that it refers to parts of a word that can be used to define speech.

Also, the method shown in FIG. 13 can be modified if desired. For example, the portion of the feeneem model string that is connected to a pre-existing feeneem model of the base form being constructed may be halved.

Furthermore, it should be noted that the feeneem baseform can be divided into phonetic element-sized word segments in several ways. Besides the N-gram synthesis method described above (which may result in overlapping phoneme sequences in the synthesized base form), longest-best synthesis can also be used. In the latter method,
The audio sequence is divided such that: (a) the number of segments used is minimized, and (b) the length of the longest segment used is maximized.

For example, a longest-best system can compute all possible Feeney-type intercepts corresponding to all possible monophonic strings in a quantity. Next, the criterion function can be defined as follows.

f = Q, "+ Q, "+ (132--...Qn, however, Q□ = length of audio string Q; Q2 = length of two audio strings; the same applies hereinafter. Therefore, QL + Q2...・
...+1n=L=length of the phonetic sequence corresponding to the desired new word.

Next, select the 1ffi intercept that maximizes f.

Note that this should correspond to a linear equation in the ideal case.

Q 1 = L Q z = Q 3...,=
φIn this case, constraint conditions Q, +Q2...+Q
f becomes maximum when n=L.

The invention was implemented on an IBM 3084 computer in the PL/1 language, which embodies both the N-gram synthesis method and the longest best method. In both cases, useful fineme base forms were synthesized.

The synthesized base form provides at least as much performance in the recognition task as its corresponding phonological base form. For example, if the phonetic error rate in a standard task is 4%, replacing all phonetic baseforms with synthesized feeneem baseforms should reduce the error rate below 4%.

The present invention saves at least 150% of the training time by recording the base forms of the 2000 most frequently occurring words and synthesizing the less frequently occurring 3ooO words.
I was able to save money (to 2/3).

DETAILED DESCRIPTION OF THE F1 INVENTION According to the present invention, a method is provided for synthesizing base forms for such other words constructed from models included in a first model group after a training period. Ru.

[Brief explanation of drawings]

Figure 1 is a schematic diagram of a speech recognition system to which the present invention can be applied, Figure 2 is a schematic diagram showing a phonetic Markov model, and Figure 3 is a label for the phonetic Markov model in Figure 2. FIG. 4 is a schematic diagram showing a grid or trellis structure showing spacing measured over several label output intervals starting from the first label in a string of labels produced by an audio processing device. A schematic diagram showing a lattice or trellis structure similar to FIG. 3; FIG.
” and the phonetic representation of the given pronunciation of the word rtTHE
Figure 6 shows three connected phonetic Markov models forming the phonetic base form of ``.
FIG. 7' is an explanatory diagram showing a lattice or trellis structure between the number label output intervals corresponding to the Feenim Markov model;
FIG. 8 is a diagram showing a fineem Markov model concatenated to form a word, FIG. 9 is a block diagram generally showing the method of the present invention, and FIG. 10 is a diagram showing a fineem base form connected to form a word. Fig. 11 is a phonetic representation of the single i "CAT", and Fig. 12 is a flowchart showing how to divide it into segments corresponding to the size of the phonetic elements. An illustration of a memory table showing the association of a feeneem string and its corresponding phonetic model in a given context, Figure 13, shows that the list in Figure 12 does not include all possible phonetic contexts. 2 is a flowchart illustrating the synthesis of feeneem baseforms, some of which are shown. Applicant International Business Machines Corporation Sub-Agent Patent Attorney Sawa 1) Toshio (1 other person) ShiFr.

Claims (1)

[Scope of Claims] A first word-based form represented by a chain of models of a first set is a second word-based form represented by a chain of models of a second set.
The following means (a) for synthesizing from word-based forms of
A word-based form synthesizer for speech recognition having (b) (a) Storage means for storing a chain of models of the first set corresponding to each of the models of the second set for each context. (b) means for determining each of the second set of models forming a second word base form of the word and a respective context; (c) means for retrieving from the storage means a corresponding chain of models of the first set based on the determined type and context of the second set of models; (d) means for combining the retrieved first set of model chains to generate a first word-based form;