JPS62178999A - Voice recognition system - Google Patents

Abstract

(57) [Summary] This bulletin contains application data before electronic filing, so abstract data is not recorded.

Description

[Detailed description of the invention] The present invention will be explained in the following order.

A. Industrial field of application B. Summary of the disclosure C8. Prior art Problems to be solved by the invention E. Means for solving the problems F. Example Fl. Labeling of audio input signals F2. Generation of word-model in feeneem units F'5. Overview of model generation and recognition procedure F4. Label vocabulary generation and conversion of speech into label strings F5. Generating a word model using label strings F6. Recognition process F7. Change of label/alphabet F8. Table G, Effects of the Invention A, Industrial Application Field The present invention relates to speech recognition, and more particularly to a speech recognition system that recognizes speech using a statistical Markov model for words in a predetermined vocabulary.

B. SUMMARY OF THE DISCLOSURE An apparatus for modeling words by a label-based Markov model in a speech recognition system is disclosed. The modeling includes sending a first audio input corresponding to a word in the snow to an acoustic processor, which converts each of the spoken words into a sequence of standard labels, and converts each of the standard labels into a sequence of standard labels over time. Assign each standard label to an acoustic type that can be assigned to an interval; ■ assign each standard label to multiple states, at least one transition from one state to another, and at least one configurable output probability at a transition; ■ sending the selected acoustic inputs to an acoustic processor, which converts the selected acoustic inputs into personalized labels, each corresponding to an acoustic type assigned to a time interval; ■ Setting each output probability as the probability of a standard label exhibited by a given model, and generating a particular personalized label at a given transition in the given model. The present invention addresses the problem of easily and automatically generating models of words in speech recognition systems.

BACKGROUND OF THE INVENTION In modern speech recognition systems, there are two commonly used approaches to acoustic modeling of words. One approach is to use word templates, and the matching process for recognizing words is based on dynamic programming (
Based on the procedure of DP). An example of this procedure is the IEEE Bulletin on Acoustics, Speech and Signal Processing, ASSP Volume 23 (
F. Itakura's paper “Speech Recognition Applying the Minimum Prediction Error Principle” (F. 1975), pages 67-72.

Itakura, Minimum Predicti
onResidual Pr1nciple Ap
plied t.

5peech Recognition”, IEE
ETransactions on Account
ics, 5peech.

and Signal Processing,
Vol, AS 5P-23, 1975, pp67
-72) and US Pat. No. 4,181,821.

Another approach uses phoneme-wise Markov models suitable for probabilistic training and decoding algorithms. A description of this technique and related procedures can be found in F.
Zielinek's paper "Continuous speech recognition using statistical methods"
(F, Jelinek, “Continuous
5peech Recognition by 5
statistical Methods”, Proc.
eedings.

of the I EEE, Vol. 641.19
76, pp.

552-556).

The following three points of these models are particularly important.

(a) Individuality of words: Word templates are
Since it is constructed from real samples of words, it can be recognized better. Because phonetic-based models are derived from basic forms representing artificial speech, they represent ideally created words that may not occur in reality.

(b) Trainability: Markov models are superior to templates because they can be trained, for example, by the forward-backward algorithm (described in the aforementioned Zielinek paper). Word templates use trained distance measures such as Itakura distance, spectral distance, etc. (described in the Itakura paper mentioned above). One exception is R Bakis' paper “Continuous Speech Recognition with Centisecond Acoustic Conditions” (R, Bakis, Conti), published in the April 1976 issue of IBM Research Report Re5971.
naughty 5peech Recognition
nVia Centisecond Acoust
ic 5tates"IBM Re5search
Report RC5971, Apri11976
) is a method that enables training of word templates.

(c) Computation speed: Markov models using alphabets output individually from acoustic processors can be matched by dynamic programming (as used by Itakura) or (
It is much faster to calculate than the continuous parameter word template (used by Bakis).

D0 Problems to be Solved by the Invention The purpose of the present invention is to develop an acoustic modeling method that has individuality of words like word templates, and that can be used with Markov models of individual alphabets, resulting in trainability. The purpose is to provide a method.

A further object of the present invention is to provide an acoustic word model for speech recognition that is simple yet fast-acting in the recognition process.

E9 Means for Solving the Problem When generating a word model according to the present invention, the acoustic signals representing the words are first transformed from the individual alphabets to standard labels. Each label represents the time interval of the respective word. Then,
Each of the standard labels is replaced by a probabilistic (eg Markov) model, forming a basic form model consisting of multiple continuous models (one model for each standard label). At this time, the probability has not been entered yet. These elementary models are then trained with speech samples that generate statistics or probabilities that are applied to the model. The model is then used for actual speech recognition.

The advantage of this method is that the generated word model is much more detailed than a phoneme-based model and is trainable; the number of parameters depends on the number of standard label alphabets (types) rather than the size of the vocabulary. The difference is that statistical matching with these label-wise models is computationally much faster than DP matching with word templates.

These advantages are significant relative to the approach presented in the aforementioned Baschi paper. In Bakis's paper, a word is defined as a sequence of states, each of which is considered distinct. Therefore, if each word was typically spread over 60 states and the vocabulary was 5000 words, the Bakis approach would account for 300,000 different states. According to the invention, each state is identified corresponding to one of on the order of 200 labels. The invention requires only a storage device that can store the 200 labels that make up a word simply as a sequence of numbers (representing the labels) rather than as 300.000 states.

Furthermore, the label-based model approach of the present invention requires less training data. Whereas Bakis's method requires each speaker to utter each word for training, in our case, each speaker must utter enough words to set the values associated with the model for 200 standard labels. Just say it out loud. Furthermore, the method shown in Bakis's paper treats two words, such as "bOypa" and "boys'", independently, but in the present invention, -1
bOyll's standard label training includes:
Also applies to “boys”.

F. Example Fl. Labeling of Speech Input Signal The preprocessing for speech recognition and model generation in this system is to convert the speech input signal into code and display it. For example, 1981 ICASSP Bulletin 11
"Continuous Speech Recognition with Automatically Selected Acoustic Archetypes Obtained from Puttstrup or Clustering" (A, Nada et al., 1993), pp. 53-1155
s et al, Continuous 5pe
ech Recognition with Out
automatically SelectedAcous
'tic Pro, totype 0btaine
d byEither Bootstrappin
g or Clustering,”Procee
dings ICASSP 1981, pp.

1153-1155). For this conversion procedure, the fixed length of the acoustic input signal is
Intervals on the order of 1 o o seconds are spectrally analyzed, and the resulting information is labeled with the label
The microphonemes obtained from the front end are called fen, eme, and so on. Each interval is assigned a label chosen from a finite set (alphabet) consisting of (basically, whatever process is used to represent the smallest acoustic type). Each label represents an acoustic type, more specifically a spectral pattern of a unique 10 millisecond speech interval. The initial selection of a unique spectral pattern, ie, the generation of a set of labels, is also described in the aforementioned Nadas et al. paper.

In the present invention, there are various label sets.

First, there is a finite alphabet (label string) of standard labels. Standard labels are generated when the first speaker speaks into a regular acoustic processor (which performs clustering and labeling using conventional methods). The species label corresponds to the first speaker's voice. As the first speaker utters each word in the vocabulary with a set standard label, the sound processor utters each word (
(as illustrated in Table 2 and FIG. 2) into columns of standard labels. A string of standard labels for each word is written to storage. Second, there are personalized label sets.

These personalized labels are generated by the next speaker (the first speaker again, or another speaker) feeding audio input to the acoustic processor after the standard labels and their columns have been set. .

Preferably, each of the alphabets of the standard label set and each personalized label set includes 200 labels (this number may be different). Standard labels and personalized labels are correlated with each other by a probabilistic model associated with each standard label. In particular, each standard label represents (a) the multiple states and transitions that extend from some 'state to some state, (b) the probability of each transition in the model, and (
c) Multiple label output probabilities (each output probability at a given transition is a model of a standard label that creates a specific personalized label at a given transition based on the acoustic input from the speaker during the next shaping) corresponding to the probability).

The transition probabilities and label output probabilities are set during a training period during which the speaker under training makes known utterances. Techniques for training Markov models are known and will be briefly described below.

An important feature of this labeling technique is that it can be performed automatically based on the acoustic signal and therefore does not require phonetic interpretation.

For details on the labeling method, see Figures 5 and 5 later.
Explanation will be made in conjunction with the figure.

F2. Generating a Pheneme-wise Word Model The present invention presents a new method for generating word models that are simple, automatic, and more accurately represented than those using phoneme-wise models. To generate a word model, the word is first uttered once and a string of standard labels is obtained by the acoustic processor (the principle is explained in Part 2).
(See Table 2 for figures and samples). Each standard label is then a Markov model (Figure 3) representing the first and last states and possible transitions between the states.
replaced by The result of concatenating the Markov models of these labels is the complete Word' model.

This model can then be trained and used in the case of other Marco 7-model speech recognition known from the literature, for example from the above-mentioned Zielynek paper. Preferably, the statistical model for each standard label is stored in storage (in the form of Table 3) as a word model formed from that label.

In Table 5, each of the standard label models M1-MN may result in three transitions, and each transition has a transition probability associated with it. Furthermore, models M1 to MN
each has 200 output probabilities for each transition (one probability for each personalized label). Each of the output probabilities is model M
1 represents the probability that a standard label corresponding to 1 will generate the respective personalized label at a particular transition. Since the transition probability and output probability may differ depending on the speaker, they are set during the training period. If desired, transition probabilities and output probabilities can be combined to form composite probabilities, each indicating the probability of producing a given output and performing a given transition.

When the first speaker utters a given word,
The order of the labels, and hence the order of their corresponding models, is determined. The first speaker says all the words in the vocabulary and
Set the respective order of standard labels (and models) for each word. Then, during training, the next speaker utters a known acoustic input (preferably a word in the vocabulary).

From these known acoustic input utterances, transition probabilities and output probabilities for a particular speaker are determined and stored to train the model. Incidentally, the next speaker only needs to utter the number of acoustic inputs necessary to set the probability of the 200 models. That is, the next speaker does not have to say every word in the idea, but instead only has to say the number of acoustic inputs needed to train the 200 models.

The use of label-wise models is even more important in adding words to the vocabulary. Adding a word to the vocabulary requires only a sequence of labels (like saying a new word). Since the label-wise model (including probabilities for particular speakers) is already written to storage, the only data needed for a new word is the order of the labels.

The advantage of this model is that it is extremely simple and easy to generate. Next, details of a preferred embodiment of the present invention will be described.

Although the preferred embodiment of the invention is shown using one acoustic processor, the processing program includes, for example, a first processor that first determines the standard label alphabet; a second processor that generates a string of labels; a fifth processor that selects an alphabet of personalized labels; and a fourth processor that converts the training input or words into a string of personalized labels. You can also do it.

Overview of Fi Model Generation and Recognition Procedures FIG. 1 shows an overview of a speech recognition system that performs model generation and actual recognition according to the present invention.

The input speech is converted in an acoustic processor 1 into a label string using a pre-generated and standard label alphabet 2, and in an initial step 6 is converted into a label string produced by the first utterance of each word. A Markov model is generated for each word using the string of standard labels created earlier and the basic fee-neem e-Markov model 4 created earlier. The label-based Markov model is stored intermediately (Fineem-based word model 5).

After the recording, in a training step 6, the next speaker's utterances of several words are model-matched on a label-by-label basis, and statistical values regarding the probability values of transitions and personalized label outputs for each model are generated.

In an actual recognition operation, the string of personalized labels resulting from the utterance to be recognized is matched with a statistical word label-by-label model, and the word or words that yield the highest probability of generating a string of personalized labels are selected. Supply the word's identifier to the output.

In Figure 1, the symbol is a label that is uttered (generated) once.
IJ song, the symbol ■ indicates a label string that trains some utterances to be recognized, and the symbol O
indicates the label-string of the utterance to be actually recognized.

F4. Label Vocabulary Generation and Conversion of Audio to Label Strings The procedure for generating label alphabets and actually converting audio into label strings will be explained with reference to FIGS. is also described, for example, in the aforementioned paper by Nadas et al.).

In general, when generating standard labels representing prototypical vectors of acoustic types (specifically spectral parameters) of speech, a speaker speaks for approximately 5 minutes to obtain a speech sample (
Block 11 in FIG. 5).

In the acoustic processor (details of which will be described in connection with FIG. 5), 30,000 vectors of audio parameters of 10 ms each are obtained (block 12). These vectors are then processed with an analysis or vector quantization operation to obtain 200 vectors, each containing approximately the same vector.
(block 13). Such a procedure has already been described in research papers, such as R, M. Gray's paper “Vector Quantization” (R,M, Gray,
Vector Quantization”, I
EEEAS SP Magazine, Apri
l 1984. pp.

4-29).

For each cluster, one prototype vector is selected and the combined 200 prototype vectors are stored for later reference (block 14).

Each such vector represents one acoustic element or label. Typical labels and alphabets first
Shown in the table.

FIG. 5 is a block diagram of a procedure for acoustically processing speech to obtain label strings of utterances.

Audio from the microphone 21 is converted into a digital display by an A/D converter 22. In block 23, 2
A 0 ms window is taken from the digital display, and the windows are acquired every 10 ms (with some overlap). Spectral analysis is performed on each of the windows using a fast Fourier transform (FFT), and each 1
Obtain a vector representing 0 milliseconds of speech (block 24 in Figure 5). The parameters of these vectors are energy values of multiple spectral bands. Each of the current vectors thus obtained is compared in block 25 with the set of prototype vectors generated in the pre-processing described above (block 14). Block 25 determines the prototype vector that is closest to the current vector, and block 26 outputs the label or identifier of this prototype. in this way,
One label appears at the output every 10 milliseconds, and the audio signal is in encoded form, i.e. 200 finems.
Available in label coded alphabet. Incidentally, the invention need not be limited to labels generated at periodic intervals; the above description merely associates each label with a respective time interval.

F5. Generating a Word Model Using Label Strings When generating a word model, each word needed in the vocabulary is uttered once and then converted to a string of standard labels as described above.

A graphical representation of the label string of a standard label for one word is shown in Figure 2, as shown in Figure 2, the yl, y2, . .
・ym Consists of 0 columns. Consider this string as the basic form of the standard label for each word.

To generate a basic model of a word that takes into account changes in the pronunciation of that word, replace each fineme yi of a string in the basic form with its basic Markov model M(yi) of the finemem.

The basic Markov model can be reduced to a very simple form as shown in FIG. Its configuration is initial state Si
, a final state Sf, transitions T1 and T2, and a null transition TO. Transition T1 leads the state from 81 to Sf and represents the output of one personalized label. The transition T2 leaves the initial state Si and returns to the initial state Si and represents one personalized output. The transition TO leads the state from Si to Sf, but the output of the personalization label is not assigned. In this basic model, (a) one personalized label appears by performing transition T1 only once, and (b) one personalized label appears by performing transition T2 several times. , (c) By performing a null transition TO, a dropped personalized label appears.

The same basic model shown in FIG. 3 can be selected for all 200 different standard labels. Of course, more complex basic models could be used or different models could be assigned to each standard label, but in this example the model of FIG. 3 is used for all standard labels.

FIG. 4 shows a complete basic form Markov model for the entire label-based basic form word shown in FIG. Its structure is the basic model M(yi) of all standard labels of words.
consists of a simple concatenation of the final states of each base model to the initial state of the next base model. Thus, a complete basic form Markov model of a word that yields a string of m standard labels contains m basic Markov models.

A typical standard number of labels (and thus number of states per word model) is approximately 30-80. The four word labels are shown in Table 2. For example, the word "thanks" is labeled P
It starts with X5 and ends with label PX2.

Generating a Markov model of the basic form of a word means defining the different states and transitions of each word and their interrelationships. To be useful for speech recognition, a model of the basic form of a word must be reshaped by uttering it several times, that is, a statistical model must be created by accumulating statistical values at each transition in the model.

Since the same basic model appears in several different words, there is no need to utter each word several times to train the model.

Such training can be carried out by the so-called "forward puncture word algorithm°" (described in research papers, for example in the above-mentioned Zielinek paper).

As a result of training, each transition in the model is assigned a probability value. For example, for one particular state, T1 is 0
．． 5. T2 may have a probability value of 0.4 and TO may have a probability value of 0.1. Furthermore, for each of the non-null transitions T1 and T2, a list of probabilities is given that indicates what the probability of occurrence is for each of the 200 personalized labels when the respective transition occurs. Ward's entire statistical model takes the form of a list or table as shown in Table 5. Each base model or standard label is shown in one column of the table, and each transition corresponds to a row or vector with elements of the probability of the individual personalized label (as well as the overall probability of taking each transition). do.

If we were to actually store models for all words, we would store one vector for each food whose components are the identifiers of the basic Markov models that make up the word, and the probabilities stored for each of the 200 standard labels of the alphabet. It is sufficient to include the values in one statistical Markov model. Therefore, the statistical word model shown in Table 3 is:
In practice, there is no need to store it that way (although it can be stored in a distributed format and the data combined as needed).

F6. Recognition Process In actual speech recognition, as explained in the first section, utterances are converted into strings of personalized labels. These personalized label strings are then matched against each of the word models to obtain the probability that the personalized label string was produced by the utterance of the word represented by that model.

In particular, matching is performed based on each word's match score. Each match score represents the "forward probability" explained by Zielinek's theory ÷ above. The recognition process using a label-wise model is similar to the known process using a phoneme-wise model. The word or words with the highest probability are selected as output.

F7. Label-Alphabet Variation The standard label alphabet used for initial word model generation and the set of personalized labels used for training and recognition, although common, may not all be identical. However, even though the probability values associated with the labels generated during recognition may differ somewhat from the personalized labels generated during training, the actual recognition results are approximately correct. This is possible due to the adaptability of Markov models.

However, if the variations in the respective training and recognition alphabets are too large, accuracy may be affected. Also, the probabilities used to train the model for subsequent speakers whose voices are very different from the first speaker may impose limits on accuracy.

F8. table Table 1 shows typical label alphabets.

In the table, the two letters roughly represent the sounds of the elements.

The two digits relate to vowels, the first digit representing the accent of the sound and the second digit representing the latest identification number.

The single digit number is associated with the consonant and represents the most recent identification number.

Table 2 shows a sample 4-word label string.

Table 2 shows Ward's statistical Markov model (Feneme unit model).

- to 1 inch NON to 10- w w w w-w w w- to N'ON (OCk Or N 唖W Ln 'OkN++
+h k's ■■'s O) ω's ω writhing cQE knee writhing rorororo sa OP (a) diagonal N's (g) rohe (a) (a) (a) (a) (a) sentry tree (a) )(stomach)(
A) Malo 00 Roro 0 Rorororo O Lo to dumb mak - to (A) size - he 1 to post size dumb ℃ (T) Ro to OO Roro 00000 - 11000 Cl00 mouth 00 Rororo αko■ w Pw F? -w w w FF- to - hit 4 inches to the bottom hero (a) Mak%O'lkoNko%O%ONo<MkoNkohhNrN, h
k watch size u') ＜+ h watch trout (h watch (i)
(I) (I) (I) Rumor! Dimensions Dimension Dimension Dimension Low N (A) Dimension 04 Rumored to Dimension ℃ Dimension Dimension Dimension Dimension Dimension Dimensions Dimensions Dimensions Dimensions Dimensions Dimensions Dimensions Dimensions Dimensions Dimensions Dimensions Dimensions Dimensions Dimensions Dimensions Dimensions Dimensions Dimensions Dimensions Dimensions Dimensions Dimensions Dimensions Low N (A) Dimensions
Rororororororororororo 0p00 (a) Dimension “1 inch to the bottom of 0r-s (h■-1〒-
--P--cN Hehehehehe N Rororo 0 Roro OO Roro OO Roro O mouth coo h- Table 6 MI SI T110.50.030.000.13T
I20.40.020.010.00T100.1--
- M2 S2 T210.73.

T220.22.

T2O0,05--- M555T510.5.

T320.45,.

T300.05-- MN SN TNI O, 6.

TN20.3.

TNOo, 1-- - G0 Effects of the Invention The present invention provides: 1, -) label-based Markov models are superior to phoneme-based models because they represent words at a much more detailed level; (b) word-based Markov models using DP matching; Unlike tenflates, label-wise word models can be trained using a forward-backward algorithm; Memory capacity increases slowly relative to increases in vocabulary size, and (d) recognition procedures using label-wise models are computationally much faster than DP matching and the use of continuous parameter word templates. , ) Word modeling can be done automatically.

[Brief explanation of drawings]

FIG. 1 is a block diagram of the model generation and recognition procedure according to the present invention; FIG. 2 is a representation of the label string of words obtained from the acoustic processor; FIG. 3 is a diagram of the basic Markov model with one label; Figure 4 is a diagram showing the basic format of words generated by replacing each of the standard labels of IJing shown in Figure 2 with the basic Markov model.
FIG. 5 is a block diagram illustrating the process of initial alphabet generation. FIG. 1...Acoustic processor, 2...Label alphabet, 3...Initial step, 4...Fineem Markov model, 5...Word model, 6... Training step, 7...Statistical Markov model, 8...Recognition process, 12.
...Sound processor. Applicant Inta Sega P4x - Mado
Generation and recognition means Fig. 1 yj y2 yi ym ---
---4-111-1----(Toooooooooo F
: W's Feneme Figure 8 M(yl) M(y2) M(y') M(y
m) Word W's Arnim's Markov Hair De'

Claims (1)

Claims: first processor means for generating a set of standard labels, each representing a sound type assignable to a time interval, in response to a first audio input; and utterance of each word in the word vocabulary. second processor means responsive to extracting a corresponding standard label from said set of standard labels to produce a sequence of standard labels for each said word; and personalization, each representing an acoustic type assigned to a time interval. A speech recognition system comprising: fifth processor means for selecting a set of labels in response to a second speech input; and means for forming a respective probabilistic model for each standard label; for each probabilistic model: (a) a plurality of states; (b) at least one transition from one state to another; (c) a transition probability for each of the transitions; (d) for the at least one transition. A speech recognition system comprising means for associating a plurality of output probabilities, each representing a probability of producing each personalized label at a given transition in a model of a given standard label.