JPH0372991B2 - - Google Patents

Description

[Detailed description of the invention]

The present invention will be explained in the following order. A. Industrial field of application B. Summary of the disclosure C. Prior art D. Problems to be solved by the invention E. Means for solving the problems F. Examples F1. Labeling of audio input signals F.. Generation F3 Overview of model generation and recognition procedures F4 Label vocabulary generation and conversion of speech into label strings F5 Generation of word models using label strings F6 Recognition process F7 Changes in label alphabets F8 Table G Effects of the invention A INDUSTRIAL APPLICATION FIELD The present invention relates to speech recognition, and more particularly to a speech recognition system that recognizes speech using a statistical Markov model for words in a predetermined vocabulary. B. Summary of the Disclosure An apparatus for modeling words in a speech recognition system using a Markov model based on labels is disclosed. The modeling includes sending a first audio input corresponding to words in the vocabulary to an acoustic processor, converting each of the spoken words into a sequence of standard labels, and converting each of the standard labels into a time interval. correspond to assignable acoustic types;
Representing each standard label as a probabilistic model with multiple states, at least one transition from one state to another, and at least one configurable output probability at a given transition; to an acoustic processor, the acoustic processor converting the selected acoustic inputs into personalized labels, each corresponding to an acoustic type assigned to a time interval; and generating a specific personalized label at a given transition in a given model. The present invention addresses the problem of easily and automatically generating models of words in speech recognition systems. C. Prior Art In modern speech recognition systems, there are two commonly used techniques for acoustic modeling of words. One approach is to use word templates, where the matching process for recognizing words is based on a dynamic programming (DP) procedure. An example of this procedure is IEEE Bulletin on Acoustics, Speech, and Signal Processing ASSP Volume 23 (1975) 67–
F. Itakura's paper “Speech recognition applying the minimum prediction error principle” on page 72 (F. Itakura,
“
MinimumPredictionResidualPrincipleAppliedt
oSpeech Recognition”, IEEE
Transactionson Acoustics, Speech,
andSignalProcessing, Vol.ASSP−23, 1975,
pp67-72) and U.S. Pat. No. 4,181,821. Another approach uses phoneme-wise Markov models suitable for probabilistic training and decoding algorithms. A description of this technique and related procedures can be found in F. Jelinek's article “Continuous Speech Recognition by Statistical Methods” in IEEE Proceedings Volume 64 (1976), pp. 532-556.
Speech Recognition by Statistical Methods”
Proceedings of the IEEE, Vol.64, 1976,
pp.532-556). Three aspects of these models are particularly important. (a) Individuality of words: Word templates are
Built from real samples of Word, so
can be recognized better. Because phonetic-based models are derived from basic forms representing artificial speech, they represent ideally created words that may not occur in reality. (b) Trainability: Markov models are
For example, it is superior to templates because it can be trained by the forward-backward algorithm (as described in the aforementioned Zielynek paper). Word templates use untrained distance measures, such as Itakura distance, spectral distance, etc. (described in the Itakura paper mentioned above).
One exception is R. Bakis' paper “Continuous Speech Recognition with Centisecond Acoustic Conditions” in IBM Research Report RC5971, April 1976 issue.
Bakis, “Continuous Speech Recognition.
Via Centisecond Acoustic States”IBM
Research Report RC5971, April 1976) is a method that enables word template training. (c) Computation speed: Markov models using alpha-beta outputs individually output from acoustic processors can be compared with dynamic programming matching (as used by Itakura) or continuous parameter word templates (as used by Bakis). The calculation speed is also quite fast. D Problems to be Solved by the Invention The purpose of the present invention is to develop an acoustic modeling method that has word individuality like word templates, and yet provides trainability that can be used with individual alphabetic Markov models. The purpose is to provide a method. A further object of the present invention is to provide an acoustic word model for speech recognition that is simple yet fast-acting in the recognition process. E. Means for Solving Problems In generating a word model in accordance with the present invention, first an acoustic signal representing a word is converted from an individual alphabet to a string of standard labels. Each label represents the time interval of the respective word. Each of the standard labels is then replaced by a probabilistic (eg Markov) model to form a basic form model consisting of multiple continuous models (one model for each standard label). At this time, the probability has not been entered yet. These elementary models are then trained with speech samples that generate statistics or probabilities that are applied to the model. The model is then used for actual speech recognition. The advantage of this method is that the generated word model is more detailed than a phoneme-by-phoneme model and is trainable; Statistical matching using these label-wise models is computationally much faster than DP matching using word templates. These advantages are significant relative to the approach presented in the aforementioned Baschi paper. In Bakis's paper, a word is defined as a sequence of states, each of which is considered distinct. Therefore, each word is typically spread over 60 states, giving a vocabulary of 5000
If it is a word, the method by Bakis is
It would be considered 300,000 different states.
According to the invention, each state is identified corresponding to one of on the order of 200 labels. The present invention requires only storage that can store the 200 labels that make up a word simply as a sequence of numbers (representing the labels) rather than as 300,000 states. Furthermore, the label-based model approach of the present invention requires less training data.
Whereas Bakis's method requires each speaker to utter each word for training, in our case, each speaker must utter enough words to set the values associated with the model for 200 standard labels. Just say it out loud. Furthermore, the method shown in Bakis's paper treats two words, such as "boy" and "boys", unrelatedly, but in the present invention, "boy"
Training on standard labels for
Also applies to “boys”. F Example F1 Labeling of Audio Input Signal The preprocessing for speech recognition and model generation in this system is to convert the audio input signal into code and display it. This is for example the case in 1981.
A. Nadasetal, “Continuous Speech Recognition with Automatically Selected Acoustic Archetypes Obtained from Bootstrapping or Clustering,” in ICASSP Bulletin, pp. 1153-1155.
Recognition with Automatically Selected
Acoustic Prototype Obtainedby Either
Bootstrappingor Clustering,”Proceedings
ICAS SP 1981, pp. 1153-1155). For this conversion procedure, intervals of constant length on the order of 1/100 seconds of the acoustic input signal are spectrally analyzed, and the resulting information is used as a “label” or feneme, which refers to the microphonemes obtained from the front end. Basically, any process that represents a minute acoustic type is included in this finite set (alphabet).
Assign a label of your choice to each interval. Each label represents an acoustic type, more specifically a spectral pattern of a characteristic 10 millisecond speech interval. The initial selection of a unique spectral pattern, ie, the generation of a set of labels, is also described in the aforementioned Nadas et al. paper. In the present invention, there are various label sets.
First, there is a finite alpha alphabet (label sequence) consisting of standard labels. Standard labels are generated when the first speaker speaks into a conventional audio processor (which performs clustering and labeling using conventional methods). Standard labels correspond to the first speaker's voice. As the first speaker utters each word of the vocabulary with a set standard label, the acoustic processor converts each word into a string of standard labels (as illustrated in Table 2 and Figure 2). A string of standard labels for each word is written to storage. Second, there are personalized label sets. These personalized labels are
After the standard label and its columns are set, the next speaker (either the first speaker again or another speaker) is generated by providing speech input to the acoustic processor. Preferably, each of the alphabets of the standard label set and each personalized label set includes 200 labels (this number may vary).
Standard labels and personalized labels are interrelated by a probabilistic model associated with each standard label.
In particular, each standard label represents (a) the multiple states and transitions that span from one state to another, (b) the probability of each transition in the model, and (c) the multiple label output probabilities (each of the transitions at a given transition). The output probability is represented by a model with (corresponding to the probability of the standard label model producing a particular personalized label at a given transition based on the acoustic input from the speaker being shaped). The transition probabilities and label output probabilities are set during a training period during which the speaker under training makes known utterances. Techniques for training Markov models are known and are briefly described below. An important feature of this labeling technique is that it can be performed automatically based on acoustic signals and therefore does not require phonetic interpretation. Details of the labeling technique will be described later in connection with FIGS. 5 and 6. F2 Generation of Pheneme-wise Word Models The present invention presents a novel method for generating word models that are simple, automatic, and more accurately represented than those using phoneme-wise models. To generate a word model, the word is first uttered once and a string of standard labels is obtained by the acoustic processor (see Figure 2 for the principle and Table 2 for the sample). Each of the standard labels is then replaced by a Markov model (Figure 3) representing the first and last states and certain transitions that can occur between the states. The result of concatenating the Markov models of these labels is the complete word model. And this model is similar to other Markov models known from the literature, e.g. from the above-mentioned Zielynek paper.
The model can be trained and used in the case of speech recognition. Preferably, the statistical model for each standard label is stored in storage (in the form of Table 3) as a word model formed from that label. In Table 3, standard label models M1~
Each MN may result in three transitions,
Each transition has a transition probability associated with it.
Additionally, each of models M1-MN has 200 output probabilities for each transition (one probability for each personalized label). Each of the output probabilities represents the probability that the standard label corresponding to model M1 will generate the respective personalized label at a particular transition. Since the transition probability and output probability may differ depending on the speaker, they are set during the training period. If desired, transition probabilities and output probabilities can be combined to form composite probabilities, each indicating the probability of producing a given output and performing a given transition. For a given word, the first speaker's utterance determines the order of the labels and therefore the corresponding models. The first speaker speaks all the words of the vocabulary and establishes the respective order of standard labels (and models) for each word.
Then, during training, the next speaker utters a known acoustic input (preferably a word in the vocabulary). From these known acoustic input utterances, transition probabilities and output probabilities for a particular speaker are determined and stored to train the model. By the way, the next speaker only needs to say the number of acoustic inputs needed to set the model probability of 200. That is,
The next speaker does not have to say every word in the vocabulary, but only the number of acoustic inputs needed to train 200 models. The use of label-wise models is even more important in adding words to the vocabulary. Adding a word to the vocabulary requires only a sequence of labels (like saying a new word).
Since the label-wise model (including probabilities for particular speakers) is already written to storage, the only data needed for a new word is the order of the labels. The advantage of this model is that it is extremely simple and easy to generate. Next, details of a preferred embodiment of the present invention will be described. Although the preferred embodiment of the invention is shown using one acoustic processor, the processing program may include, for example, a first processor that first determines the alpha alphabet of the standard label; a second processor that generates a string of labels; a third processor that selects an alphabet of personalized labels; and a fourth processor that converts a training input or word into a string of personalized labels. You can also. F3 Overview of model generation and recognition procedure FIG. 1 shows an overview of a speech recognition system that performs model generation and actual recognition according to the present invention. The input speech is converted into a label string in an acoustic processor 1 using a pre-generated standard label alphabet 2, and in an initial step 3, it is converted into a label string by using the first utterance of each word. A Markov model is generated for each word using the string of standard labels created earlier and the basic Finim Markov model 4 created earlier. The label-based Markov model is intermediately stored (Fineem-based word model 5). Then, in a training step 6, the utterances of several words by the next speaker are matched with the model for each label, and statistics regarding the probability values of transitions and personalized label outputs for each model are generated. In the actual recognition operation, a string of personalized labels resulting from the utterance to be recognized is matched with a statistical word label-by-label model.
The word or multi-word identifier that yields the highest probability of generating a string of personalized labels is provided at the output. In Figure 1, symbols indicate label strings that are uttered (generated) once, symbols indicate label strings that train several utterances to be recognized, and symbols indicate label strings of utterances that are actually to be recognized. shows. F4 Label vocabulary generation and conversion of speech into label strings The steps for generating label alphanumeric characters and actually converting speech into label strings are explained below.
5 and 6 (a description of this procedure can also be found, for example, in the aforementioned Nadas et al. paper). In general, when generating standard labels representing prototypical vectors of acoustic types (specifically, spectral parameters) of speech, a speaker spends about five minutes to obtain speech samples (block 11 of FIG. 5).
In the acoustic processor (details of which will be explained in connection with Figure 6), 30000 vectors of audio parameters of 10 ms each are obtained (block 1).
2). These vectors are then processed in an analysis or vector quantization operation to classify them into 200 clusters, each containing approximately the same vectors (block 13). Such a procedure has already been described in research papers, such as the Arm M. Gray paper “Vector Quantization” (RMGray,
Quantization”, IEEE ASSP Magazine, April
1984, pp. 4-29). For each of the clusters, choose one archetype vector and the resulting 200 archetype vectors as
Store for later reference (block 14). Each such vector represents one acoustic element or label. Typical label alphabets are shown in Table 1. FIG. 6 is a block diagram of a procedure for acoustically processing speech to obtain label strings of utterances.
Audio from the microphone 21 is converted into a digital display by an A/D converter 22. At block 23, a 20 ms window is taken from the digital display, and the windows are taken every 10 ms (with some overlap). For each window, we perform a spectral analysis using Fast Fourier Transform (FFT) to obtain a vector representing each 10 ms of audio (block 2 in Figure 6).
4). The parameters of these vectors are energy values of multiple spectral bands. Each of the current vectors thus obtained is compared in block 25 with the set of prototype vectors generated in the preprocessing described above (block 14). Block 25 determines the prototype vector closest to the current vector, and block 26 outputs the label or identifier of this prototype. Thus, one label appears at the output every 10 milliseconds and the audio signal is available in a coded format, ie a coded alphabet of 200 finem labels. Incidentally, the invention need not be limited to labels generated at periodic intervals; the above description merely corresponds to each label's respective time interval. F5 Generating a Word Model Using Label Strings When generating a word model, each word needed in the vocabulary must be uttered once and then converted to a string of standard labels as described above. . The graphical representation of the label string of a standard label for a word consists of the sequences y ¹ , y ² , ... y ^m that appeared at the output of the sound processor when the word was uttered, as shown in Figure 2. . Consider this string as the basic form of the standard label for each word. To generate a basic model of a word that takes into account changes in the pronunciation of that word, replace each finem yi of a string in the basic form with the basic Markov model M(yi) of that finem. The basic Markov model can be reduced to a very simple form as shown in FIG. Its configuration is initial state Si, final state Sf, transition T1 and
T2, as well as the null transition T0. Transition T1 leads the state from Si to Sf and represents the output of one personalized label. The transition T2 leaves the initial state Si and returns to the initial state Si and represents one personalized output.
Transition T0 leads the state from Si to Sf, but the output of the personalization label is not assigned. In this basic model, (a) one personalized label appears by performing transition T1 only once, (b) several personalized labels appear by performing transition T2 several times, and (c ) By performing the null transition T0, a dropped personalization label appears. The same basic model shown in Figure 3 can be selected for all 200 different standard labels. Of course, more complex basic models could be used or different models could be assigned to each standard label, but in this example,
The model in Figure 3 is used for all standard labels. FIG. 4 shows a complete basic form Markov model for the entire label-based basic form word shown in FIG. Its construction consists of a simple concatenation of basic models M(yi) of all standard labels of a word, connecting the final state of each basic model to the initial state of the next basic model. Thus, a complete basic form Markov model for a word that yields a string of m standard labels includes m basic Markov models. A typical standard number of labels (and thus number of states per word model) is approximately 30-80. The four word labels are shown in Table 2. For example, the word "thanks" begins with label PX5 and ends with label PX2. Generating a Markov model of the basic form of a word means defining the different states and transitions of each word and their interrelationships. To be useful for speech recognition, a model of the basic form of a word must be reshaped by uttering it several times, that is, a statistical model must be created by accumulating statistical values at each transition in the model. Since the same basic model appears in several different words, there is no need to utter each word several times to train the model. This kind of training is called the “forward-backward algorithm” (research paper,
For example, as described in the above-mentioned paper by Zielinek). As a result of training, each transition in the model is assigned a probability value. For example, for one particular state, T1 may have a probability value of 0.5, T2 may have a probability value of 0.4, and T0 may have a probability value of 0.1. Furthermore, for each of the non-null transitions T1 and T2, when the respective transition occurs,
For each of the 200 personalized labels, a list of probabilities is given that indicates how likely it is to appear. Ward's entire statistical model takes the form of a list or table as shown in Table 3. Each base model or standard label is shown in one column of the table, and each transition corresponds to a row or vector with elements of the probability of the individual personalized label (as well as the overall probability of taking each transition). do. When actually storing models for all words, one vector is stored for each word whose components are the identifiers of the basic Markov models that make up the word, and each of the 200 standard labels of Alphabet has a stored probability It is sufficient to include the values in one statistical Markov model. Therefore,
The statistical word model shown in Table 3 does not actually need to be stored as such, but can be stored in a distributed format and the data combined as desired. F6 Recognition Process When actually performing speech recognition, as explained in the first section, utterances are converted into strings of personalized labels. These personalized label strings are then matched against each of the word models to obtain the probability that the personalized label string was produced by the utterance of the word represented by that model. In particular, matching is performed based on each word's match score. Each match score represents a "forward probability" as described in the aforementioned Zielinek paper. The recognition process using a label-wise model is similar to the known process using a phoneme-wise model. The word or words with the highest probability are selected as output. F7 Change in Label Alphabet The standard label alpha used to generate the initial word model and the set of personalized labels used for training and recognition are common, but may not all be the same. However, even though the probability values associated with the labels generated during recognition may differ somewhat from the personalized labels generated during training, the actual recognition results are generally correct. This is possible due to the adaptability of Markov models. However, if the changes in the respective training and cognitive alphas are too large,
Accuracy may also be affected. Also, the probabilities used to train the model for subsequent speakers whose voices are very different from the first speaker may impose limits on accuracy. F8 Table Table 1 shows typical label alphabets. In the table, the two letters roughly represent the sounds of the elements. The two digits relate to vowels, the first digit representing the accent of the sound and the second digit representing the latest identification number. The single digit number is associated with the consonant and represents the most recent identification number. Table 2 shows a sample 4-word label string. Table 3 shows Ward's statistical Markov model (Feneme unit model).

【table】

【table】

【table】

【table】

【table】 · · ·
・ ・ ・
MN SN TN1 0.6 ・ ・ ・

TN2 0.3 ・ ・ ・

TN0 0.1 − − −

G. Effects of the Invention According to the present invention, (a) the label-based Markov model is superior to the phoneme-based model because it represents words at a detailed level, and (b) unlike the word template using DP matching, the label-based Markov model The unit word model can be trained using a forward-backward algorithm, and (c) the number of parameters is determined by the size of the finial alpha rather than the size of the vocabulary, so the required storage capacity depends on the size of the vocabulary. (d) recognition procedures using label-wise models are computationally faster than DP matching and the use of continuous parameter word templates; (e) word models can be done automatically.

[Brief explanation of drawings]

1 is a block diagram of the model generation and recognition procedure according to the present invention; FIG. 2 is a diagram representing the label string of words obtained from the acoustic processor; and FIG.
The figure shows a basic Markov model with one label,
Figure 4 shows the basic form of the word generated by replacing each of the standard labels of the string shown in Figure 2 with the basic Markov model, and Figure 5 shows the process of initial generation of the standard label alphabet. Block Diagram FIG. 6 is a block diagram illustrating the operation of a sound processor to derive a personalized label string for a spoken word. 1...Acoustic processor, 2...Label alphabet, 3...Initial step, 4...Fineem Markov model, 5...Word model, 6...Training step, 7...Statistical Markov model , 8... Recognition process, 12
...Sound processor.

Claims (1)

[Scope of Claim] A speech recognition device characterized by having one of the following components (a), (b) and (c). (a) Means for generating a corresponding sequence of recognition labels in response to an unknown audio input from a set of recognition labels, each representing an acoustic type that can be assigned to a minute time interval. (b) A Markov model sequence formed by selecting a plurality of Markov models from a set of Markov models corresponding to each of a set of standard labels, each representing an acoustic type that can be assigned to a minute time interval. A way to store as a word model. These word models extract the corresponding standard labels according to the pronunciation of each word in the word vocabulary to form a string of standard labels generated for each word, and extract the standard labels in the string. is defined by connecting Markov models corresponding to each. Markov labels corresponding to each of the standard labels above.
The model is common to all words in the word vocabulary above, and includes (i) multiple states, (ii) at least one transition from one state to another, and (iii) at least one transition. , the probability that each of the set of recognition labels is generated by the label generation means. (c) Means for generating a probability that a sequence of labels generated in response to the unknown speech input is generated for each word in the word vocabulary by referring to the word model in the storage means. 2. The speech recognition device according to claim 1, wherein the acoustic type of the set of standard labels is different from the acoustic type of the set of recognition labels.