Classifications

• • G—PHYSICS
  • G10—MUSICAL INSTRUMENTS; ACOUSTICS
  • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
  • G10L15/00—Speech recognition
  • G10L15/08—Speech classification or search
• • G—PHYSICS
  • G10—MUSICAL INSTRUMENTS; ACOUSTICS
  • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
  • G10L15/00—Speech recognition
  • G10L15/08—Speech classification or search
  • G10L15/14—Speech classification or search using statistical models, e.g. Hidden Markov Models [HMMs]
  • G10L15/142—Hidden Markov Models [HMMs]
  • G10L15/144—Training of HMMs

Definitions

• This invention relates to pattern classification, and more particularly, to a method and apparatus for classifying unknown speech utterances into specific word categories.
• Speech recognition is the process of analyzing an acoustic speech signal to identify the linguistic message or utterance that was intended so that a machine can correctly respond to spoken commands. Fluent conversation with a machine is difficult because of the intrinsic variability and complexity of speech. Speech recognition difficulty is also a function of vocabulary size, confusability of words, signal bandwidth, noise, frequency distortions, the population of speakers that must be understood, and the form of speech to be processed.
• a speech recognition device requires the transformation of the continuous signal into discrete representations which may be assigned proper meanings, and which, when comprehended, may be used to effect responsive behavior.
• the same word spoken by the same person on two successive occasions may have different characteristics. Pattern classifications have been developed to help in determining which word has been uttered.
• the hidden Markov model is the most popular statistical model for speech.
• the hidden Markov model characterizes speech signals as a stochastic state, or random distribution, machine with different characteristic distributions of speech signals associated with each state.
• a speech signal can be viewed as a series of acoustic sounds, where each sound has a particular pattern of harmonic features.
• harmonic patterns there is not a one-to-one relationship between harmonic patterns and speech units.
• the decision tree classifier is another pattern classification technique Decision tree classifiers imply a sequential method of determining which class to assign to a particular observation Decision tree classifiers are most commonly used in the field of artificial intelligence for medical diagnosis and in botany for designing taxonomic guides
• a decision tree can be described as a guide to asking a series of questions, where the latter questions asked depend upon the answers to earlier questions For example, in developing a guide to identifying bird species, a relevant early question is the color of the bird
• discrete output hidden Markov models For the application of isolated word recognition, a specific class of hidden Markov models is employed, commonly referred to as discrete output hidden Markov models.
• the spoken words are represented by a sequence of symbols from an appropriately defined alphabet of symbols.
• the method used to transform a set of acoustic patterns into a set of discrete symbols is known in the art as vector quantization.
• Prior art implementations of speech recognition systems using hidden Markov models considered the contribution of all parts of the utterance simultaneously.
• the prior art required that the feature extraction and vector quantization steps be performed on the entire utterance before classification can begin.
• hidden Markov model computations require complex mathematical operations in order to estimate the probabilities of particular utterance patterns. These operations include the use of the complex representation of floating point numbers as well as large quantities of multiplications, divisions, and summations.
• This invention overcomes the disadvantage of the prior art by providing a speech recognition system that does not use complex mathematical operations and does not take a great deal of time to classify utterances into word categories.
• a decision tree classifier designed using the hidden Markov model, yields a final classifier that can be implemented without using any mathematical operations.
• This invention moves all of the mathematical calculations to the construction, or design, of the decision tree. Once this is completed, the decision tree can be implemented using only logical operations such as memory addressing and binary comparison operations. This simplicity allows the decision tree to be implemented directly in a simple hardware form using conventional logic gates or in software as greatly reduced set of computer instructions. This differs greatly from previous algorithms which required that a means of mathematical calculation of probabilities be provided in the classification system.
• This invention overcomes the problems of earlier decision tree classifier methods by introducing the hidden Markov modeling technique as an intermediate representation of the training data
• the first step is obtaining a collection of examples of speech utterances From these examples, hidden Markov models corresponding to each word are trained using the Baum-Welch method.
• the training speech is discarded. Only the statistical models that correspond to each word are used in designing the decision tree.
• the statistical models are considered a smoothed version of the training data. This smoothed representation helps the decision tree design prevent over-specialization.
• the hidden Markov model imposes a rigid parametric structure to the probability space, preventing variability in the training data from ruining the tree design process. Estimates for unobserved utterances are inferred based on their similarity to other examples in the training data set.
• FIG. 1 is a block drawing of a speech recognition system that was utilized by the prior art
• FIG. 2 is a block diagram of the apparatus of this invention.
• FIG. 3 is the decision tree
• FIG. 4 is a table of the time index and corresponding vector quantizer symbols for sample utterances; and FIG. 5 is a block diagram of a decision tree design algorithm.
• the reference character 11 represents a prior art speech recognition system for the words “yes”, “no” and “help”.
• the words “yes”, “no” and “help” have been selected for illustrative purposes. Any words or any number of words may have been selected.
• Speech recognition system 11 comprises: a feature extractor 12; a vector quantizer 13; a hidden Markov Model (HMM) for the word “yes” 14; a hidden Markov Model (HMM) for the word “no” 15; a hidden Markov Model (HMM) for the word “help” 16 and a choose maximum 17.
• HMM hidden Markov Model
• HMM hidden Markov Model
• Feature extractor 12 consists of a frame buffer and typical speech recognition preprocessing. Extractor 12 is disclosed in a book by Lawrence R. Rabiner and Biing Hwang Juang entitled “Fundamentals of Speech Recognition,” Prentice-Hall, Inc. , Englewood Cliffs. 1993. which is herein incorporated by reference. An implementation entails the use of a frame buffer of 45 ms with 30 ms of overlap between frames.
• Each frame is processed using a pre-emphasis filter, a Hamming windowing operation, an autocorrelation measurement with order 10, calculation of the Linear Prediction Coefficients (LPC's) followed by calculation of the LPC cepstral parameters.
• Cepstral parameters provide a complete source-filter characterization of the speech.
• the cepstral parameters are a representation of the spectral contents of the spoken utterance and contain information that includes the location and bandwidths of the formants of the vocal tract of the speaker
• An energy term provides additional information about the signal amplitude
• the output of extractor 12 are the aforementioned LPC cepstral parameters of order 10 and a single frame energy term that are coupled to the input of vector quantizer 13.
• Vector quantizer 13 utilizes the algorithm disclosed by Rabiner and Juang in "Fundamentals of Speech Recognition" to map each collection of 1 1 features received from extractor 12 into a single integer
• the above vector quantizer 13 is developed using a large quantity of training speech data.
• the integer sequence that constitutes a discrete version of the speech spectrum is then coupled to the inputs of HMM for the word "yes” 4, HMM for the word “no” 15 and HMM for the word “help” 16 HMM 14, HMM 15 and HMM 16 each individually contains a set of hidden Markov model parameters.
• HMM 15 and HMM 16 each individually contains a set of hidden Markov model parameters.
• this algorithm is known as the forward algorithm.
• HMM 14, HMM 15 and HMM 16 performs their own computations. These computations are time consuming.
• the outputs of HMM 14, HMM 15 and HMM 16 are coupled to the input of choose maximum 17.
• Choose maximum 17 is a conventional recognizer that utilizes a computer program to determine whether HMM 14, HMM 15, or HMM 16 has the highest probability estimate If HMM 14 had the highest probability estimate, then me system concludes the word "yes was spoken into device 18 and if HMM 1 5 had the highest probability estimate the word "no" was spoken into device 18
• the output of choose maximum 17 is utilized to effect some command input to other systems to place functions in these other systems under voice control
• FIG. 2 is a block diagram of the apparatus of this invention.
• the output of speech input device 18 is coupled to the input of feature extractor 12 and the output of feature extractor 12 is coupled to the input of vector quantizer 13.
• the output of vector quantizer 13 is coupled to the input of decision tree classifier 20.
• Decision tree classifier 20 has a buffer that contains the vector quantizer values of the entire spoken utterance, i e yes", “no” "help”
• the decision tree classifier 20 may be represented by the data contained in the table shown in FIG. 3 and the associated procedure for accessing the aforementioned table
• the table shown in FIG. 3 is generated using the statistical information collected in the hidden Markov models for each of the words in the vocabulary, i e ' yes", “no” and "help
• FIG 4 contains several examples of spoken utterances for illustrative purposes, and will be used in the discussions that follow
• the method used to classify utterances with decision tree classifier 20 involves a series of inspections of the symbols at specific time instances, as guided by the information in FIG 3 We begin with step SO for each classification task. This table instructs us to inspect time index 3 and to proceed to other steps based on the symbols found at time index 3, or to announce a final classification when no further inspections are required We will demonstrate this procedure for three specific utterances chosen from FIG
• step SO This line of the decision tree tells us to look at the vector quantizer symbol contained at time index 3 (FIG. 4) for any spoken word. Let us assume that the word "no" was spoken. The symbol at time index 3 for the first word “no” is 7. The column for symbol 7 (FIG. 3) at time 3 tells us to proceed to step S3. This step tells us to look at time index 6 (FIG. 4) The word "no” has a symbol O 96/13830 PCMJS95/13416
• step S3 tells us to proceed to step S8
• step S8 tells us to refer to time index 9 (FIG 4)
• time index 9 At time index 9 for the word "no", we find a symbol value of 8 which informs us to go to column
• step 8 This tells us to classify the answer as a "no", which is correct
• step SO of FIG 3 This line of the table tells us to look at time index 3 (FIG 4) for our chosen word
• the symbol at time index 3 for the first sample utterance for the word "yes” is 7
• the column for symbol 7 (FIG 3) at time 3 tells us to proceed to step S3
• This step tells us to look at time index 6 (FIG 4)
• the word "yes” has a symbol 3 at time index 6
• the symbol 3 column of step S3 (FIG 3) tells us to classify this input correctly, as the word "yes '
• step SO of FIG 3 this line of the table tells us to look at time index 3 (FIG 4) for our chosen word
• the symbol at time index 3 for the word ' help" is 6
• the column for symbol 6 (FIG 3) at time 3 tells us to proceed to step S2
• This step tells us to look at time index 4 (FIG 4)
• the word "help” has a symbol 6 at time index 4
• the symbol 6 column of step S2 (Fig 3) tells us that the word "help should be selected
• the ' EOW" column of FIG 3 tells us where to go when the end of a word has been reached at the time index re ⁇ uested for observation
• the label "n/a” was included for utterances too short to qualify as any of the words in the training set
• the decision table shown in FIG 3 can be used to correctly classify all of the examples presented in FIG 4
• the above decision tree and examples would not perform well on a large data set, as it is far too small to capture the diversity of potential utterances
• Real implementations of the algorithm, which are hereinafter described, would generate tables with thousands of lines and at least several dozen columns
• the above example is meant to illustrate the output of the hidden Markov model conversion to decision tree algorithm, and the procedure implemented in decision tree classifier 20 that uses the resultant data table
• FIG 5 is a flow chart of a decision tree design algorithm
• General decision tree design principles are disclosed by Ro ⁇ ney M Goodman and Padhraic Smyth in their article entitled “Decision tree design from a communications theory standpoint", IEEE Transactions on Information Theory, Vol., 34, No 5, pp 979-997 Sept 1988 which is herein incorporated by reference
• the decision tree is designed using an iterative process
• the precise rule applied for the expansion of each branch of the decision tree is specified
• the desired termination condition depends highly on specific needs of the application, and will be discussed in broader terms
• the termination condition is the total number of nodes in the resulting tree This is directly related to the amount of memory available for the storage of the classifier data structures Thus, this termination condition is well suited to most practical implementations
• This algorithm is contained in box 24 and is repeated iteratively, converting the terminal, or leaf, nodes of the decision tree into internal nodes one at a time, and adding new leaf nodes in proportion to the number of entries in the vector quantizer This continues until the prespecified maximum number of tree nodes has been reached.
• the resulting data structures can then be organized into the, decision tree as has been shown in FIG 3, and stored in the decision tree classifier 20
• the following decision tree design algorithm may be utilized in block 24
• the tree design algorithm proceeds as follows: Two sets of sets are defined.
• the first set, 7, is the collection of observation sets associated with internal tree nodes.
• the second set, L is the collection of observation sets associated with leaf nodes of the tree.
• the set 7 is set to the empty set, and L is a set that contains one element, and that element is the empty set.
• the main loop of the algorithm moves elements from the set L to the set 7 one at a time until the cardinality, or number of elements, in set 7 reaches the pre-specified maximum number.
• the determination of which set contained in L should be moved to set 7 at any iteration is determined using an information theoretic criterion.
• the goal is to reduce the total entropy of the tree specified by the collection of leaf nodes in L.
• the optimal greedy selection for the collection of observation sets is given by
• the optimal set X contained in L is moved to the set 7
• the optimal set X is used to compute the optimal time index for 10 further node expansion. This is done using an information theoretic criteria specified by ⁇ arg min ; ⁇ _(w ⁇ X J ⁇ (/, s) ⁇ ) ?(X ⁇ (/, s) ⁇ )
• the time index specified by this relationship is used to expand the collection of leaf nodes to include all possible leaf nodes associated with the

Landscapes

• Engineering & Computer Science (AREA)
• Computational Linguistics (AREA)
• Health & Medical Sciences (AREA)
• Audiology, Speech & Language Pathology (AREA)
• Human Computer Interaction (AREA)
• Physics & Mathematics (AREA)
• Acoustics & Sound (AREA)
• Multimedia (AREA)
• Image Analysis (AREA)

Applications Claiming Priority (3)

Publications (2)

Family

ID=23285181

Family Applications (1)

Country Status (5)

Families Citing this family (9)

Family Cites Families (2)

• 1995
  • 1995-10-19 WO PCT/US1995/013416 patent/WO1996013830A1/en not_active Ceased
  • 1995-10-19 CA CA002203649A patent/CA2203649A1/en not_active Abandoned
  • 1995-10-19 EP EP95937519A patent/EP0789902A4/de not_active Withdrawn
  • 1995-10-19 AU AU39608/95A patent/AU3960895A/en not_active Abandoned
  • 1995-10-19 JP JP8514641A patent/JPH10509526A/ja active Pending

Also Published As

Similar Documents

Legal Events