JPH06266386A - Word spotting method - Google Patents

Abstract

PURPOSE:To detect a key word in an input speech precisely at a high speed in synchronism with the time of the input speech. CONSTITUTION:A likelihood calculation part inputs a time series of speech feature quantities from a speech analytic part 2 and finds the likelihood between all partial word series consisting of recognition object words and unknown words, which are receivable from the start state of automaton set in a storage part 9 to respective stages and the feature series of the input speech from a speech start end to each time in synchronism with time by using a key word and a garbage hidden Markov model showing partial word sequences in storage parts 7 and 8. A posterior probability calculation part 4 finds the posterior probability at the end of the vocalization of each partial word series including a recognition object word as a tail word and posterior probability in the vocalization of each partial word series at each time. A recognition decision part 5 compares those posterior probability values with each other at each time to decide a recognition object word present in a partial word series.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a word spotting method in which a machine recognizes or detects a keyword present in a voice freely uttered by a human.

[0002]

2. Description of the Related Art In recent years, research and development of voice recognition technology have been actively carried out, and some of them have been commercialized. In particular, if continuous speech recognition can be performed on sentences that are continuously uttered by humans, the interface between many humans and machines can be dramatically improved. However, at present, continuous speech recognition is possible only with a limited number of vocabularies of several hundreds, and if there is an unknown word that is not registered in the recognition device in the input speech, a correct recognition result is obtained. The problem arises that you cannot get it.

The word spotting technique aims to solve such a problem, and continuously detects a sentence voice that is continuously uttered, ambient noise during utterance, or the like in an input voice signal that is added before and after a voice section. This is a technique for estimating whether or not a keyword registered in the recognition device exists at the position, and is a recognition technique that allows the existence of an unknown word in the input voice.

As a conventional technique of such a word spotting method, for example, 29th to 3rd of Proceedings I of the Acoustical Society of Japan Spring Research Presentation Meeting I (March 1990) 29-3.
The article "Telephone voice spotting with HMM" on page 0 (hereinafter referred to as the first method) and I
EEE Transactions on Acoustics, Speech, and S
ignal Processing, Vol. 38, No. 11 (November 1990), pages 1870-1878, "Automatic Recognition of Keywordsin Unco."
nstrained Speech Using Hidden Markov Models "
(Hereinafter referred to as the second method).

In the first method, the recognizing device has a statistical stochastic acoustic model (keyword hidden Markov model) only for the recognition target word, and assuming that each time of the input speech is the end of each recognition target word, The method is to search for a unique start time. A keyword is detected when the estimated temporal length of the word or the likelihood from the stochastic acoustic model for the estimated section is within the threshold range determined for each keyword. ing. Therefore, according to the first method, even if an unknown word is present in the input voice, the keyword that ends at that time can be obtained at high speed in synchronization with the time.

On the other hand, in the second method, the order of appearance of keywords and unknown words in the input speech is defined by a finite state automaton, and a stochastic acoustic model (keyword hidden Markov model) representing the recognition target word and other than speech are used. We use a stochastic acoustic model (garbage hidden Markov model) created by using the noise intervals of and the assumed multiple unknown words. The recognition is performed by finding the likelihood of the input speech for all word model strings including unknown words in the middle that can be accepted by a given automaton, and detecting the word string with the maximum likelihood. It is a thing. As described above, in the second method, by considering the appearance order of words, it is possible to detect a keyword at an incorrect position and reduce omission of correct answer.

[0007]

In the above-mentioned prior art, in the first method, even if an unknown word is present in the input voice, a keyword that terminates at that point in synchronization with time can be obtained at high speed. However, depending on the setting of the threshold value range for keyword detection, there is a problem that a keyword is detected at a wrong position or a correct answer is dropped. To deal with this problem, complicated post-processing using word spotting results is necessary. Also, information on the order of appearance of keywords and unknown words in the input voice,
Since we do not use the stochastic acoustic model for unknown words,
The problem of partial matching that a short word existing in a long word in time may be detected may occur.
In order to solve this, there is a problem that it is necessary to perform post-processing on the word pair for which partial matching may occur, using information on the mutual positional relationship.

On the other hand, in the second method, the order of appearance of the keywords and unknown words in the input speech is considered,
A keyword hidden Markov model that represents the recognition target word,
By using the Garbage Hidden Markov Model created using noise intervals other than utterance and multiple unknown words that are supposed to be used, detection of keywords at the wrong position in the first method, omission of correct answer, partial matching It is possible to reduce However, in this method, the unknown word is regarded as one word, and it is equivalent to performing continuous word recognition to estimate which word string the input speech is accepted by the automaton. It is impossible to find a keyword that ends at each time point in synchronization with the input time, as in the method described in (1), and the recognition result is obtained if the input voice section is fixed, that is, if the utterance has not ended. There is a problem that there is no.

The present invention solves the problems of the conventional word spotting methods represented by the above first and second methods, and synchronizes with the time of the input voice while considering the appearance order of keywords and unknown words. It is therefore an object of the present invention to provide a word spotting method capable of detecting a keyword existing in an input voice and a temporal chain of several keywords at high speed and with high accuracy.

[0010]

In order to achieve this object, in the present invention, first, a finite state automaton that defines the order relation in which a recognition target word to be detected and other unknowns appear, and a recognition target word are defined. We create a keyword hidden Markov model that represents a time series of speech features and a garbage hidden Markov model that comprehensively represents a time series of speech features of unknown words and non-speech feature time series such as noise. Next, when a voice is input from the speaker, all subword sequences consisting of recognized words and unknown words that can be accepted from the set state of the automaton to each state, and the characteristics of the input voice from the beginning of the voice to each time With respect to the sequence, the likelihood is sequentially calculated in synchronization with time by using a keyword representing a partial word string and a garbage hidden Markov model. Further, using this likelihood, the posterior probability when each time is the end of the utterance of each partial word sequence that makes the recognition target word the last word, and when each time is in the middle of utterance of each partial sequence Calculate the posterior probability of. These posterior probabilities are compared for each time, and when the one showing the maximum value corresponds to the case where the utterance end of a certain partial word sequence corresponds, the recognition target word existing in the partial word sequence is a partial word. Recognize that they appear by the time when the maximum value is detected, in the order of appearance in the sequence.

[0011]

According to the present invention, a finite state automaton that defines the appearance order of a recognition target word and an unknown word prepared in advance and a garbage hidden Markov model that represents the recognition target word and a garbage hidden Markov model that comprehensively represents unknown words and noises. Using, all the partial word series consisting of recognition target words and unknown words that can be accepted from the start state of the automaton set to each state, and between the feature series of the input speech from the speech start end to each time, The likelihood is sequentially calculated in synchronization with time, and from this likelihood, the posterior probability when each time is the end of utterance of each partial word sequence such that the recognition target word is the last word, and each time is The posterior probability when the partial word sequence is in the middle of utterance is calculated. Furthermore, these posterior probabilities are compared for each time, and when the one showing the maximum value corresponds to the case where the utterance end of a certain partial word sequence corresponds, the recognition target words existing in the partial word sequence are It is a method to recognize that it appeared by that time. That is, the present invention synchronizes with the time of input whether or not a part of a series of word chains as defined by the automaton is uttered before the input voice is uttered to the end and the voice section is determined. It is a method that can be detected.

[0012]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below with reference to the drawings.

FIG. 1 is a block diagram of a recognition apparatus showing an embodiment of the present invention. In this figure, 1 is a voice input unit, 2 is a voice analysis unit, 3 is a likelihood calculation unit, 4 is a posterior probability calculation unit, 5 is a recognition determination unit, 6 is a recognition result output unit, and 7 is a keyword hidden Markov model storage. Section, 8 is a Carbage hidden Markov model storage section, 9 is an automaton storage section, 10
Is the overall control unit.

The main part of the operation of this recognizing device is the likelihood calculating unit 3, the posterior probability calculating unit 4, and the recognition determining unit 5. First, the speech analyzing unit 2, the keyword hidden Markov model storage unit 7, and the garbage hidden Markov model. The model storage unit 8 and the automaton storage unit 9 will be described below. In addition,
Here, a recognition target word set {N ^w } consisting of N ^w words as a target of word spotting, and N ^g words representing an unknown word or noise other than the recognition target words appearing in the input speech. Unknown word set consisting of {N ^g }
, And ^{let the} vocabulary consisting of N = N ^w + N ^g } in total be the word set {N} used in the finite automaton.

In this recognition apparatus, there are restrictions on the appearance order of the recognition target words and the unknown words other than the recognition target words in the recognizable input speech, and the number of states as shown in FIG. 2 is Q +.
It is assumed that it is specified by one finite state automaton.
In the automaton of FIG. 2, state 0 is a start state, and while this state is used as a starting point, one of the words written on the transition branch is output, and the states are transitioned one after another. As a result, The output word string obtained up to a certain state becomes a word string that can be accepted by this automaton up to that state, that is, a word string that can be recognized by this recognition device. The transition of outputting the word n from the state p to the state q defined by such an automaton will be expressed as δ (p, n) = q. The automaton storage unit 9 stores in advance a finite state automaton that defines the order relationship in which such recognition target words and other unknown words appear.

The voice signal input to the voice input unit 1 is feature-extracted by the voice analysis unit 2 and is converted into a feature amount x _t at a fixed time interval (hereinafter, referred to as a frame). . The feature amount extracted by the voice analysis unit 2 includes a linear prediction analysis method, a Fourier transform method,
Various methods such as a filter bank analysis method can be used.

For each recognition target word and unknown word, the time series of the feature quantity x _t obtained as an output from the voice analysis unit 2 is
It can be expressed by a hidden Markov model that expresses the appearance order and appearance frequency of each word. The basic parameters that characterize the structure of the hidden Markov model of each word n are the number of states J ⁿ , the probability π _j ⁿ that the state j of the hidden Markov model becomes the initial state, and the transition probability a from the state i to the state j. ⁿ _ij, is the symbol output probability of outputting a feature amount x _t in the state i of voice inputted in the state transition to state ^{_{j b ij n (x t)}} . Of the states of the hidden Markov model of each word n, the initial state probability π
_{If the set of j} ⁿ that can be in the initial state instead of 0 is {SI
ⁿ }, and a set of final states that represent the end points of words is represented as {SF ⁿ }. It is assumed that these parameters for the recognition target word are stored in the keyword hidden Markov model storage unit 7 and the unknown words are stored in the garbage hidden Markov model storage unit 8, respectively. These initial state probability π _j ⁿ , state transition probability a _ij ⁿ ,
For the symbol output probability b _ij ⁿ (x _t ), for example, I
EEE ASSP Magazine, Vol.3, No.1 (198
The paper "A
By applying the Baumwelhi re-estimation method introduced in “Introduction to Hidden Markov Models”, it is possible to set the optimum value for each word by using the learning data.

Next, some variables appearing in the description of the operations of the likelihood calculating section 3, the posterior probability calculating section 4 and the recognition determining section 5 in FIG. 1 will be defined as follows.

L _q ⁿ (t, j): Cumulative likelihood up to the frame time t in the state j of the hidden Markov model of the word n leading to the state q of the automaton.

B _q ⁿ (t, j): Back pointer to the optimum state transition path corresponding to L _q ⁿ (t, j).

L _q (t): Maximum cumulative likelihood of a hidden Markov model of a word string reaching the state q of the automaton at frame time t.

N _q (t): The last word name of the word string corresponding to L _q (t).

B _q (t): A value obtained by subtracting 1 from the start frame time of the word corresponding to N _q (t).

Q _q (t): The state number immediately before the state q of the word string corresponding to L _q (t).

P _q ⁿ (t, j): posterior probability until the frame time t in the state j of the hidden Markov model of the word n leading to the state q of the automaton.

PF _q ⁿ (t): posterior probability that the word n reaching the state q of the automaton is the end of utterance at frame time t.

PC _q ⁿ (t): posterior probability that the word n reaching the state q of the automaton is in the middle of utterance at the frame time t.

S _q ⁿ (t): Optimal final state of the hidden Markov model at the frame time t of the word n leading to the state q of the automaton.

FIG. 3 shows an overall flow chart of the word spotting procedure in the recognition apparatus of FIG. 1. Likelihood calculation, posterior probability calculation, and recognition determination are performed in the likelihood calculation unit 3 and the posterior probability calculation unit 4, respectively. The processing performed by the recognition determination unit 5. Here, each process of likelihood calculation, posterior probability calculation, and recognition determination is repeatedly performed at each frame time (). Further, during this period, the processes of likelihood calculation and posterior probability calculation are first repeated for each word reaching the state q of the automaton (and), and then repeated for each state of the automaton (and). The control unit 10 controls this repetitive control. Further, the control unit 10 performs a predetermined initial setting prior to the word spotting process.

The word spotting procedure in the embodiment of the present invention will be described in detail below. Word spotting operates by repeating the following steps 1 to 21. Note that steps 1, 2 and 21 are the processes in the control unit 10, steps 3 to 11 are the likelihood calculator 3, steps 12 to 14 are the posterior probability calculator 4, and steps 15 to 20 are the same. Are processes performed by the recognition determination unit 5, respectively.

<Initial Setting> Step 1 (Initial Setting) First, the following values are set to each variable as initial setting before voice input.

[0032]

[Equation 1]

<Repetition Control of Frame> Step 2 (Repeat every frame time ) For frame time t = 1, 2, ..., T, step 3
To step 21 are repeated. However, here T
Is the total number of frames of input speech.

<Likelihood Calculation> Step 3 (Repeat for each state of automaton ) Steps 4 to 11 are repeated for states q = 1, 2, ..., Q of the automaton.

Step 4 (reaching the state q of the automaton )
Repeat for each word) Steps 5 to 10 for all words n up to the state q of the automaton as given by
Repeat up to.

[0036]

[Equation 2]

Step 5 (Repeat every initial state of word n )
Then , steps 6 to 7 are repeated for all initial states jε {SI ⁿ } of word n.

Step 6 (Determination of Optimal Path> If the initial state j of the hidden Markov model of the word n leading to the state q of the otomaton, the cumulative likelihood L _q ⁿ (t-1, j) up to the frame time t-1. ) Satisfies the following condition,
Perform step 7.

[0039]

[Equation 3]

Step 7 (of the back pointer of the optimum path )
Accumulated likelihood L to frame time t-1 in the initial state j of the hidden Markov model of the state q leading word n reconfiguration) automaton
_q ⁿ a (t-1, j) and optimal path back pointer B _q ⁿ of the corresponding (t-1, j), and re-set as follows.

[0041]

[Equation 4]

Step 8 (repeat every state of word n ) Repeat step 9 for each state (j = 1, 2, ..., J ⁿ ) of word n.

Step 9 (Calculation of Likelihood and Optimal Path) Cumulative likelihood L up to the frame time t in each state j of the hidden Markov model of the word n reaching the state q of the automaton.
_q ⁿ (t, j) and its corresponding back pointer B _q ⁿ (t, j) of the optimal path are calculated by using the parameters of the hidden Markov model of the word n and the feature quantity x _t of the input speech at the frame time t. , Calculate as follows:

[0044]

[Equation 5]

By the operation of this step 9, all partial word sequences consisting of recognition target words and unknown words that can be accepted from the start state of the automaton to each state, and the feature sequence of the input voice from the voice start end to the frame time t Between and, the keywords that represent the partial word string and the likelihood calculated by the Garbage Hidden Markov Model are obtained.

Step 10 (optimum maximum at frame time t
Determination of Final State) The optimal final state S _q ⁿ (t) of the hidden Markov model at the frame time t of the word n that reaches the state q of the automaton is determined by the following equation.

[0047]

[Equation 6]

Step 11 (determination of optimum word string) Each word n reaching the state p of the automaton at frame time t
Of the following L _q (t), N _q (t),
Find _Bq (t) and _Qq (t).

[0049]

[Equation 7]

<Posterior Probability Calculation> Step 12 (Repeat for Each State of Automata ) Steps 13 to 14 are repeated for states q = 1, 2, ..., Q of the automaton.

Step 13 (reaching the state q of the automaton
Repeat for each word) Step 14 is repeated for the word n that reaches the state q of the automaton as given by the following equation.

[0052]

[Equation 8]

Step 14 (Calculation of Posterior Probability) The posterior probability PF _q ⁿ (t) that the recognition target word nε {N ^w } reaching the state q of the automaton is the utterance end at the frame time t at the time t is calculated by the following equation. Then, the posterior probability PC _q ⁿ (t) that the recognition target word or the unknown word nε {N} reaching the state q of the automaton is in the middle of utterance at the frame time t is obtained.

[0054]

[Equation 9]

<Recognition Determination> Step 15 (Determination of Word with Maximum Posterior Probability) Posterior probability PF _q ⁿ (t) that the recognition target word n reaching the state q of the automaton obtained in step 14 is the end of utterance at frame time t. And the posterior probability PC _q ⁿ (t) that the recognition target word or the unknown word n reaching the state of the otomaton is in the middle of utterance at the frame time t, the maximum posterior when the frame time t is the end of utterance of the word Word n with probability
^F and the state q ^{F of} the automaton corresponding thereto, and the word n ^C having the maximum posterior probability when the word is in the middle of utterance and the state q ^C of the automaton corresponding thereto are obtained as follows.

[0056]

[Equation 10]

Here, each maximum posterior probability is defined as follows.

[0058]

[Equation 11]

In step 15, the magnitudes of A and B are further compared. If A is larger, it is determined that there is a word spotting result consisting of the word n ^F ending at the current frame time t. , Go to step 16. When B is larger, it is determined that there is no word spotting result, and the process proceeds to step 21.

Step 16 (Word spotting result
Determination of each word string of) In order to obtain each recognition target word that constitutes the word string such that the last word reaching the state q ^F of the word-spotted automaton is n ^F , q ₀ = q ^F and b ₀ =
Steps 17 to 19 are repeated with t, i = 0 and k = 0.

Step 17 (determination as to whether or not it is a recognition target) If N _qi (b _i ) is a recognition target word, k is incremented by 1 as shown in the following expression and, at the same time, it is registered as a recognition result W _k .

[0062]

[Equation 12]

Step 18 (Termination time and status of the immediately preceding word
Determining the state) _End frame time b _{i + 1} of the word immediately before N _qi (b _i ),
The state of the automaton q _{i + 1 at} which the word has arrived is determined by the following equation.

[0064]

[Equation 13]

Step 19 (whether the beginning of the voice has been reached
Emergence of determination) if, if b i _{+ 1} = 0, all become retrieved finished with it the words back to the beginning of the speech, and then proceeds to step 20. Otherwise, set i = i + 1 and go to step 1
Return to 7.

Step 20 (Word spotting result
The output of the word string W _k , W _k−1 , ..., W ₁ consisting of k recognition target words ending at the frame time t is output to the recognition result output unit 6.

<Frame Time Update Control> Step 21 (Frame Time Update) The frame time t is advanced by one, and if the end of the input voice has not been reached, the process returns to step 2.

With the above-described operation, in the embodiment shown in FIG. 1, the order relation in which the recognition target word and the other unknown words appear, which are created in advance and stored in the respective storage units 7, 8 and 9, appear. We use a defined finite state automaton, a keyword hidden Markov model that represents the speech feature time series of recognition target words, and a garbage hidden Markov model that comprehensively represents the speech feature time series of unknown words and non-speech feature time series such as noise. Under the repetitive operation control of the control unit 10, the likelihood calculation unit 3 detects all the partial word sequences consisting of the recognition target word and the unknown word that are acceptable from the start state of the automaton to each state, and from the speech start end. The likelihood calculated by the keyword representing the subword sequence and the Garbage Hidden Markov Model is synchronized with the time to the feature sequence of the input speech up to the time. They are sequentially required. Similarly, the posterior probability calculation unit 4 uses this likelihood to calculate the posterior probability and the time when each time is the end of utterance of each partial word sequence in which the recognition target word is the last word. The posterior probability when each partial word sequence is in the middle of utterance is calculated. In the recognition determination unit 5, when the one showing the maximum value among these posterior probabilities corresponds to the case where the utterance end of a certain partial word series corresponds, the recognition target word existing in the partial word series is a partial word. It is determined that they have appeared in the order of appearance in the series, and the recognition result is output from the recognition result output unit 6.

Therefore, in synchronization with the time of the input voice,
It is possible to detect a chain of keywords existing up to each time at high speed, and by defining a word string by an automaton, it is possible to minimize detection and omission of erroneous keywords.

[0070]

As described above, in the word spotting method according to the present invention, even a part of a series of word chains that is estimated by an automaton is detected before the input voice is uttered to the end and the voice section is determined. Whether it has been achieved can be detected synchronously with the time of input. Therefore, it corresponds to the problem of the conventional representative method, that is, the problem that the keyword is detected at the wrong position or the correct answer is lost, like the first method, and the partial matching. In order to do this, post-processing using information about the mutual positional relationship of word pairs is necessary. Also, like the second method above, in synchronization with the time of input,
At the same time, the problem that it is not possible to find the keyword that terminates at each point in time and the recognition result is not obtained unless the input voice section is fixed (utterance ends) is solved. In consideration of the appearance order of, the synchronization with the time of the input voice is fast and accurate,
It becomes possible to detect the keywords present in the input speech and the temporal chain of several keywords.

[Brief description of drawings]

FIG. 1 is a block configuration diagram of an embodiment of a recognition device to which the present invention is applied.

FIG. 2 is a diagram showing an example of a finite state automaton used in an embodiment of the present invention.

FIG. 3 is an overall flowchart of a word spotting procedure in the embodiment of the present invention.

[Explanation of symbols]

 1 voice input unit 2 voice analysis unit 3 likelihood calculation unit 4 posterior probability calculation unit 5 recognition determination unit 6 recognition result output unit 7 keyword hidden Markov model storage unit 8 garbage hidden Markov model storage unit 9 automaton storage unit 10 control unit

Claims (1)

[Claims] 1. A finite-state automaton that defines in advance the order relation in which a recognition target word and other unknown words appear.
Create a keyword hidden Markov model that represents the speech feature time series of the recognition target word and a garbage hidden Markov model that comprehensively represents the speech feature time series of unknown words and non-speech feature time series such as noise, and start the automaton. Between all the partial word sequences consisting of recognition target words and unknown words that can be accepted from each state to each state, and the characteristic sequence of the input speech from the beginning of the speech to each time, the keyword representing the partial word string and the garbage hiding Using the Markov model, the likelihood is sequentially obtained in synchronization with time, and using the above likelihood, when each time is the end of utterance of each part word sequence such that the recognition target word is the last word The posterior probability and the posterior probability when each time is in the midst of utterance of each partial word sequence are calculated. When corresponding to the case where the voice termination, word spotting method for the recognition target words that are present in that part word sequence appeared occurrence in sequence in the partial word sequence recognized, characterized in that.