JPH09258767A - Voice spotting device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To output information sounding like an unknown word interval by spotting word and phrase voice from voice containing unknown words, and also spotting the unknown word interval. SOLUTION: Heuristic forward likelihood 7 and heuristic backward likelihood 8 are computed using a heuristic phrase model 12 modeled on a time series of feature vectors of phrase voice of various talk contents. Phrase spotting is performed using a phrase model 13 modeled on a time series of spectral feature vectors of phrase voice to be a spotted object, and the heuristic forward likelihood 7 and heuristic backward likelihood 8. With an unknown word model 15 modeled on a time series of spectral feature vectors of optional voice, and the heuristic forward likelihood 7 and heuristic backward likelihood 8 as input, the voice interval of an unknown word is spotted.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a voice spotting device for spotting words and phrase voices from a voice containing an unknown word, and also spotting an unknown word section to output information about an unknown word section. is there.

[0002]

2. Description of the Related Art Conventionally, a technique for detecting a specific word or phrase, or a partial sentence having a semantic cohesion from a continuously uttered voice is generally called spotting. Here, the word spotting is taken as an example to explain the spotting technique. To spot a word means, specifically, from the continuously uttered voice, the utterance start time (hereinafter referred to as start time), the utterance end time (hereinafter referred to as end time), and the spotting score of the word to be spotted. It means asking. Here, the spotting score is a numerical representation of the reliability of whether or not the word actually exists in the spotted voice section, and generally, the spotting score is the spectral feature of the word voice to be spotted. It is obtained by calculating the similarity between the word model that models the sequence and the spectral feature time series of the input speech.

In the method of performing continuous speech recognition by spotting words, after all the words to be usually spotted are spotted, the spotting result is sent to a module for performing language processing (hereinafter referred to as language processing section). The section connects the spotted words by using various linguistic information to assemble a sentence and performs processing such as meaning extraction. At this time, the language processing section can use syntax rules as the language information,
It is also possible to perform the above processing using only the semantic information of words without using syntax rules. Therefore, the degree of freedom of recognizable sentences is much higher than that of the sentence-driven continuous speech recognition method in which the recognizable sentence syntax, that is, the connection rule between words is given to the top-down and the recognition process is performed in sentence units. It has the advantage of becoming large.

However, in the method of performing continuous speech recognition by spotting words, it is necessary to compare the spotting scores of the words in the process of connecting the spotted words and assembling a sentence. Spotting method ("Word spotting based on HMM phoneme recognition", IEICE technical report, SP
88-23, IEEJ Vol.88 No.92, pages 17-22, Kawabata
Australia, Toshiyuki Hanazawa, Kiyohiro Shikano, June 24, 1988, published by The Institute of Electronics, Information and Communication Engineers), the comparison of spotting scores between words spotted at different times on the time axis is accurate. There was a problem of lacking.
Further, in the start-end free spotting method, even in a section in which the word to be spotted does not exist in the input speech, the word to be spotted is often erroneously included in a section having a high acoustic similarity to the word. There is a problem in that a phenomenon called "welling error" is likely to occur that is spotted.

In recent years, as a word spotting method for reducing the above two problems, a method called "heuristic spotting" is "word spotting in conversational speech using a heuristic language model".
(Journal of the Institute of Electronics, Information and Communication Engineers, Vol.J78-D-II No.7, pages 1013-1020, Tatsuya Kawahara, Toshihiko Sotsugi, Shuji Doshita, July 1995.
The proposal was made on the 25th, published by the Institute of Electronics, Information and Communication Engineers Information and Systems Society (hereinafter referred to as Reference 1).

FIG. 5 is a block diagram showing the word spotting method described in Document 1 above. In the figure, 1
Is an input end of the audio signal, 2 is an audio signal input from the input end 1 of the audio signal, and 3 is an analysis means for calculating the time series of the acoustic feature vector of the audio signal 2. Also 4
Is a time series of the feature vector output from the analysis unit 3, and 5 is a heuristic language model that models the time series of the feature vector of speech of various utterance contents,
Reference numeral 6 is a heuristic language model matching that inputs a time series 4 of feature vectors of the input speech and uses a heuristic language model 5 to calculate a heuristic forward likelihood 7 and a heuristic backward likelihood 7 for the time series 4 of the feature vectors of the input speech. It is a means. Furthermore, 9 is a word model that models the time series of the spectral feature vector of the word speech to be spotted,
Numeral 10 is a spotting means for inputting a word model 9, a time series 4 of feature vectors, a heuristic forward likelihood 7 and a heuristic backward likelihood 8 and performing word spotting using the word model 9. 11 is a word spotting result.

In such a configuration, the heuristic language model 5 and the word model 9 are HMM (Hidden Marko).
v Model, Hidden Markov Model). Since the HMM is used, the similarity between the heuristic language model 5 or the word model 9 and the time series 4 of the feature vector is a probabilistic measure that the time series 4 of the feature vector is generated from the heuristic language model 5 or the word model 9. Likelihood is used. In this spotting method, it is assumed that the input speech does not have disordered utterance content but satisfies a certain linguistic constraint, and the linguistic constraint is expressed by a model called a heuristic language model. Then, the heuristic language model 5 and the word model 9 to be spotted are used together to calculate a spotting score in consideration of the entire input voice.

Specifically, as shown in FIG. 6, a heuristic language model 5 is connected before and after the spotting target word model 9 to model the entire input voice. That is, the section in which the spotting target word exists in the input speech is modeled by the spotting target word model 9, and the other speech section is modeled by the heuristic language model 5, and is characterized by the method described later. The likelihood calculation for the time series 4 of the vector is performed to obtain the spotting score.

With this configuration, a case of spotting words relating to personal schedule management will be described according to Reference 1. Therefore, as the word model 9 to be spotted, a model of a plurality of words determined by the task of the personal schedule management is used. As a matter of course, the plurality of words are words that are considered to be included in the input voice. As the heuristic language model 5, as shown in FIG. 7, a model in which word models that are exactly the same as the word model 9 to be spotted are connected so that arbitrary chains can be established between the models is used. In FIG. 7, the word model 9 is displayed surrounded by a square.

Assuming that the input speech contains only the words that make up the heuristic language model 5, all the feature vectors are generated by the heuristic language model 5 that allows arbitrary chains between word models. Series 4 can be modeled. On the other hand, when the input speech contains a word other than the words constituting the heuristic language model 5 (hereinafter, such a word is referred to as an unknown word), the heuristic language model 5 configured as described above Although it is not possible to model the time series 4 of the feature vector of the unknown word section, the effect of not being modeled will be described later.

Next, an operation example of this spotting method will be described with reference to FIG. Audio signal 2 inputted from the input terminal 1 of the audio signal, X ₁ X ₂ X ₃ is a time series 4 of feature vectors by the analysis unit 3, ..., is converted into X _T. Here, X represents a feature vector, and the subscript represents the time of each feature vector. This feature vector X is, for example, an LPC cepstrum. The heuristic language model matching means 6 receives the time series 4 of the feature vector output from the analysis means 3 as an input, and uses the heuristic language model 5 to calculate the heuristic forward likelihood 7 S _fw (t) by the equation (3). (t = 1 to T) becomes (6)
S _bw (t) (t = 1 to T), which is the heuristic backward likelihood 8, is calculated by the formulas and output.

As a preparation for explaining the calculation method of the heuristic forward likelihood 7 and the heuristic backward likelihood 8, the forward likelihood α (S _i , t) and the backward likelihood β (S _i , T
t) is defined as in equations (1) and (2), respectively.

[0013]

[Equation 1]

[0014]

[Equation 2]

That is, α (S _i , t) starts a transition from the initial state of the heuristic language model 5 modeled by the HMM at time 0, and is a partial section of the time series 4 of the feature vector X ₁ X ₂ X ₃ , ..., Probability of reaching the state S _i at time t by outputting up to X _t . Also, β (S _i ,
t) shows the time T from the final state of the heuristic language model 5 modeled by the HMM with the time axis reversed.
Probability of starting the transition to and outputting up to X _T X _T-1 X _T-2 , ..., X _{t + 1} , which is a subinterval of the time series 4 of the feature vector, and reaches state S _i at time t. Is.

Using the forward likelihood and backward likelihood defined in this way, heuristic forward likelihood 7 is S _fw (t) (t = 1 to T) and heuristic backward likelihood 8 is S _bw ( t) (t = 1 to T) is calculated. Heuristic forward likelihood 7 S _fw (t) (t = 1 to T)
Using the heuristic language model 5, (3)
X ₁ X ₂ X which is the time series 4 of the feature vector according to the formula
_3, ..., it is obtained by calculating the forward likelihood for X _T.

[0017]

(Equation 3)

The forward likelihood α (S _J , t) is obtained by calculating the recurrence formulas (4) and (5) in the forward direction on the time axis. In the expressions (4) and (5), a _ji is the probability that a transition from the state S _j to the state S _i occurs, b _ji (X _t ).
Is the probability of outputting the feature vector X _t at the transition from the state S _j to the state S _i . [Initial value setting when t = 0]

[0019]

(Equation 4)

[Recursion formula calculation for t = 1 to T, S _i (i = 1 to J)]

[0021]

(Equation 5)

Further, S _bw (t) (t = 1 to T), which is the heuristic backward likelihood 8, uses the heuristic language model 5 to calculate the backward likelihood for the time series 4 of the spectral feature vector by the equation (6). It is obtained by calculating.

[0023]

(Equation 6)

The backward likelihood β (S ₁ , t) is obtained by calculating the recurrence formulas (7) and (8) in the opposite direction on the time axis. [Initial value setting when t = T]

[0025]

(Equation 7)

[Recursion formula calculation for t = T-1 to 1, S _i (i = 1 to J)]

(Equation 8)

The spotting means 10 receives the time series 4 of the feature vector, the heuristic forward likelihood 7 and the heuristic backward likelihood 8 as input, and uses the word model 9 according to equation (9) for each spotting target word. F ⁽ⁿ⁾ (t) which is the spotting score of each word
Calculate (t = 1 to T). Here, the subscript n on the shoulder is the number of the word model, and n = 1, 2, 3, ..., N (N: total number of words to be spotted). Then, when the spotting score F ⁽ⁿ⁾ (t) is equal to or greater than a predetermined threshold value, the word model number n, the start time time ⁽ⁿ⁾ (t), and the end time obtained by the equation (10). The t and the spotting score F ⁽ⁿ⁾ (t) are output as the spotting result 11.

[0028]

[Equation 9]

[0029]

(Equation 10)

The product of the numerator on the right side of the equation (9) is obtained by connecting the heuristic language model 5 after the word model 9 to be spotted, and X ₁ X ₂ X ₃ , which is a time series 4 of feature vectors. ..., which means to obtain the score of X _T as a whole.

The denominator on the right side of the equation (9) is the heuristic forward likelihood 7 at time T, and the likelihood for the entire X ₁ X ₂ X ₃ , ..., X _T which is the time series 4 of the feature vector. Is the likelihood when the word models that form the heuristic language model 5 are connected so that Since all word models 9 to be spotted are included in the word models forming the heuristic language model 5, the right side of the formula is always (11).
The relationship of formulas is established, and the spotting score F ⁽ⁿ⁾ (t)
Takes a value of 0 to 1.

[0032]

[Equation 11]

The forward likelihood α ⁽ⁿ⁾ (S _J ,
t) is the same as the heuristic forward likelihood 7, and is expressed by the following equations (12) and (14) in the forward direction on the time axis.
It is obtained by calculating the recurrence formula of the formula (16). However, as can be seen from the equation (14), the calculation method in the initial state is different from the calculation of the heuristic forward likelihood 7, which is because the heuristic language model 5 is connected in front of the word model 9 to be spotted. , Word forward likelihood α (S _J , t) is calculated. In addition, BTK ⁽ⁿ⁾ (S _J , the following (13), which is a back pointer that stores the start time of word n
It can be obtained by the recurrence formula calculation of formula, formula (15), formula (17), and formula (18). [Initial value setting when t = 0]

[0034]

(Equation 12)

[0035]

(Equation 13)

[T = 1 to T, initial state of word model S ₁
Recurrence formula

[0037]

[Equation 14]

[0038]

(Equation 15)

[T = 1 to T, recurrence formula calculation for S _i (i = 2 to J) other than the initial state of the word model]

[0040]

(Equation 16)

[0041]

[Equation 17]

However,

[0043]

(Equation 18)

As described above, in this spotting method, the heuristic language model 5 is connected before and after the word model 9 to be spotted, and the section where the spotting target word exists in the input voice is The word model 9 to be spotted is modeled, and the other speech sections are modeled by the heuristic language model 5.

X ₁ which is the time series 4 of the feature vector
X ₂ X ₃ , ..., X _{T The} whole likelihood is calculated and the spotting score is obtained. Therefore, the spotting score is always X ₁ X ₂ X which is the time series 4 of the feature vector.
_3, ..., it is assumed in consideration of the entire X _T, comparison of spotting scores between words which are spotted at different times on the time axis can be accurate. In addition, since the heuristic language model 5 models the voice section before and after the word to be spotted, it is possible to reduce the occurrence error.

By the way, the effect of reducing the spring-out error is
The heuristic language model 5 is subject to linguistic restrictions that are assumed when modeling the speech sections before and after the word to be spotted. For example, as described in Reference 1, as the heuristic language model 5, in addition to the chain of the word models 9 described in this example, the time series of the feature vector of the syllable speech appearing in Japanese is obtained for each syllable. A syllable chain model or the like is conceivable in which the modeled syllable models are connected between syllable models to model the time series of the feature vector of an arbitrary syllable string.

The difference between the properties of the above two heuristic language models will be described with an example. Consider, for example, the case where the word "today" is spotted from the voice uttered "today, go to Tokyo." When using the syllable chain model as a heuristic language model, the speech is
Spotting word model (“Today”) + heuristic language model (“to” + “U” + “Kyo” +
A model connection of “U” + “He” + “I” + “KU”) and a heuristic language model (“Kyo” + “U”)
+ And ”+“ U ”) + word model for spotting (“ Today ”) + heuristic language model (“ to ”+
Modeling is possible with two types of connection of models "i" + "ku"). As a result, the word model "today"
Will be spotted not only in the correct position but also in the “Kyo” part of “Tokyo”, causing an error.

On the other hand, when the word chain model is used as the heuristic language model, and the word models constituting the heuristic language model are four words “today”, “Tokyo”, “he”, and “go”, The above-mentioned input voice "Today, go to Tokyo"
Can be modeled only by connecting the wording model for spotting (“Today”) + the heuristic language model (“Tokyo” + “To” + “Go”). It is possible to prevent "today" from being spotted in a part. On the other hand, when the word chain model is used as the heuristic language model, when an unknown word is included in the input speech, it causes a spelling error.

For example, assuming that the word model constituting the heuristic language model is the four words "today", "Tokyo", "he", and "go" as in the above case, the user went to "Tokyo yesterday". In the voice uttered as "," the words "Kiyo" and "I went" are unknown words, but the word models that make up the heuristic language model are "Today", "Tokyo", "He", and "Go." The model having the highest degree of similarity will be forced to model the above-mentioned voice segments of “Kino” and “I performed”. That is, the heuristic language model is modeled by the connection of the model (“Today” + “Tokyo” + “He” + “Go”). At this time, if the word "today" is spotted, the "today" of the word model to be spotted and the "today" of the word models constituting the heuristic language model are the same model. Therefore, the model "today" is modeled by the heuristic language model. The word model "today" of the spotting target word model is spotted in the converted speech section, and a spelling error occurs.

Since the spotting score is a value normalized by the likelihood when the entire input speech is modeled by the heuristic language model as shown in the equation (9), it is composed of the above-mentioned word chain. In the speech segment modeled as "today" by the adopted heuristic language model, the spotting score of "today", which is the target word model, is the highest value (1.0), and unknown words ("kino") appear in the input speech. However, the information that the unknown word may be included is not reflected in the spotting score at all.

On the other hand, when the syllable chain model is used as the heuristic language model, the voice uttered as "I went to Tokyo yesterday" is the syllable model which constitutes the heuristic language model ("Ki"). +
"No" + "U" + "To" + "U" + "Kyo" + "U" +
It can be modeled by connecting "he" + "i" + "tsu" + "ta"), and is a heuristic language model in the "Kinou" section of the uttered speech ("Ki").
Since the likelihood of modeling with “+” of “+” ”is higher than that of modeling with the word model“ today ”to be spotted, in the section of“ Kinou ”in the uttered speech,
Even if the word "today" is spotted, the spotting score is never the highest (1.0).

As described above, as the heuristic language model imposes stronger linguistic restrictions on input speech,
Although the effect of suppressing the source error increases, the source error may increase on the contrary for speech that does not satisfy the assumed linguistic constraint. That is, there is a trade-off relationship between the effect of suppressing the brow-out error and the robustness with respect to speech that does not satisfy the assumed linguistic constraint.

[0053]

DISCLOSURE OF THE INVENTION Problems to be Solved by the Invention
In the spotting method described in (1), there is a trade-off relationship between the suppression effect of the source error and the robustness against speech that does not satisfy the assumed linguistic constraint, and both the suppression effect of the source error and the robustness should be kept high. Had the problem of being difficult. In addition, when using a heuristic language model composed of a chain of word models to be spotted, even if an unknown word is included in the input speech, information that the unknown word may be included is included in the spotting score. May not be reflected at all in this case, and at this time, there is a problem that there is no means for obtaining information that the unknown voice may be included in the input voice.

The present invention has been made in order to solve the above problems, and spots both a speech unit such as a word or a phrase to be spotted and an unknown word section, and an unknown word is included in the input speech. It is an object of the present invention to provide a voice spotting device that can output information that there is a possibility.

[0055]

SUMMARY OF THE INVENTION A voice spotting device according to the present invention comprises an analysis means for performing an acoustic analysis of an input voice and outputting a time series of acoustic feature vectors of the input voice, and a feature vector of voice of various utterance contents. , A heuristic language model that models the time series of the, and a heuristic language model matching means that calculates the heuristic forward likelihood and the heuristic backward likelihood using the time series of the feature vector of the input speech as the input and the spotting target. The speech model to be spotted that models the time series of the spectral feature vector for each speech unit, and the speech model to be spotted using the time series of the input speech feature vector, the heuristic forward likelihood, and the heuristic backward likelihood as inputs. , A spotting means for spotting speech units to be spotted, an unknown word model that models the time series of the spectral feature vector of an arbitrary voice, the time series of the feature vector of the input voice, and the heuristic forward likelihood. , And an unknown word spotting means for spotting the speech section of the unknown word using the unknown word model with the heuristic backward likelihood as input, and the spotting result of the speech unit to be spotted and the speech section of the unknown word. It outputs both the spotting results.

In the speech spotting apparatus according to the next invention, the heuristic language model is a heuristic clause model in which a time series of feature vectors of clause speech having various utterance contents is modeled, and a speech unit to be spotted is a clause. As the spotting target speech model, a bunsetsu model in which a time series of spectral feature vectors of bunsetsu speech to be spotted is modeled is used, and the unknown word spotting means spots a section including an unknown word in units of bunsetsu sections. It is used as a word segment interval spotting means.

Further, the voice spotting device according to the next invention is provided with backward unknown word spotting means for spotting unknown words in a direction opposite to the time axis as unknown word spotting means. This is so that the spotting result of the voice section including is obtained.

Further, the voice spotting device according to the next invention comprises, as the unknown word spotting means, a plurality of unknown word spotting means for spotting a voice section containing a plurality of unknown words having the same end time but different start times. Is.

[0059]

Embodiments of the present invention will be described below with reference to the drawings.

Embodiment 1. FIG. 1 in which parts corresponding to those in FIG. 5 are assigned the same reference numerals shows the configuration of Embodiment 1 of the voice spotting device according to the present invention. Also in the first embodiment, the spotting task is related to the personal schedule management as in the conventional case. The characteristic part of the present invention different from FIG. 5 described in the related art is the unknown word model 15 and the unknown word phrase section spotting means 1
6 is newly provided to spot a phrase section including an unknown word.

The heuristic language model is a heuristic bunsetsu model 12 constructed as a chain of bunsetsu models, and the spotting means 10 also uses the bunsetsu model 13 to perform spotting in units of phrases instead of words. It is a point. FIG.
In the above, the bunsetsu model 13 uses a model in which several models of adjunct words are connected to a model of an independent word constituting a bunsetsu, and a bunsetsu model is prepared for all bunsetsu to be spotted.

Further, the heuristic clause model 12
Is configured to allow all connections between clause models 13, as shown in FIG. In the figure, the portion surrounded by a square represents a word model, and the phrase model 12 is configured as a network of word models. By incorporating the models of all clauses appearing in the spotting task into the heuristic clause model 12, the heuristic clause model 12 can be used to model the time series of the feature vectors for all the utterances if the utterances are within the task.

The use of the heuristic bunsetsu model 12 composed of a chain of bunsetsu models 13 as a heuristic language model is based on the assumption that "the utterance content of the input voice can be expressed as a connection of bunsetsu". This is a language constraint that is stronger than the chain of words described above, but in Japanese, this assumption is satisfied in many cases. It is possible to reduce the occurrence of mistakes.

On the other hand, when a chain of clause models is used as a heuristic language model, a problem arises when the input speech contains an unknown word. In this case,
For the same reason as explained in the case of using a chain of word models as a heuristic language model in the past, even in a bunsetsu section including an unknown word, a feature vector of a bunsetsu section including the unknown word in the spotting target bunsetsu The phrase model with the highest likelihood is spotted for the time series of, which causes a spelling error, and the fact that unknown words may be included in the input speech is completely reflected in the spotting score. Not done.

Therefore, in the voice spotting device according to the present invention, unknown word bunsetsu section spotting means 16 is newly provided, and a bunsetsu section including an unknown word is spotted, so that an input word can include an unknown word. Get information that you have a potential. The unknown word model 15 is modeled by HMM. Unknown word model 15
Needs to be able to model the time series of feature vectors of arbitrary speech in Japanese. In this embodiment, for example, a garbage model is used as the unknown word model 15. The garbage model here means that the transition probability a _ij and the output probability b _ij (X), which are parameters of the garbage model, are calculated by using a large amount of voice data that includes all syllable voices appearing in Japanese without bias. It is a learned model and can model the time series of feature vectors of arbitrary Japanese speech.

However, since the garbage model models the time series of feature vectors of arbitrary Japanese speech with a small number of parameters, the accuracy of modeling is lower than that of the bunsetsu model 13 constituting the heuristic bunsetsu model 12. Therefore, the likelihood for the input voice "To Tokyo" is
It is lower than the phrase model 12 that models “To Tokyo”. However, when the input speech contains an unknown word, the unknown word model 15 composed of the garbage model is more likely than the clause model 16 forming the heuristic clause model 15 in the phrase section including the unknown word. Can be expected to increase.

Next, the operation will be described. The heuristic language model matching unit 6 operates in the same manner as the heuristic language model matching unit 6 described above except that the heuristic clause model is used as the heuristic language model, and the heuristic forward likelihood 7 and the heuristic backward likelihood 8 are obtained. Calculate and output. The spotting means 10 receives the time series 4 of the feature vector, the heuristic forward likelihood 7 and the heuristic backward likelihood 8 as inputs, and uses the clause model 13 to perform the same operation as the conventional spotting means 10, The spotting result 14 of the phrase model 9 is output.

Unknown word phrase section spotting means 16
Spots a phrase segment containing an unknown word. The method of calculating the spotting score of an unknown word is the same as the method of calculating the spotting score in the spotting means 10, but the spotting means 10 uses the phrase model 13 to spot a phrase.
The unknown word phrase section spotting means 16 is different in that the unknown word model 15 is used to spot a phrase section containing an unknown word.

That is, unknown word spotting means 16
Is a spotting score F ^(u) (t ⁾ of the unknown word by using the unknown word model 15 and the equation (19) with the time series 4 of the feature vector, the heuristic forward likelihood 7 and the heuristic backward likelihood 8 as inputs. ) (T = 1 to 1
Calculate T). And the spotting score F
^{If (u)} (t) is greater than or equal to a predetermined threshold value, (2
0) start time time ^(u) (t),
The end time t and the spotting score F ^(u) (t) are output as the unknown word section spotting result 17. If the unknown word is not included in the input speech, the likelihood of the unknown word model 15 configured by the garbage model as described above is lower than the likelihood of the heuristic bunsetsu model 12, and therefore the equation (19) is used. The calculated unknown word spotting score F ^(u) (t) takes a low value in all sections of the time series 4 of the feature vector.

On the other hand, when an unknown word is included in the input speech, the bunsetsu model 13 constituting the heuristic bunsetsu model 12 does not forcefully model the bunsetsu section including the unknown word, but rather the unknown word model. The likelihood is higher when the bunsetsu section including the unknown word is modeled in 15, and as a result, in the bunsetsu section including the unknown word, the spotting score F ^(u) (t) of the unknown word calculated by the equation (19). Has a high value. That is, the unknown word model 15 is spotted in the phrase section including the unknown word. The section in which the unknown word model 15 is spotted can be a bunsetsu section instead of a word because the input speech is modeled for each bunsetsu section because the heuristic bunsetsu model 12 is used as a heuristic language model. is there.

[0071]

[Equation 19]

[0072]

(Equation 20)

Α ^(u) (S _J , t) in equation (19) is the unknown word forward likelihood. In addition, BTK ^(u) (S _J ,
t) is an unknown word forward back pointer. The unknown word forward likelihood is calculated using the unknown word model 15 as follows (2
It is obtained by calculating the recurrence formulas of formulas (1), (23), and (25). Further, the unknown word forward back pointer uses the unknown word model 15 to express the following equation (22):
It is obtained by calculating the recurrence formulas of the formulas (24), (26), and (27). [Initial value setting when t = 0]

[0074]

(Equation 21)

[0075]

(Equation 22)

[T = 1 to T, initial state S of unknown word model S
Recurrence formula calculation for ₁ ]

[0077]

(Equation 23)

[0078]

(Equation 24)

[T = 1 to T, recurrence formula calculation for S _i (i = 2 to J) other than initial state of unknown word model]

[0080]

(Equation 25)

[0081]

(Equation 26)

However,

[0083]

[Equation 27]

Embodiment 2 FIG. 3 in which parts corresponding to those in FIG. 1 are assigned the same reference numerals is the structure of Embodiment 2 of the voice spotting apparatus according to the present invention. A characteristic point of this second embodiment is that the backward unknown word spotting means 18 is used as the unknown word spotting means.

The operation will be described. The heuristic language model matching means 6 operates in the same manner as in the first embodiment to calculate and output the heuristic forward likelihood 7 and the heuristic backward likelihood 8. The spotting means 10 performs the same operation as that of the first embodiment, and outputs the spotting result 14 of each phrase model 9. The backward unknown word spotting means 18 receives the time series 4 of the feature vector, the heuristic forward likelihood 7 and the heuristic backward likelihood 8 as inputs, and uses the unknown word model 15 by the equation (28) to calculate the backward spotting score of the unknown word. F ^(u) _bw (t) (t = T˜1) is calculated.

If the spotting score F ^(u) _bw (t) is greater than or equal to a predetermined threshold value, the start time t, (29)
The end time etime ^(u) (t) and the spotting score F ^(u) _bw (t) obtained by the formula are output as the backward unknown word segment spotting result 19. As is clear from the expression (19) described in the first embodiment and the expression (28) in the second embodiment, in the first embodiment, the spotting score of the unknown word section is the time at all the end times. However, since the start time is determined by the equation (20), the spotting score of the unknown word section at any start time could not be obtained. On the other hand, in the second embodiment, the spotting score of the unknown word section is obtained at all times with respect to the start time. The end time is (2
It is determined by the equation 9).

[0087]

[Equation 28]

[0088]

(Equation 29)

The product of the right side of equation (28) is the unknown word model 1
5 is connected to the heuristic clause model 12 before the time series X ₁ X ₂ X of the feature vector of the input speech.
_3, ..., it is meant to determine the score of the entire X _T. The backward likelihood β ^(u) (S ₁ , t) is the same as the backward likelihood of the heuristic language model described in the conventional description, and is expressed in the following equations (30) and (32) in the reverse direction on the time axis.
It is obtained by calculating the recurrence formula of the formula (34). However, as can be seen from the equation, the calculation method in the final state is different from the backward likelihood calculation of the heuristic language model. This is because the heuristic language model is connected behind the unknown word model and the backward likelihood β ^(u) means to calculate (S ₁ , t). Also, BTK ^(u) _bw (S ₁ , t), which is a back pointer that stores the end time of the unknown word, can be expressed by the following equation (31), (3)
It can be obtained by the recurrence formula calculation of the formulas (3), (35), and (36). [Initial value setting when t = T]

[0090]

[Equation 30]

[0091]

(Equation 31)

[T = T-1 to 1, recurrence formula calculation for final state S _J of unknown word model]

[0093]

(Equation 32)

[0094]

[Equation 33]

[T = T-1 to 1, recurrence formula calculation for S _i (i = 1 to J-1) other than final state of unknown word model]

[0096]

(Equation 34)

[0097]

(Equation 35)

However,

[0099]

[Equation 36]

Third Embodiment FIG. 4 in which parts corresponding to those in FIG. 1 are assigned the same reference numerals shows a configuration of a third embodiment of a voice spotting device according to the present invention. A characteristic point of the third embodiment is that the plural unknown word spotting means 20 is used as the unknown word spotting means.

The operation will be described. The heuristic language model matching means 6 performs the same operation as in the first embodiment to calculate and output the heuristic forward likelihood 7 and the heuristic backward likelihood 8. The spotting means 10 performs the same operation as in the first embodiment,
The spotting result 14 of each phrase model 9 is output.
The plural unknown word spotting means 20 receives the time series 4 of the feature vector, the heuristic forward likelihood 7 and the heuristic backward likelihood 8 as inputs, and the unknown word model 1
F ^(u) (t) (t = 1 to T), which is the spotting score of the unknown word, is calculated by the equation (37) using 5.

Then, when the spotting score F ^(u) (t) is equal to or more than a predetermined threshold value, the start times time ^(u) _k (t), (k = 1 to
K), end time t, and spotting score F _(u) ^k (t)
(k = 1 to K) Multiple unknown word section spotting result 2
Output as 1. Here, the subscript k is for distinguishing K spotting results having the same end time but different start times.

[0103]

(37)

[0104]

(38)

Α ^(u) _k (S _J , t) in the equation (37) is the unknown word forward likelihood. Also, BTK ^(u) _k (S _J ,
t) is an unknown word forward back pointer. The unknown word forward likelihood α ^(u) _k (S _J , t) and the unknown word forward back pointer BTK ^(u) (S _J , t) _k have been described in the first embodiment (2).
Formula 3), Formula (24), Formula (25), Formula (26), (2
It can be calculated by replacing the operation of selecting the maximum value in the equation 7) with the operation of selecting the top K items.

[0106]

As described above, according to the present invention, the time series of the feature vector of the input speech is input, and the heuristic language model that models the time series of the feature vector of the speech of various speech contents is used. The heuristic forward likelihood and the heuristic backward likelihood are calculated, and the time series of the spectral feature vector of the speech unit to be spotted is input by inputting the time series of the feature vector of the input speech, the heuristic forward likelihood and the heuristic backward likelihood. Using the modeled spotting target voice model, spotting is performed for each voice unit to be spotted, and an unknown word model that models the time series of the spectral feature vector of an arbitrary voice and the time series of the feature vector of the input voice. Heuristic forward likelihood , The heuristic backward likelihood and the unknown word model are used to perform the spotting of the voice section of the unknown word, so that the start of the voice section that seems to be the unknown word section in addition to the spotting result for each voice unit to be spotted. It is possible to realize a voice spotting device capable of outputting unknown word spotting in which the time, the end time, and the degree of the unknown word interval are digitized.

According to the next invention, since the heuristic language model is a heuristic bunsetsu model that models the time series of the feature vectors of bunsetsu speech of various utterance contents, the bunsetsu model is used as the spotting target speech model. When bunsetsu spotting is performed, linguistic constraints based on the heuristic bunsetsu model work effectively, and bunsetsu spotting with few spoiling errors can be performed. At the same time, since the heuristic bunsetsu model is used, the input speech is modeled for each bunsetsu section, and the unknown word spotting unit can spot a section containing an unknown word in bunsetsu section units and output the result. Since it is possible to assume that a word is always a phrase, not a word or a syllable, etc., the search interval of language processing can be significantly narrowed.

Further, according to the next invention, the unknown word spotting means performs the unknown word spotting in the opposite direction to the time axis, so that the speech section including the unknown word is spotted at any start time. Since the result can be obtained and the start time hypothesis of the voice section including the unknown word can be freely generated in the course of the language processing and the spotting score can be referred to, flexible language processing can be performed.

According to the next invention, since a plurality of unknown word spotting means are provided as the unknown word spotting means, it is possible to spot a voice section containing a plurality of unknown words having the same end time but different start times. , It is possible to obtain the same effect as that provided with the backward unknown word spotting means.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of a first embodiment of a voice spotting device according to the present invention.

FIG. 2 is a schematic diagram for explaining a heuristic clause model.

FIG. 3 is a block diagram showing a configuration of a second embodiment of a voice spotting device according to the present invention.

FIG. 4 is a block diagram showing a configuration of a third embodiment of a voice spotting device according to the present invention.

FIG. 5 is a block diagram showing a configuration of a conventional word spotting device.

FIG. 6 is a schematic diagram for explaining a method for connecting a heuristic language model and a word model.

FIG. 7 is a schematic diagram for explaining a heuristic language model configured as a chain of word models.

[Explanation of symbols]

 1 voice signal input end 2 voice signal input from voice signal input end 3 analysis means 4 feature vector time series 5 heuristic language model 6 heuristic language model matching means 7 heuristic forward likelihood 8 heuristic backward likelihood 9 words Model 10 Spotting Means 11 Word Spotting Results 12 Heuristic Phrase Model 13 Phrase Model 14 Phrase Spotting Results 15 Unknown Word Model 16 Unknown Word Phrase Section Spotting Means 17 Unknown Word Phrase Section Spotting Results 18 Backward Unknown Word Spotting Means 19 Backward Unknown Words Section spotting result 20 Plural unknown word spotting means 21 Plural unknown word Section spotting result

Claims (4)

[Claims] 1. An analysis means for acoustically analyzing an input voice to output a time series of acoustic feature vectors of the input voice, and a heuristic language model modeling a time series of voice feature vectors of various utterance contents. Heuristic language model matching means for calculating heuristic forward likelihood and heuristic backward likelihood using the heuristic language model with the time series of the feature vector of the input voice as input, and the spectral feature vector of the voice unit to be spotted A speech model targeted for spotting, a time series of the feature vector of the input speech, the heuristic forward likelihood and the heuristic backward likelihood as inputs, and the spotting speech model is used as a spotting target. Spotting means for performing spotting for each voice unit, an unknown word model that models a time series of a spectral feature vector of an arbitrary voice, a time series of the feature vector of the input voice, the heuristic forward likelihood, and a heuristic backward likelihood. And an unknown word spotting means for spotting the voice section of the unknown word using the unknown word model as input, both the spotting result of the voice unit to be spotted and the spotting result of the voice section of the unknown word. An audio spotting device characterized by outputting 2. The heuristic language model is
As a heuristic bunsetsu model that models the time series of the verse speech feature vector of various utterances, the speech unit to be spotted is a bunsetsu, and the spectral feature vector of the bunsetsu speech to be spotted as the spotting target voice model. A bunsetsu model that models a time series is used, and the unknown word spotting means is an unknown word bunsetsu section spotting means for spotting a section including an unknown word in units of a bunsetsu section. The described voice spotting device. 3. The unknown word spotting means,
A backward unknown word spotting means for spotting an unknown word in a direction opposite to a time axis is provided, and a spotting result of a voice section including an unknown word is obtained at an arbitrary start time. The audio spotting device according to claim 1 or 2. 4. The unknown word spotting means,
3. The voice spotting device according to claim 1, further comprising a plurality of unknown word spotting means for spotting a voice segment including a plurality of unknown words having the same end time but different start times.