JP2002073078A - Voice recognition method and medium recording program thereon - Google Patents

Abstract

PROBLEM TO BE SOLVED: To decrease a calculated quantity a hypothetical history correction. SOLUTION: A hypothesis rejected by hypothetical narrowing down is stored in storage areas 43, 44, 45, etc., of a buffer memory 26 different for every rejection time, final word, or phoneme history. Only a hypothesis stored in the same storage area as that of the starting time of the final word of the hypothesis to be reappraised when correcting the history and a word just before, or phoneme histories until the final word is read from the buffer memory 26, and it is used for a history correction.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a speech recognition method capable of efficiently searching for a hypothesis closest to an input speech from many hypotheses that can be generated by a given grammar, and a program recording thereof. Medium.

[0002]

2. Description of the Related Art In a speech recognition process shown in FIG. 4, an input speech 11 is converted into a time series of feature parameter vector data by an analysis processing unit 12, and a grammar / language model (grammar model and language) is searched by a search processing unit 13. Model) 16
Is checked against the allowable hypothesis. The score, which is the evaluation value of the result of collation with this hypothesis, is stored in the acoustic model 15 corresponding to the hypothesis.
And a sound score indicating the likelihood (similarity) of the input speech 11 and a language score corresponding to the probability that a hypothesis exists. The hypothesis having the highest score is output as the recognition result 14.

[0003] As signal processing in the analysis processing section 12,
Commonly used are filter bank analysis, linear predictive analysis (referred to as LPC),
LPC cepstrum, LPC
Delta cepstrum, mel-frequenc
y cepstral coefficients, called MFCC), delta MFCC, log power, and the like. Acoustic model 15
Hidden Markov Model (HMM) modeled based on probability and statistical theory
Is called the law). The details of the HMM method are disclosed, for example, in Seiichi Nakagawa, "Speech Recognition by Stochastic Model", edited by the Institute of Electronics, Information and Communication Engineers.

[0004] The grammar / language model 16 defines the connection of words for defining a sentence to be recognized, and uses a word network with words as branches, a language probability model, or the like. The probability model of a language uses the existence probability of a single word and the probability that two or more words are connected.
The details of the stochastic model of this language are disclosed, for example, in "Speech Recognition by a Stochastic Model" by Seichi Nakagawa, edited by the Institute of Electronics, Information and Communication Engineers.

[0005] The search processing unit 13 collates the acoustic model 15 corresponding to the word sequence on the word network indicating the connection relationship of the words specified by the grammar with the time series of the characteristic parameter vector data, and the acoustic likelihood. While obtaining an acoustic score indicating (similarity), a language score is obtained from the language model 16 corresponding to the word sequence, and the score of a hypothesis composed of the acoustic score and the language score is calculated as the time (time in units of unit time of the recognition process). Hereafter, the time in this specification means this), the hypothesis with a low hypothesis score is discarded, the hypothesis with a high hypothesis score is left, and the next time is replaced with the hypothesis left at the previous time. On the other hand, if necessary, word expansion is performed, and evaluation is performed again based on the acoustic model 15 and the language model 16.

Next, the most general processing flow for each time in the collation calculation will be described with reference to FIG.
The search processing unit 13 has a list of hypotheses to be calculated at that time, and each hypothesis in the list includes a score of the hypothesis up to the current time and information of a word series representing the history of the hypothesis. Shall be held. First, in the hypothesis generation unit 21, if there is a hypothesis that has been calculated up to the end of the last word of the hypothesis history in the list of hypotheses calculated at the immediately preceding time, the last word of the hypothesis is calculated. Next, a connectable word is obtained from the network of words defined by the grammar, and a new hypothesis is generated by adding the hypothesis to the end of the hypothesis. Make a list of

The in-word score calculation unit 22 calculates the corresponding acoustic model 1 for each hypothesis in the list of hypotheses at the current time.
5 is compared with the input voice, and the acoustic score is added to the score of the hypothesis. Then, the word end score calculation unit 23
In the list of hypotheses at the current time, for the hypothesis in which the acoustic score is added up to the end of the last word of the hypothesis, the grammar / language model 16 is used to calculate the language score corresponding to the existence probability of the final word. to add.

Next, in the hypothesis narrowing unit 24, among the hypotheses that have the same final word in the list of the hypotheses at the current time and the addition of the acoustic score ends up to the same part of the final word, the final word The hypothesis that the word immediately before is the same, or that a certain number of phonemes after the immediately preceding word are the same,
Only the hypothesis with the highest score is left in the list of hypotheses at the current time, and the other hypotheses are rejected. And increase the time by 1,
The process returns to the hypothesis generation unit 21 again.

The calculation for each time is performed at all times of the input voice. In the list of hypotheses at the end time of the input voice, the highest score among the hypotheses whose calculation is completed up to the end of the final word is obtained. Select a hypothesis and use it as the recognition result. Before starting the calculation for each time, one hypothesis with no history and a score of 0 is put in the hypothesis list. In this search method, the hypothesis narrowing unit 24 suppresses an increase in the number of hypotheses to be calculated. As a method of narrowing down hypotheses, a word pair approximation (Word Pair Approximatio) is used to narrow down hypotheses in which the last word of the hypothesis and the word immediately before it are the same.
n), there is a method called phoneme history approximation that narrows down a hypothesis in which a certain number of phonemes are the same from the last word of the hypothesis and the last part of the word immediately before it.
R. Schwartz, S. of the International Conference of EEE ICASPSP'91
Austin, “A Comparison of Several Approximate Alg
orithm for Finding Multiple (N-best) Sentence Hypoth
eses ", phoneme history approximation is based on the IEICE technical report SP96-102 Yoshiaki Noda, Shoichi Matsunaga,
It is disclosed in "Study of approximate calculation method in large vocabulary continuous speech recognition using word graph" by Shigeki Sagayama. Hereinafter, the word immediately before the last word of the hypothesis is referred to as the preceding word, and the arbitrary number of phoneme histories at the end of the word immediately before the last word of the hypothesis is referred to as the preceding phoneme history.

Next, the flow of processing when the search processing unit 13 has a history correction unit and a buffer memory for storing rejected hypotheses will be described with reference to FIG. In the figure, the same parts as those shown in FIG. 4 are denoted by the same reference numerals, and redundant description is omitted. Hypothesis generation unit 21, in-word score calculation unit 22,
The processing up to the word end score calculation unit 23 is the same as that in FIG. The processing of the hypothesis narrowing unit 24 is the same as that of FIG. 4 for narrowing the hypotheses, but records the hypothesis rejected when the narrowing is performed in the buffer memory 26. The history correction unit 25, which is the next process of the word end score calculation unit 23, stores the hypothesis rejected by the hypothesis narrowing unit 24 in the buffer memory 26.
, And replaces the history up to the preceding word of the hypothesis evaluated by the word end score calculation unit 23 with the history of the hypothesis held in the buffer memory 26 before the time during the evaluation, and calculates the score. If the score when replaced with the history of the hypothesis that gives is higher than the score when not replaced, the history of the hypothesis that gives the highest score is the history up to the preceding word of the current hypothesis,
The score of the hypothesis is defined as the highest score.

The reason why such a history correction is required will be described below. When word pair approximation is used, the last word and the preceding word remain the only hypothesis with the highest score in the same hypothesis, so the word sequence from the beginning to the preceding word is
The hypothesis different from the hypothesis with the highest score is lost, and only the hypothesis with the same word sequence from the beginning to the preceding word remains in the hypothesis list. However, when a language model (for example, a word trigram) that gives a language score to three or more word sequences is used, the last word of the hypothesis, the preceding word, and one or more word sequences before the preceding word are used. The language score is calculated using the hypothesis, the final word and the preceding word are the same, and the word sequence up to the preceding word is different. Score may be higher. this is,
This is because language scores for three or more word sequences take different values depending on the word sequences up to the preceding word. Therefore, at a time later than the current time, a hypothesis in which the score of the hypothesis becomes higher than the scores of the other hypotheses is lost by narrowing. Hereinafter, this phenomenon will be referred to as a narrowing error.

Similarly, when the phoneme history approximation is used, only one hypothesis with the highest score remains in the hypothesis in which the last word and the preceding phoneme history are the same, so that the starting hypothesis in the same preceding phoneme history is the same. Hypotheses whose word series from the first word to the preceding word differ from the hypothesis with the highest score are rejected, and only the same hypothesis from the beginning to the preceding word remains in the list of hypotheses for each preceding phoneme history. However, when a language model (for example, a word bigram) that gives a language score to two or more word sequences is used, the language score is calculated based on one or more word sequences up to the last word of the hypothesis and the preceding word. However, if the hypothesis and the final word and the preceding phoneme history are the same and the word sequence up to the preceding word is different, calculating and adding the linguistic score to the already rejected hypothesis may increase the score of the hypothesis. is there. This is because the language score takes different values depending on the word sequence up to the preceding word.

As described above, in the word pair approximation, a language model that gives a language score to three or more word sequences, and in a phoneme history approximation, a language model that gives a language score to two or more word sequences. When used, narrowing errors can occur. Dela as a method to correct history
There is a method called yeard bigram (delayed bigram). All hypotheses recorded in the buffer memory that ended at the start time of the last word of the hypothesis,
This is a method in which the hypothesis up to the preceding word of the hypothesis is replaced with the hypothesis up to the preceding word of the hypothesis, and the history of the hypothesis having the highest score is evaluated based on the bigram (Bigram) probability. Delayed B
igram is an international conference of IEEE ICASPSP '96
M. Woszczyna and M. Finke, “Minimizing search errors
due to delayed bigrams in real-time speech recogn
ition systems ". Delayed
In Bigram, the hypothesis that has been rejected at each time and the score calculation up to the end of the word has been completed is recorded in the buffer memory. Based on the start time of the last word of the hypothesis to be reevaluated, the history correction unit reads out the rejected hypotheses at the start time one by one from the buffer memory and replaces the hypothesis with the history of the reevaluation hypothesis to calculate the score. repeat.

Next, a method of recording a hypothesis rejected at each time will be described with reference to FIG. FIG. 6 shows in detail a portion of passing the hypothesis rejected from the hypothesis narrowing section 24 to the buffer memory 26 in FIG.
Assuming that the time is t, the hypothesis 31 rejected by the hypothesis narrowing unit 24 is added to the area 35 for storing the hypothesis at the time t in the buffer memory 26 by the passing process 27 of the hypothesis. Hereinafter, an area for storing a hypothesis is referred to as a hypothesis storage area. Here, the hypothesis 33 is a hypothesis stored in the hypothesis storage area 35 at the time t before the hypothesis 31 in the process of the hypothesis narrowing unit 24 at the time t. Therefore, hypothesis 31
Are also stored in the area 35 as the hypothesis 33.
Further, the hypothesis storage area in the buffer memory 26 before the time t (for example, the hypothesis storage area 34 at the time t-1).
, The hypothesis rejected at each time (for example, hypothesis 32) is recorded, and nothing is recorded yet in the hypothesis storage area at the time after time t (for example, hypothesis storage area 36 at time t + 1). . With this storage, the start time of the last word of the hypothesis to be re-evaluated is used to store data in the buffer memory 2.
From 6, it is possible to easily find and read out a plurality of hypotheses to be used for correcting the history.

However, if such a hypothesis recording / reading method is used, if the number of hypotheses recorded at the same time increases, the hypothesis recorded in the hypothesis storage area at each time in the buffer memory 26 is increased. And the amount of processing required for the reevaluation of the hypothesis in the history correction unit increases.

[0016]

SUMMARY OF THE INVENTION Delayed Bi
In the history correction method based on the gram, the hypothesis for which the score calculation up to the end of the rejected word has been completed is recorded in the buffer memory for each time, and when the history is corrected,
All the hypotheses recorded in the hypothesis storage area in the buffer memory corresponding to the start time of the last word of the hypothesis to be reevaluated are recorded at the same time in order to calculate the score by replacing the hypothesis with the history of the hypothesis to be reevaluated. If there are many hypotheses,
There was a problem that the amount of calculation required to correct the history was large.

[0017]

According to the present invention, each hypothesis rejected by the hypothesis narrowing is converted into a hypothesis rejection time indicating the rejection time and an arbitrary word up to the last word of the word history or the end of the last word. The phoneme history representing the number of phoneme strings, that is, classified into each history attribute, recorded in the buffer memory, and the same hypothesis rejection time as the start time of the last word of the hypothesis to be reevaluated at the time of history correction, and the same history attribute, Modification of the history of the hypothesis that reads only the hypothesis in the buffer memory that is classified as the same as the preceding word representing the word before the last word of the hypothesis to be re-evaluated or the phoneme history up to the end of the previous word and re-evaluates Use for When a phoneme history is used, the number of phonemes may be approximately the same as the number of phonemes used for the preceding phoneme history in the phoneme history approximation method described in the section of the related art.

[0018]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a diagram showing a method for recording a rejected hypothesis according to an embodiment of the present invention.
In the figure, the same parts as those shown in FIG. 6 are denoted by the same reference numerals, and redundant description is omitted. Hereinafter, this embodiment will be described with reference to FIG. DelayedBi, a conventional method
In the gram, as shown in FIG. 6, the rejected hypothesis is recorded in a separate hypothesis storage area at each time of the buffer memory 26. In contrast, in this embodiment, the rejected hypothesis is stored in the buffer memory 26. Separately for each time,
Further, the history attribute, that is, the last word or phoneme history is separately classified and recorded.

When the time is t, the hypothesis narrowing unit 2
The hypothesis 41 rejected in 4 is added to the hypothesis storage area 46 corresponding to the same phoneme history ac as the phoneme history ac of the hypothesis 41 at time t in the buffer memory 26 by the hypothesis passing process 27. Here, the hypothesis 43 is a hypothesis added to the hypothesis storage areas 45 and 46 corresponding to each phoneme history ab or ac at the time t before the hypothesis 41 in the process of the hypothesis narrowing unit 24 at the time t. . Also, in the hypothesis storage areas of the buffer memory 26 before the time t (for example, the hypothesis storage areas 43 and 44 at the time t-1), the hypotheses rejected at each time (for example, the hypothesis 42) are stored in the phoneme history (for example, Each of the phoneme histories bc, de) is recorded, and nothing is recorded in the hypothesis storage areas at times after time t (for example, the hypothesis storage areas 47, 48 at time t + 1).

As described above, the rejected hypotheses are recorded in the buffer memory for each phoneme history at each time, and the history correction unit determines the start time of the last word of the hypothesis to be reevaluated by the hypothesis rejection time. And finds a hypothesis storage area in the buffer memory where the preceding phoneme history coincides with the phoneme history, for example, the area 45, reads out only the hypothesis recorded in the hypothesis storage area 45, and uses it for history correction. . The rejected hypothesis is stored in a separate hypothesis storage area for each time and for each final word, and when the history is modified, the preceding word of the hypothesis rejection time that matches the start time of the final word of the hypothesis to be re-evaluated is rejected. Only the hypothesis in the hypothesis storage area that matches the last word of the hypothesis may be used for history correction.

As a result, the number of hypotheses recorded in the hypothesis storage area for each last word or phoneme history at each time is determined by the Delayed Bi recorded in the hypothesis storage area for each time.
Since the number is smaller than in the case of the gram, the number of times of changing the history and calculating the score can be suppressed when the history of the hypothesis is corrected. The method of the present invention can also be achieved by causing a computer to execute a program. FIG. 2 shows a functional configuration in that case. Each part is a bus 59
And an analysis program is installed in the memory 54 via a CD-ROM, a magnetic disk, or the like, or a communication line. Similarly, a search processing program is installed in the memory 57. When a voice is input from the input unit 51, the analysis program in the memory 54 is executed by the CPU 58 while being stored in the storage unit 53 as needed, and is converted into a time series of feature parameter vector data. In this example, a word network memory 56a and a language model database 56b are provided as the grammar / language model database 56.

A search processing program for the memory 26 is executed by a CPU.
58, the hypothesis list is generated for the time series of the vector data of the characteristic parameters as described above, that is, at each time as shown in FIG.
1) For the hypothesis, an acoustic score is calculated using the acoustic model in the acoustic model database 55 and added (S)
2) Regarding the hypothesis at the end of the final word, a language score is obtained by using a language model with respect to the added value of the acoustic score up to that point, and the result is added to obtain a hypothesis score (S3). For the largest hypothesis obtained, the buffer memory 26 searches for a hypothesis storage area in which the start time of the last word matches the history attribute immediately before it, reads out the hypothesis in that area, and performs history correction processing ( S4).
Hypothesis narrowing is performed using the hypothesis score after the history correction processing, and rejection hypotheses are classified according to the time and the history attribute and recorded in the storage area of the buffer memory 26 (S
5). If the input speech has not ended (S6), the process returns to step S1, and if it has ended, the hypothesis with the largest score in the hypothesis reaching the end of the final word at that time is output from the output unit 52 as a recognition result ( S7).

The present invention may use a grammar model that allows connection from one word to all words.

[0024]

As described above, according to the present invention, hypotheses rejected during hypothesis narrowing are recorded for each history attribute (last word or phoneme history) at each time, and when hypothesis correction is performed, re-creation is performed. The start time of the last word of the hypothesis to be evaluated matches the hypothesis rejection time, and the preceding word or preceding phoneme history of the hypothesis corresponds to the corresponding history attribute, that is, the last word or phoneme history. Since only the recorded hypotheses are read and used to correct the history, when correcting the history of each hypothesis included in the list of hypotheses at each time, the number of times to replace the history and calculate the score has been This has the effect of reducing the number of calculations and the amount of calculation for history correction.

[Brief description of the drawings]

FIG. 1 is a diagram showing an example of a recording state of a rejected hypothesis in the method of the present invention.

FIG. 2 is a diagram showing an example of a functional configuration when a method of the present invention is caused to function by a computer.

FIG. 3 is a flowchart showing a processing procedure according to the embodiment of the present invention.

FIG. 4 is a diagram showing an outline of a conventional speech recognition process.

FIG. 5 is a diagram showing an outline of a speech recognition process having a history correction unit and a buffer memory.

FIG. 6 is a diagram showing a recording example of a conventional rejected hypothesis.

Claims (2)

[Claims] At each time, a new hypothesis is generated, an acoustic score indicating the closeness of the input speech in the word of the hypothesis to the hypothesis and the corresponding acoustic model is calculated, and the word at the word end of the hypothesis is calculated. Using a language model that defines the connection relationship between the language and the language score corresponding to the probability that the hypothesis exists, a hypothesis indicating the likelihood of the hypothesis regarding the input speech content allowed by the grammar from the acoustic score and the language score Of the hypothesis, reject the hypothesis with a low score of the hypothesis, record the rejected hypothesis in the buffer memory, and store the word history representing the word sequence from the start time of the input speech of the hypothesis to the current time in the buffer. The hypothesis is re-evaluated by correcting it based on the hypothesis rejected at the past time recorded in the memory, and the hypothesis with the highest score when all voices are input In a speech recognition method that outputs the rejected hypothesis, a hypothesis rejection time indicating the time when the hypothesis was rejected, a final word indicating the last word in the word history,
Alternatively, the data is classified into phoneme histories representing a predetermined number of phoneme strings up to the end of the last word, recorded in the buffer memory, and the same hypothesis rejection time as the start time of the last word of the hypothesis to be reevaluated, and reevaluation Only the hypothesis in the buffer memory classified as the last word or phoneme history, which is the same as the preceding word or the phoneme history up to the last word representing the word before the last word of the hypothesis to be read, is reevaluated. A speech recognition method, which is used for correcting a hypothesis history. 2. A recording medium on which a program for causing a computer to execute the method according to claim 1 is recorded.