JP3819896B2 - Speech recognition method, apparatus for implementing this method, program, and recording medium - Google Patents

Description

  The present invention relates to a speech recognition method, an apparatus for implementing the method, a program, and a recording medium, and more particularly, speech recognition that outputs a reliability of a recognition result accompanying a recognition result obtained by speech recognition processing on an input speech signal. The present invention relates to a method, an apparatus for implementing the method, a program, and a recording medium.

  In speech recognition, in general, the likelihood between a speech feature parameter sequence obtained by analyzing input speech and an acoustic model that models speech is calculated, and a vocabulary or word connection that is a set of words to be recognized. The candidate with the highest likelihood is output as a recognition result within the linguistic constraint of a language model representing ease of operation and rules. However, if the utterance of the input speech is ambiguous, noise is superimposed on the speech, or an acoustic signal other than speech is input, an erroneous recognition result may be obtained even for the highest likelihood candidate. The possibility of output increases. Furthermore, when the input speech is an unregistered word, a correct recognition result cannot be output.

For the above problems, by giving reliability to the speech recognition result, if the reliability is high, the speech recognition result is accepted, and if the reliability is low, the speech recognition result is rejected, or the speaker As a result, it is possible to suppress an unexpected operation of a speaker due to a voice recognition error in various devices that employ voice recognition.
Here, as a method of giving reliability to the speech recognition result, a method is disclosed in which the probability obtained by using the difference between the probabilities obtained from the target category and the non-target category (conflict model) is used as the reliability of the recognition result. (See Patent Document 1). And the method of calculating | requiring the reliability of a recognition result based on the posterior probability of the word in the N best candidate which is a word graph or a word to a top N rank, or a word string is disclosed (refer nonpatent literature 1).
JP-A-11-85188 Frank Wessel, Ralf Schluter, KIaus Macherey and Hermann Ney, “Confidence Measures for Large Vocabulary Continuous Speech Recognition”, IEEE Transactions on Speech and Audio Processing, Vol. 9, No. 3, March 2001.

However, since the method described in Patent Document 1 needs to perform recognition processing for each of the target category model and the non-target category model, it requires a larger amount of calculation than the normal recognition processing. There was a problem.
Further, in the method described in Non-Patent Document 1, it is necessary to obtain a word graph (word lattice) that is an intermediate expression of a recognition result when obtaining a posterior probability of a word. In the 2-pass search in which the detailed search is performed in the second pass, the word graph that is the output of the first pass must be used, or the word graph must be generated in the second pass. However, the word graph that is the output of the first pass is a result of a rough search, so the accuracy is low, and therefore the accuracy of the reliability obtained therefrom is also poor. Further, when the word graph is generated in the second pass, the third pass has to be executed in order to output the final N best candidate, and there is a problem that a necessary processing amount increases.

The method for obtaining the posterior probabilities of words using the N best candidates described in Non-Patent Document 1 is necessary because the obtained N best candidates need to be aligned using dynamic programming. There was a problem that the amount of calculation increased.
The present invention obtains a score difference per acoustic analysis frame between a first word and a word having the highest score among words different from the first word in the N best candidates obtained as a speech recognition result. , A speech recognition method that adopts a configuration that normalizes the score difference in successive frame sections, and can obtain a highly reliable reliability with respect to a speech recognition result with the same amount of calculation as a normal recognition process, and this method The present invention provides a device, a program, and a recording medium for performing the above.

Claim 1: An input speech signal is converted into a digital signal, an acoustic feature parameter is extracted from the digital signal, and the extracted acoustic feature parameter is linguistically subject to given linguistic constraints. Calculate the score based on the probability output by the probability model that expresses the characteristics of each category of the unit, and show the highest score among the N best recognition result candidates that are word strings up to the top N ranking with the highest score In the speech recognition method for outputting the recognition result candidate and the reliability of the word in the candidate , the first recognition result candidate among the second and lower recognition result candidates for each frame constituting the word in the first recognition result candidate. If there is a different word from the recognition result candidate of the rank, the difference between the highest score among the recognition result candidates of the second rank and lower and the score of the first recognition result candidate is obtained as a score difference, If there is no such thing, a step of obtaining a preset value as a score difference, a sum of score differences of all frames constituting the word, and normalizing this by the number of frames constituting the word are the first. to constitute a speech recognition method for chromatic and outputting a confidence score for the word in place of the recognition result candidates.

According to the second aspect of the present invention, the speech recognition method is configured to output the total reliability score of the words constituting the first recognition result candidate as the reliability score for the word string of the first recognition result candidate .

Here, claim 3 is : an acoustic analysis unit 120 that converts an input audio signal into a digital signal and extracts an acoustic feature parameter series 130 from the digital signal; an acoustic model storage unit 140 that stores an acoustic model; A dictionary / language model storage unit 150 that stores vocabulary and a language model, and an acoustic model and dictionary / language model are input from the acoustic model storage unit 140 and the dictionary / language model storage unit 150, and an acoustic feature parameter series of the input speech is input. The search unit 160 searches for the recognition result candidate and obtains the N best recognition result candidate that is a word string up to the top N ranks together with the score, and the first recognition result candidate based on the recognition result candidate and the score. A reliability calculation unit 190 that calculates a reliability score for the word of the first word, and the reliability calculation unit 190 is the first recognition result candidate. For each frame constituting the word, if there is a word that is different from the first recognition result candidate in the second and lower recognition result candidates, the highest score among the second and lower recognition result candidates The difference from the score of the first recognition result candidate is obtained as a score difference, and when there is no different word, a preset value is obtained as a score difference, and the total score difference of all frames constituting the word is obtained. The speech recognition apparatus is configured to output a result obtained by normalizing the number of frames constituting the word as a reliability score for the word in the first recognition result candidate .

According to a fourth aspect of the present invention, there is provided a speech recognition apparatus that outputs the sum of the reliability scores of the words constituting the first recognition result candidate as a reliability score for the word string of the first recognition result candidate.

Moreover, they claim 5 converts the audio signal inputted to digital signal, extracting an acoustic feature parameter sequence from the digital signal, an acoustic model and a dictionary and language model from the acoustic model storage unit and the dictionary and language model storage unit Input, search for recognition result candidates for the acoustic feature parameter series of the input speech, obtain N best recognition result candidates that are word strings up to the top N, together with the score, and are among the first recognition result candidates For each frame constituting the word, if there is a word that is different from the first recognition result candidate in the second and lower recognition result candidates, the highest score among the second and lower recognition result candidates The difference from the score of the first recognition result candidate is obtained as a score difference, and when there are no different words, a preset value is obtained as a score difference, and all frames constituting the word Seeking total score difference, and outputs the normalized with the number of frames constituting the word as confidence score for the word in the first of the recognition result candidates, and executes the command to the CPU speech A recognition program was constructed.
Claim 6: A command for outputting the sum of the reliability scores of the words constituting the first recognition result candidate as the reliability score for the word string of the first recognition result candidate is executed to the CPU. A speech recognition program was constructed.
Furthermore, the recording medium which memorize | stored the voice recognition program as described in Claim 7: Claim 5 and 6 was comprised.

  According to the present invention, the calculation for determining the reliability is the calculation of the score difference per frame in the N best candidates and the normalization thereof, which is executed with a very small amount of calculation compared with the speech recognition processing. Can do. Further, as shown in the experimental results of the embodiment, it is possible to obtain a highly reliable reliability as compared with the conventional method.

The best mode for carrying out the invention will be described with reference to the drawings.
FIG. 1 is a diagram for explaining an embodiment of a speech recognition apparatus. The input speech 110 is converted into an acoustic feature parameter series 130 by the acoustic analysis unit 120. The acoustic feature parameter series is an LPC cepstrum, MFCC, or other parameter series obtained by analyzing input speech in units called frames of several tens of msec. The search unit 160 uses the acoustic model storage unit 140 and the dictionary / language model storage unit 150 to search for recognition result candidates for the input speech for the acoustic feature parameter series. As a result of the search, N best candidates up to the top N are output as a speech recognition result 170 together with the score 180. The reliability calculation unit 190 calculates and outputs a reliability score 200 for the speech recognition result 170 based on the speech recognition result 170 and the score 180.

  With reference to the flowchart of FIG. 2, the process of calculating the reliability score for the speech recognition result executed in the reliability calculation unit 190 will be described. When the word sequence and the score of the speech recognition result for the input speech are obtained by the speech recognition processing, first, the word number w and the frame number t are initialized to 1 in step (S01). In step (S02), the word reliability score accumulation variable A is initialized to zero. In step (S03), if the word w is different from the word w in the frame t and the answer is Yes in the N best candidates, the score of the word w in the frame t and the opposite candidate word in the frame t in step (S04). A score difference D (t) from the score is obtained. If the conflict candidate is No in the N best candidates, a fixed value designated in advance is substituted for D (t) in step (S05). This fixed value is set so as to be larger than the score difference D (t) when there is an opposing candidate. In step (S06), the score difference D (t) obtained in the frames in the word section is accumulated. In step (S07), if the frame t is No which is not the last frame of the word w, the frame t is advanced by one in step (S08), and the process returns to step (S02). If the frame t is Yes, which is the final frame of the word w, in step (S07), as a normalization process, the confidence in the section of the word w is obtained by dividing the cumulative score A by the number of frames of the word w in step (S09). Calculate the average of degree scores. If the word w is No in step (S10) and is not the final word in the recognition result word string, the word w is advanced by one in step (S11) and the process is repeated. If the word w is Yes in step (S10), which is the final word of the recognition result word string, the reliability score C (w) for each word of the recognition result and the reliability score C ( W) = ΣC (w) is output.

In FIG. 2, step (S03) constitutes the conflict candidate detection unit 400 that detects whether or not a word different from the word w exists in the N best candidates in each frame t of the word w. Steps (S04) and (S05) constitute a score difference calculation unit 410 that calculates a score difference D (t) between the score of the word w in the frame t and the score of the opponent candidate word in the frame t. Step (S06) constitutes a score difference accumulation calculation unit 420 that accumulates the score difference D (t) of each frame t. Step (S09) constitutes a reliability score calculation unit 430 that obtains an average of reliability scores in the section of word w by dividing cumulative score difference A by the number of frames of word w as normalization processing. The reliability calculation unit 190 includes the conflict candidate detection unit 400, the score difference calculation unit 410, the score difference accumulation calculation unit 420, and the reliability score calculation unit 430 as main components.

FIG. 3 is a block diagram for explaining the operation of the reliability calculation unit.
With reference to FIG. 3, the process of calculating the reliability score in the reliability calculation unit 190 of FIG. 1 will be described. 210 indicates the first candidate of the speech recognition result, 220 indicates the second candidate of the speech recognition result, and 230 displays the third candidate of the speech recognition result. The first candidate 210 is composed of word A, word B, and word C, the second candidate is composed of word A, word D, and word C, and the third candidate is composed of word E, word F, and word C. Here, one square represents a frame 240 of acoustic feature parameters.

First, the reliability of the first candidate word A is obtained. In each frame of the word A, the score difference 250 between the frame having the highest score and the frame having the highest score and the frame of the word A among the second and lower opposing candidates is calculated. The score difference 250 calculated in each frame is normalized, that is, a value obtained by averaging is used as the reliability of the word A. Similarly, the reliability is also obtained for the words B and C. When there is no conflict candidate with different words like the word C, it is considered that the reliability of the word C is high, and a sufficiently large predefined fixed value is given as a score difference.
In addition, for non-category sections consisting of poses and noises such as noise that are not included in the category to be output as words or other recognition results, the reliability can be obtained using the score difference in the same way. Thus, the reliability of the recognition result can be obtained. In the case of isolated word recognition, compared to the case of continuous word recognition, the non-category section occupies the entire speech section to be recognized, and the score is based on whether the section is recognized as a part of the word or as a non-category. It is desirable to calculate the reliability including the non-category section because of the large difference.

In the conventional calculation method of the reliability using the conflict model, it is necessary to perform a recognition process using the conflict model in order to obtain the reliability. However, according to the method of the present invention, the N best obtained as a speech recognition result. The reliability of the recognition result can be obtained only by calculating a candidate and a simple score difference between the scores and an addition average.
In addition, the conventional calculation method of reliability using posterior probabilities requires using a low-precision word graph or preparing a high-precision word graph for calculation of reliability. Accordingly, the reliability of the speech recognition result can be obtained with a small amount of processing by using the N best candidate obtained as a result of the normal two-pass recognition. The traditional method of finding the posterior probability from the N-best candidates is at least to align the resulting word string,
(Number of words (sentence length)) The calculation amount proportional to the number of ² × N best candidates, that is, the calculation amount proportional to the square of the sentence length is required. ,
The amount of calculation is proportional to the number of (frame number (sentence length)) × N best candidates, that is, the amount of calculation proportional to the length of the sentence. Become.

  The voice recognition method described above can be implemented by causing a CPU to execute a command via a program. For example, referring to FIG. 4, 330 is a CPU. An acoustic model storage unit 140 stores an acoustic model. A dictionary / language model storage unit 150 stores vocabulary and language models. Reference numeral 320 denotes a voice recognition program memory, which is installed from a CD / ROM, a hard disk or other storage medium, or installed via a communication line. A storage unit 340 temporarily stores data during acoustic analysis, search, and reliability score calculation. A speech recognition apparatus embodying the present invention is configured by interconnecting the above-described constituent members via a bus 310. The speech recognition apparatus obtains a speech recognition result 170 and a score 180 for the input speech signal 110, and then obtains a reliability score 200 by the method described above.

表１を参照するに、実施例による信頼度は、従来例より少ない処理時間で従来例の半分以下の等誤り率を達成していることを認識することができる。 The effect of this invention was evaluated in an isolated word recognition experiment. In the company name recognition task with a vocabulary size of 5114 words, the reliability using the conventional conflict model was compared with the reliability according to this example. First, the reliability for the recognition result is obtained by each method, and when the reliability is larger than the threshold value, it is accepted, and when it is small, the acceptance rate is obtained, that is, the probability of accepting the wrong recognition result is obtained. At the same time, the false rejection rate, that is, the probability of rejecting the correct recognition result is obtained. There is a trade-off relationship between the false acceptance rate and the false rejection rate. When the threshold value is increased, the false acceptance rate decreases and the false rejection rate increases, and when the threshold value is reduced, the opposite is true. The evaluation was performed based on an equal error rate, which is a probability when the false acceptance rate and the false rejection rate are equal when the threshold value is variously changed. The accuracy rate of the first candidate of the speech recognition result was 85.5%. Table 1 shows the reliability evaluation results. Further, the processing time ratio of the method according to the present invention when the processing time of the conventional example is set to 1 is shown.

Referring to Table 1, it can be recognized that the reliability according to the embodiment achieves an equal error rate of half or less that of the conventional example in a processing time shorter than that of the conventional example.

The effect of this invention was evaluated in continuous word recognition. We evaluated the keyword part of speech utterance for a spoken dialogue device. Table 2 shows the evaluation results of the airline ticket reservation task (keyword accuracy: 78.7%) with a vocabulary size of 1124 words. Table 3 shows the evaluation result of the store search task (keyword accuracy: 86.8%) with a vocabulary size of 421 words.

From Table 2 and Table 3, it can be recognized that the reliability according to the example achieves an equal error rate lower than that of the conventional example in continuous word recognition with a processing time shorter than that of the conventional example.
By using the speech recognition method according to the present invention, in the speech dialogue apparatus, not all the recognition results are confirmed with respect to the speaker, but only the recognition results with low reliability are confirmed with the speaker, In the wake-up device, it is possible to develop an efficient speech recognition application device by not displaying the recognition result of the portion with low reliability.

The block diagram explaining an Example. The flowchart explaining the flow of a process of the reliability calculation part of an Example. The block diagram which shows the structure of the reliability calculation part of an Example. The block diagram explaining the Example which employ | adopts CPU.

Explanation of symbols

DESCRIPTION OF SYMBOLS 110 Input speech signal 120 Acoustic analysis part 130 Acoustic feature parameter series 140 Acoustic model storage part 150 Dictionary and language model storage part 160 Search part 170 Speech recognition result 180 Score 190 Reliability calculation part 200 Reliability score 210 First candidate 220 2nd candidate 230 3rd candidate 240 Frame 250 Score difference 310 Bus 320 Speech recognition program memory 330 CPU
340 Storage unit 400 Opposition candidate detection unit 410 Score difference calculation unit 420 Score difference accumulation calculation unit 430 Reliability score calculation unit

Claims (7)

The input speech signal is converted into a digital signal, and acoustic feature parameters are extracted from the digital signal. Each category of linguistic units is applied to the extracted acoustic feature parameters under given linguistic constraints. Together with the first recognition result candidate indicating the highest score among the N best recognition result candidates that are word strings up to the top N ranks with the highest score. In the speech recognition method for outputting the reliability of the word in the candidate ,
For each frame composing a word in the first recognition result candidate, if there is a difference between the second recognition result candidate and the first recognition result candidate, the second or lower recognition result candidate Obtaining a difference between the highest score in the recognition result candidates and the score of the first recognition result candidate as a score difference, and when there is no different word, obtaining a preset value as a score difference;
Obtaining a total score difference of all the frames constituting the word, and normalizing it by the number of frames constituting the word, and outputting as a reliability score for the word in the first recognition result candidate; speech recognition method which is characterized in that chromatic. The input speech signal is converted into a digital signal, and acoustic feature parameters are extracted from the digital signal. Each category of linguistic units is applied to the extracted acoustic feature parameters under given linguistic constraints. Together with the first recognition result candidate indicating the highest score among the N best recognition result candidates that are word strings up to the top N ranks with the highest score. in the speech recognition method for outputting a reliability of that,
For each frame constituting each word in the first recognition result candidate, if there is a word that is different from the first recognition result candidate in the second or lower recognition result candidate, the second or lower recognition result candidate Obtaining a difference between the highest score in the recognition result candidates and the score of the first recognition result candidate as a score difference, and when there is no different word, obtaining a preset value as a score difference;
Obtaining a total score difference of all the frames constituting the word and normalizing it by the number of frames constituting the word as a confidence score for the word;
A speech recognition method, comprising: outputting a total reliability score of each word constituting a first recognition result candidate word string as a reliability score for the first recognition result candidate word string . An acoustic analysis unit 120 that converts an input speech signal into a digital signal and extracts an acoustic feature parameter series 130 from the digital signal;
An acoustic model storage unit 140 in which an acoustic model is stored;
A dictionary and language model storage unit 150 to store the vocabulary and language model,
The acoustic model and the dictionary / language model are input from the acoustic model storage unit 140 and the dictionary / language model storage unit 150 , and the recognition result candidates are searched for the acoustic feature parameter series of the input speech . A search unit 160 that obtains N best recognition result candidates that are word strings together with scores;
A reliability calculation unit 190 that calculates a reliability score for the word in the first recognition result candidate based on the recognition result candidate and the score,
The reliability calculation unit 190
For each frame constituting the word in the first recognition result candidate, if there is a difference in word from the first recognition result candidate in the second or lower recognition result candidate, the second or lower recognition result Find the difference between the highest score among the result candidates and the score of the first recognition result candidate as a score difference, and if there are no different words, obtain a preset value as the score difference,
The total score difference of all the frames that make up a word is calculated, and this is normalized by the number of frames that make up that word, and output as a confidence score for the word in the first recognition result candidate. A speech recognition apparatus characterized by being. An acoustic analysis unit 120 that converts an input speech signal into a digital signal and extracts an acoustic feature parameter series 130 from the digital signal;
An acoustic model storage unit 140 in which an acoustic model is stored;
A dictionary and language model storage unit 150 to store the vocabulary and language model,
The acoustic model and the dictionary / language model are input from the acoustic model storage unit 140 and the dictionary / language model storage unit 150 , and the recognition result candidates are searched for the acoustic feature parameter series of the input speech . A search unit 160 that obtains N best recognition result candidates that are word strings together with scores;
A reliability calculation unit 190 that calculates a reliability score for the first recognition result candidate based on the recognition result candidate and the score,
The reliability calculation unit 190
For each frame constituting each word in the first recognition result candidate, if there is a word that is different from the first recognition result candidate in the second or lower recognition result candidate, the second or lower recognition result candidate Find the difference between the highest score in the recognition result candidates and the score of the first recognition result candidate as a score difference, and if there are no different words, obtain a preset value as the score difference,
Find the total score difference of all the frames that make up a word, normalize this with the number of frames that make up that word, and make it the confidence score for the word in the first recognition result candidate,
A speech recognition apparatus, characterized in that a sum of reliability scores of words constituting a word string of a first recognition result candidate is output as a reliability score for a word string of a first recognition result candidate. Convert the input audio signal into a digital signal,
An acoustic feature parameter series is extracted from the digital signal,
Input the acoustic model and dictionary / language model from the acoustic model storage unit and dictionary / language model storage unit, search the recognition feature candidates for the acoustic feature parameter series of the input speech, and the word string up to the top N N best recognition result candidates that are
For each frame constituting the word in the first recognition result candidate, if there is a difference in word from the first recognition result candidate in the second or lower recognition result candidate, the second or lower recognition result Find the difference between the highest score among the result candidates and the score of the first recognition result candidate as a score difference, and if there are no different words, obtain a preset value as the score difference,
Seeking total score difference of all the frames constituting the word, and outputs the normalized with the number of frames constituting the word as confidence score for the word in the first of the recognition result candidates, the command speech recognition program and executes against the CPU. Convert the input audio signal into a digital signal,
An acoustic feature parameter series is extracted from the digital signal,
Input the acoustic model and dictionary / language model from the acoustic model storage unit and dictionary / language model storage unit, search the recognition feature candidates for the acoustic feature parameter series of the input speech, and the word string up to the top N N best recognition result candidates that are
For each frame constituting each word in the first recognition result candidate, if there is a word that is different from the first recognition result candidate in the second or lower recognition result candidate, the second or lower recognition result candidate Find the difference between the highest score in the recognition result candidates and the score of the first recognition result candidate as a score difference, and if there are no different words, obtain a preset value as the score difference,
Find the total score difference of all the frames that make up a word, normalize this with the number of frames that make up that word, and find the confidence score for the word in the first recognition result candidate,
A command is output to the CPU for outputting a total reliability score of words constituting the word string of the first recognition result candidate as a reliability score for the word string of the first recognition result candidate. Voice recognition program. A recording medium storing the voice recognition program according to claim 5 .

Images