JP2005250071A - Method and device for speech recognition, speech recognition program, and storage medium with speech recognition program stored therein - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To improve efficiency of word addition during update by performing update processing for a recognition dictionary at high speed. <P>SOLUTION: Speech recognition processing is carried out by using the prepared recognition dictionary, a related vocabulary is estimated by using the recognition result, and the estimated vocabulary is added to the recognition dictionary to generate an extended recognition dictionary, which is used to perform speech recognition. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a speech recognition method and apparatus, a speech recognition program, and a storage medium storing the speech recognition program, and in particular, based on a recognition result obtained by speech recognition processing on an input speech signal, a word related to the input speech. The present invention relates to a speech recognition method and apparatus, a speech recognition program, and a storage medium storing the speech recognition program for estimating and adding those words to a recognition dictionary and performing recognition processing again to obtain a final recognition result.

  In speech recognition, the likelihood between the acoustic feature parameter sequence obtained by analyzing the input speech and the acoustic model that models the speech is calculated, and a recognition dictionary, which is a set of words to be recognized, The candidate with the highest likelihood is output as a recognition result within the language constraint of a language model representing ease and rules. Although the speech recognition accuracy can be improved by improving the accuracy of the acoustic model and the language model, a correct recognition result cannot be output for unregistered words that are not included in the recognition dictionary.

  On the other hand, if the number of words included in the recognition dictionary prepared in advance to reduce the number of unregistered words is increased, many resources such as time required for recognition processing and storage capacity must be spent. As long as the number of words included in the recognition dictionary is limited, the problem of unregistered words cannot be solved completely.

  For the above problems, by estimating whether or not an unregistered word is included in the input speech, it becomes possible to inquire the user again about the unregistered word portion in the spoken dialogue system. Further, by recognizing the unregistered word part by subwords such as phonemes and syllables, the phoneme sequence of the unregistered word part can be estimated. Based on the result of recognizing the input speech, a related document is searched, and the acquired document is reconstructed into a language model, whereby unregistered words for the input speech can be reduced.

  In the first method of estimating whether or not unregistered words are included in the input speech, the input speech is recognized based on the probabilities obtained by using the difference in probabilities obtained from the target category and the non-target category (conflict model). A method for determining whether or not the word is included in a dictionary is disclosed (see Patent Document 1).

  In addition, as a language model generation method for estimating the interval, class, and reading of unregistered words, including unregistered words based on subword units based on word class N-gram model and subword unit N-gram model A method for generating a statistical language model is disclosed (for example, see Patent Document 2 and Non-Patent Document 1).

In addition, a method for obtaining a recognition result for the utterance of a word not included in the recognition dictionary is disclosed (for example, see Non-Patent Document 2).
Japanese Patent Laid-Open No. 11-85188 JP 2001-236089 A Yoshifumi Hirose, Katsunobu Ito, Kiyohiro Shikano, Satoshi Nakamura, “Long-vocabulary continuous speech recognition using a high coverage language model based on“ reading ”information”, Proceedings of the Acoustical Society of Japan, 2-1-8, pp. 69-70,1999-9 Thomas Kemp and Alex Waibel, “Reducing the OOV Rate in Broadcast News Speech Recognition”, Proceedings of the ICSLP 98, pp. 1839-1842, December, 1998

  However, in the above-described method for estimating whether or not an unregistered word is included in the input speech, it is determined whether or not the input speech is a utterance of a word included in a recognition dictionary prepared in advance. In the case of utterance of a word that is not included in the recognition dictionary prepared in advance, there is a problem that the utterance content cannot be output as a recognition result.

  In addition, in the conventional method for generating a language model for identifying an unregistered word section, class, and reading, a subword sequence of a utterance portion of a word that is not included in a prepared recognition dictionary, that is, reading or kana Although the corresponding result can be output, the result as a notation including kanji cannot be output.

  Further, in the conventional method of obtaining a recognition result for a utterance of a word that is not included in a recognition dictionary prepared in advance, a newly acquired document is added to learning data to reconstruct a recognition dictionary and a language model. Therefore, there is a problem that many processes are required for updating the model and time is required. In addition, since all the words appearing in the acquired related document are added to the recognition dictionary, many words that are not necessary for improving the coverage ratio are added, and there is a problem that the efficiency of adding words is poor.

  The present invention has been made in view of the above points. In a speech recognition method in which a recognition document prepared in advance is updated based on a recognition result, and recognition processing is executed again, update processing of the recognition dictionary is performed. An object of the present invention is to provide a speech recognition method and apparatus, a speech recognition program, and a storage medium storing the speech recognition program, which can be performed at high speed with a small amount of processing, and are efficient in adding words at the time of update.

  FIG. 1 is a diagram for explaining the principle of the present invention.

The present invention extracts an acoustic feature parameter from an input digital signal, and for the acoustic feature parameter, a language score given by a language model and a recognition dictionary prepared in advance, an acoustic score given by the acoustic model, In a speech recognition method for outputting a language unit sequence having the highest score calculated based on the language score and the acoustic score as a speech recognition result,
A first speech recognition step (step 1) for performing speech recognition processing for obtaining a provisional speech recognition result using the recognition dictionary prepared in advance when speech is input;
A vocabulary acquisition step (step 2) of acquiring a related vocabulary for the content from a provisional speech recognition result from a database storing information on the relationship between words;
A vocabulary expansion step (step 3) for creating an extended dictionary in which the acquired related vocabulary is added to a recognition dictionary prepared in advance;
A second speech recognition step (step 4) is performed in which the speech recognition process is performed again using the extended dictionary.

In the vocabulary acquisition step, the present invention
When acquiring the related vocabulary from the provisional speech recognition result, a relevance calculation step of calculating the relevance between the speech recognition result and the related vocabulary based on the reliability of the provisional speech recognition result is included.

In the second speech recognition step, the present invention provides:
When performing speech recognition processing using an extended dictionary, linguistic restrictions corresponding to the class to which each word belongs are given to each word of the added related vocabulary.

  FIG. 2 is a principle configuration diagram of the present invention.

The present invention converts an input speech signal into a digital signal and extracts an acoustic feature parameter series from the digital signal, an acoustic model storage means for storing an acoustic model, and a language model. Language model storage means, dictionary storage means for storing a prepared recognition dictionary, acoustic model storage means, language model storage means, dictionary storage means, acoustic models and language models are prepared in advance, respectively. A recognition dictionary is input, and for the acoustic feature parameter series of the input speech, a language score given by the language model and the recognition dictionary prepared in advance, and an acoustic score given by the acoustic model are calculated, Search means for outputting a language unit sequence having the highest score calculated based on the language score and the acoustic score as a speech recognition result; To a voice recognition device,
From the provisional first speech recognition result 180 obtained using the recognition dictionary prepared in advance in the dictionary storage means for the input speech, the related vocabulary for the content is stored, and the relationship information between the words is stored. Vocabulary expansion means 190 for creating an extended dictionary 200 obtained from the vocabulary database 380 and adding the acquired related vocabulary to the dictionary storage means;
Re-search means 210 for outputting a speech recognition result obtained using the extended dictionary 200.

The vocabulary expansion means 190 of the speech recognition apparatus of the present invention
The degree of association for calculating the degree of association between the speech recognition result and the related vocabulary based on the reliability of the provisional first speech recognition result 180 when acquiring the related vocabulary from the provisional first speech recognition result Includes calculation means.

The re-search means 210 of the speech recognition apparatus of the present invention
When speech recognition processing is performed using the extended dictionary 200, a means for giving a linguistic restriction corresponding to a class to which each word belongs is added to each word of the added related vocabulary.

The present invention extracts an acoustic feature parameter from an input digital signal, and for the acoustic feature parameter, a language score given by a language model and a recognition dictionary prepared in advance, an acoustic score given by the acoustic model, A speech recognition program that outputs a speech unit as a speech recognition result with a language unit sequence having the highest score calculated based on the language score and the acoustic score,
On the computer,
A first speech recognition step for performing speech recognition processing for obtaining a provisional speech recognition result using a recognition dictionary prepared in advance when speech is input;
A vocabulary acquisition step of acquiring a related vocabulary for the content from a temporary speech recognition result from a database storing information on the relationship between words;
A vocabulary extension step of creating an extended dictionary in which the acquired related vocabulary is added to a recognition dictionary prepared in advance;
A second speech recognition step of performing speech recognition processing again using the extended dictionary.

In the vocabulary acquisition step of the speech recognition program of the present invention,
When acquiring the related vocabulary from the provisional speech recognition result, a relevance calculation step of calculating the relevance between the speech recognition result and the related vocabulary based on the reliability of the provisional speech recognition result is executed.

In the second speech recognition step of the speech recognition program of the present invention,
When performing speech recognition processing using an extended dictionary, the method includes a step of giving a linguistic constraint corresponding to a class to which each word belongs to each word of the added related vocabulary.

The present invention extracts an acoustic feature parameter from an input digital signal, and for the acoustic feature parameter, a language score given by a language model and a recognition dictionary prepared in advance, an acoustic score given by the acoustic model, A storage medium storing a speech recognition program that outputs a speech unit recognition result as a language unit sequence having the highest score calculated based on the language score and the acoustic score,
On the computer,
A first speech recognition step for performing speech recognition processing for obtaining a provisional speech recognition result using a recognition dictionary prepared in advance when speech is input;
A vocabulary acquisition step of acquiring a related vocabulary for the content from a temporary speech recognition result from a database storing information on the relationship between words;
A vocabulary extension step of creating an extended dictionary in which the acquired related vocabulary is added to a recognition dictionary prepared in advance;
A storage medium storing a program for executing a second speech recognition step for performing speech recognition processing again using an extended dictionary.

In the vocabulary acquisition step of the program stored in the storage medium of the present invention,
When acquiring the related vocabulary from the provisional speech recognition result, a relevance calculation step of calculating the relevance between the speech recognition result and the related vocabulary based on the reliability of the provisional speech recognition result is executed.

In the second speech recognition step of the program stored in the storage medium of the present invention,
When performing speech recognition processing using an extended dictionary, the method includes a step of giving a linguistic constraint corresponding to a class to which each word belongs to each word of the added related vocabulary.

  As described above, in the present invention, the vocabulary related to the input speech is required based on the speech recognition result obtained for the input speech, using the database (vocabulary database) in which the relationship between words is accumulated. It is possible to estimate efficiently so that no word is included.

  In addition, the recognition dictionary can be updated at high speed with a small amount of processing because only the related vocabulary is added to the recognition dictionary prepared in advance. By executing the speech recognition process again using the updated extended recognition dictionary, it is possible to obtain a highly accurate speech recognition result using the extended recognition dictionary with few unregistered words for the input speech.

  As described above, according to the present invention, the related vocabulary can be efficiently acquired by acquiring the related vocabulary instead of the related document based on the recognition result of the input speech. In addition, the recognition dictionary can be updated at high speed with a small amount of processing because only the related vocabulary is added to the recognition dictionary prepared in advance. Therefore, it is possible to perform the speech recognition process of updating the recognition dictionary based on the recognition result for the input speech and performing the recognition again using the extended dictionary with high speed and high accuracy.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 3 shows the configuration of the speech recognition apparatus in one embodiment of the present invention.

  The speech recognition apparatus shown in the figure includes an acoustic analysis unit 120, a search unit 170, a re-search unit 210, a vocabulary expansion unit 190, an acoustic model 140, a language model 150, a recognition dictionary 160, and an extended recognition dictionary 200.

  FIG. 4 is a flowchart of the operation of the speech recognition apparatus in one embodiment of the present invention.

  The acoustic analysis unit 120 converts the input speech 110 into the acoustic feature parameter series 130 (step 101). Here, the acoustic feature parameter series 130 is a parameter series such as an LPC cepstrum or MFCC obtained by analyzing the input speech 110 in units called frames of several tens of msec.

  The search unit 170 searches for a recognition result candidate for the input speech 110 using the acoustic model 140, the language model 150, and the recognition dictionary 160 prepared in advance, and obtains a provisional speech recognition result (first time) 180. Obtain (step 102).

  The vocabulary expansion unit 190 estimates the vocabulary related to the input speech based on the provisional speech recognition result (first time) 180 and adds the estimated vocabulary to the prepared recognition recognition dictionary 160. 200 is generated (step 103). Details will be described in detail with reference to FIGS.

  The re-search unit 210 searches for a recognition result candidate again using the acoustic model 140, the language model 150, and the extended recognition dictionary 200, and outputs a final speech recognition result 220 (step 104).

  Next, a process of creating the extended recognition dictionary 200 in the vocabulary extension unit 190 will be described.

  FIG. 5 shows the configuration of the vocabulary extension unit in one embodiment of the present invention, and FIG. 6 shows a flowchart of the operation of the vocabulary extension unit in one embodiment of the present invention.

  The vocabulary expansion unit 190 includes a related vocabulary acquisition unit 310 and an extended recognition dictionary generation unit 410.

  When the word sequence of the provisional speech recognition result (first time) 180 is input from the search unit 170, the related vocabulary acquisition unit 310 outputs the related vocabulary 320 for the input word sequence (document). 370, a vocabulary database 380 in which information on the degree of association between words is stored, and a related vocabulary output unit 400 that outputs a related vocabulary 320 based on a predefined relevance threshold or the number of words to be acquired; Have

  Note that the information on the relationship between words in the vocabulary database 380 can be obtained using, for example, a thesaurus disclosed in Japanese Patent Laid-Open No. 8-147324, and is disclosed in Japanese Patent Laid-Open No. 2000-137718. It is also possible to obtain by using co-occurrence information of words in such a large amount of text data.

  The vocabulary relevance calculation unit 370 of the related vocabulary acquisition unit 310 refers to the vocabulary database 380 and relates to each word of the word sequence of the temporary speech recognition result (first time) 180 input from the search unit 170. A value (degree of association) representing the high degree of relevance of the word is obtained and output together with the highly relevant word (step 201).

  The related vocabulary output unit 400 outputs the related vocabulary for the word related to the word series (document) output from the vocabulary relevance calculation unit 370 based on a predefined relevance threshold or the number of words to be acquired. (Step 202).

  The extended recognition dictionary generation unit 410 refers to the recognition dictionary 160 prepared in advance among the words in the related vocabulary 320, and extracts words that are not included in the recognition dictionary 160 (step 203). The expanded vocabulary dictionary 200 added to 160 is created (step 205).

  The operations shown in FIGS. 4 and 6 may be performed by executing a program using a computer. For example, as shown in FIG. 7, each unit is connected to a bus 510, and a voice recognition program is installed in the voice recognition program memory 520 from a CD-ROM, a hard disk, or the like or via a communication line. By executing this speech recognition program, a speech recognition result (first time) 180 for the input speech 110 is obtained using the acoustic model 140, the language model 150, and the recognition dictionary 160 prepared in advance, and then FIG. The extended recognition dictionary 200 is obtained by the method shown. Then, the final speech recognition result 220 is output using the acoustic model 140, the language model 150, and the extended recognition dictionary 200. The storage unit 540 is used for temporarily storing data during acoustic analysis, search, or creation of an extended recognition dictionary.

  The present invention is not limited to the above-described embodiment, and various modifications and applications can be made within the scope of the claims.

  Evaluation by experiment of the speech recognition processing in the above configuration is shown below.

  In the experiment, broadcast news sound was evaluated.

  For the evaluation, a news program recorded for about 20 minutes was used. Nine items of news are included, and the total number of words is about 2700 words. An extended recognition dictionary was created by obtaining a provisional recognition result for each news item, and recognition was performed again using the created extended recognition dictionary.

  As the language model, a word trigram model was constructed using learning data of about 600,000 sentences and 15 million words such as transcriptions of news programs. The recognition dictionary prepared in advance consists of words that appear 10 times or more in the learning data, and the number of vocabularies is about 25000 words.

  For the acoustic model, a state sharing triphone HMM of 3 states and 8 mixed trained using about 300 hours of reading speech and news speech was used.

  The vocabulary database for determining the relationship between words was learned based on word co-occurrence in each sentence of newspaper articles for one year. About 150,000 words are registered in the vocabulary database, and each word has a 100-dimensional co-occurrence vector.

  In the vocabulary expansion, 100 words were acquired for each news item, and an expanded recognition dictionary was created by adding those not included in the recognition dictionary prepared in advance. In the recognition process, the appearance probability of the unregistered word was used as the linguistic restriction of the added word.

  The experimental results are shown in Table 1. The “recognition dictionary prepared in advance” shown in Table 1 refers to the recognition dictionary shown in the prior art. The unregistered word rate is obtained by the following formula.

  Unregistered word rate = (number of unregistered words included in evaluation data / total number of words in evaluation data) × 100

上記の表１を見ると、本発明の方法による音声認識方法により、入力音声中の認識辞書に含まれない単語を関連語彙として獲得して、予め用意されている認識辞書に追加した拡張認識辞書を生成することで、入力音声に対する未登録語を削減することができ、音声認識精度が改善されることが確認できる。また、拡張認識辞書による音声認識には、予め用意された認識辞書と同じ言語モデルを用いるため、言語モデルをコンピュータのメモリ上に展開しておけば、拡張認識辞書を生成してメモリに展開するだけで、すぐに拡張認識辞書による認識処理を開始することができる。

Referring to Table 1 above, an extended recognition dictionary obtained by acquiring a word not included in the recognition dictionary in the input speech as a related vocabulary by the speech recognition method according to the method of the present invention and adding it to a prepared recognition dictionary By generating, it can be confirmed that unregistered words for the input speech can be reduced and speech recognition accuracy is improved. In addition, since the same language model as the recognition dictionary prepared in advance is used for speech recognition by the extended recognition dictionary, if the language model is expanded on the memory of the computer, the extended recognition dictionary is generated and expanded in the memory. With this, recognition processing by the extended recognition dictionary can be started immediately.

  INDUSTRIAL APPLICABILITY The present invention can be applied to the development of a speech recognition system that can recognize specialized vocabulary related to input contents with few recognition errors due to unregistered words in a transcription system such as broadcast audio and conference audio. It is.

It is a figure for demonstrating the principle of this invention. It is a principle block diagram of this invention. It is a block diagram of the speech recognition apparatus in one embodiment of this invention. It is a flowchart of operation | movement of the speech recognition apparatus in one embodiment of this invention. It is a block diagram of the vocabulary expansion part in one embodiment of this invention. It is a flowchart of operation | movement of the vocabulary expansion part in one embodiment of this invention. It is a structural example in the case of performing the speech recognition process in one embodiment of this invention with a computer.

Explanation of symbols

110 input speech 120 acoustic analysis unit 130 acoustic feature parameter series 140 acoustic model 150 language model 160 recognition dictionary 170 search unit 180 tentative first speech recognition result, speech recognition result (first time)
190 Vocabulary expansion means, vocabulary expansion section 200 Extended dictionary, extended recognition dictionary 210 Re-search means, re-search section 220 Speech recognition result 310 Related vocabulary acquisition section 320 Related vocabulary 370 Vocabulary relevance calculation section 380 Vocabulary database 410 Extended recognition dictionary generation section 520 Voice recognition program memory 530 CPU
540 storage unit

Claims (12)

An acoustic feature parameter is extracted from the input digital signal, and a language score given by a language model and a recognition dictionary prepared in advance and an acoustic score given by the acoustic model are calculated for the acoustic feature parameter, In the speech recognition method of outputting a language unit sequence having the highest score calculated based on the language score and the acoustic score as a recognition result,
A first speech recognition step for performing speech recognition processing for obtaining a provisional speech recognition result using the recognition dictionary prepared in advance when speech is input;
A vocabulary acquisition step of acquiring, from the provisional speech recognition result, a related vocabulary for the content from a database storing information on the relationship between words;
A vocabulary expansion step of creating an extended dictionary in which the acquired related vocabulary is added to the recognition dictionary prepared in advance;
And performing a second speech recognition step of performing speech recognition processing again using the extended dictionary. In the vocabulary acquisition step,
A relevance calculation step of calculating a relevance between the speech recognition result and the related vocabulary based on a reliability of the temporary speech recognition result when acquiring the related vocabulary from the temporary speech recognition result. The speech recognition method according to 1. In the second speech recognition step,
The speech recognition method according to claim 1 or 2, wherein when performing speech recognition processing using the extended dictionary, a linguistic restriction is applied to each word of the added related vocabulary according to a class to which each word belongs. An acoustic analysis unit that converts an input speech signal into a digital signal and extracts an acoustic feature parameter series from the digital signal, an acoustic model storage unit that stores an acoustic model, and a language model that stores a language model A storage unit, a dictionary storage unit storing a prepared recognition dictionary, an acoustic model storage unit, a language model storage unit, and a dictionary storage unit, respectively. Input language recognition dictionary, and for the acoustic feature parameter series of the input speech, calculate the language score given by the language model, the recognition dictionary prepared in advance, and the acoustic score given by the acoustic model. And a search unit that outputs a language unit sequence having the highest score calculated based on the language score and the acoustic score as a speech recognition result. A speech recognition apparatus having bets,
From the provisional first speech recognition result obtained by using the previously prepared recognition dictionary of the dictionary storage means for the input speech, the related vocabulary for the content and the information on the relationship between the words are stored Vocabulary expansion means for creating an extended dictionary obtained by acquiring from the vocabulary database and adding the acquired related vocabulary to the dictionary storage means;
Re-search means for outputting a speech recognition result obtained by using the extended dictionary; The vocabulary expansion means is:
The degree of association for calculating the degree of association between the speech recognition result and the related vocabulary based on the reliability of the provisional first speech recognition result when acquiring the related vocabulary from the provisional first speech recognition result The speech recognition apparatus according to claim 4, further comprising calculation means. The re-search means includes
The speech recognition according to claim 4 or 5, further comprising means for giving a linguistic constraint corresponding to a class to which each word belongs to each word of the added related vocabulary when performing speech recognition processing using the extended dictionary. Method. An acoustic feature parameter is extracted from the input digital signal, and a language score given by a language model and a recognition dictionary prepared in advance and an acoustic score given by the acoustic model are calculated for the acoustic feature parameter, A speech recognition program that outputs a language unit sequence having the highest score calculated based on the language score and the acoustic score as a speech recognition result,
On the computer,
A first speech recognition step for performing speech recognition processing for obtaining a provisional speech recognition result using the recognition dictionary prepared in advance when speech is input;
A vocabulary acquisition step of acquiring, from the provisional speech recognition result, a related vocabulary for the content from a database storing information on the relationship between words;
A vocabulary expansion step of creating an extended dictionary in which the acquired related vocabulary is added to the recognition dictionary prepared in advance;
And executing a second speech recognition step of performing speech recognition processing again using the extended dictionary. In the vocabulary acquisition step,
Claims: When acquiring a related vocabulary from the provisional speech recognition result, executing a relevance calculation step of calculating a relevance between the speech recognition result and the related vocabulary based on the reliability of the provisional speech recognition result. Item 8. The speech recognition program according to item 7. In the second speech recognition step,
The speech recognition according to claim 7 or 8, further comprising a step of providing a linguistic constraint corresponding to a class to which each word belongs to each word of the added related vocabulary when performing speech recognition processing using the extended dictionary. program. An acoustic feature parameter is extracted from the input digital signal, and a language score given by a language model and a recognition dictionary prepared in advance and an acoustic score given by the acoustic model are calculated for the acoustic feature parameter, A storage medium storing a speech recognition program that outputs a language unit sequence having the highest score calculated based on the language score and the acoustic score as a speech recognition result,
On the computer,
A first speech recognition step for performing speech recognition processing for obtaining a provisional speech recognition result using the recognition dictionary prepared in advance when speech is input;
A vocabulary acquisition step of acquiring, from the provisional speech recognition result, a related vocabulary for the content from a database storing information on the relationship between words;
A vocabulary expansion step of creating an extended dictionary in which the acquired related vocabulary is added to the recognition dictionary prepared in advance;
A storage medium storing a speech recognition program, wherein a program for executing a second speech recognition step for performing speech recognition processing again using the extended dictionary is stored. In the vocabulary acquisition step,
Claims: When acquiring a related vocabulary from the provisional speech recognition result, executing a relevance calculation step of calculating a relevance between the speech recognition result and the related vocabulary based on the reliability of the provisional speech recognition result. A storage medium storing the voice recognition program according to Item 10. In the second speech recognition step,
12. The speech recognition according to claim 10 or 11, further comprising a step of providing a linguistic constraint corresponding to a class to which each word belongs to each word of the added related vocabulary when performing speech recognition processing using the extended dictionary. A storage medium that stores a program.

Images