JP6410491B2 - Pronunciation dictionary expansion system, expansion program, expansion method, acoustic model learning method, learning program, and learning system using the extended pronunciation dictionary obtained by the expansion method - Google Patents

Description

  The present invention relates to a technology for improving the accuracy of speech recognition for speech data including utterances that are slowly and carefully uttered, and more particularly to a technology for improving the accuracy of speech recognition by improving a pronunciation dictionary.

  In a large vocabulary continuous speech recognition (LVCSR) system, speech in a different utterance style is often input, such as correcting utterances for misrecognition and singing songs. The utterance deformation peculiar to corrected utterances is called hyper articulation, and there are examples such as dividing into syllables, increasing inflection, etc., but especially observed is deformation that extends the duration of phonemes and syllables, It is an utterance style that speaks slowly and carefully.

  In conventional LVCSR systems with a probabilistic duration model, a mismatch between the probabilistic duration model and the extended duration is given as input with some phonemes and syllables extended. As a result, correct recognition cannot be performed.

  On the other hand, the construction of the LVCSR system requires a large amount of transcription text of speech to be recognized. It is very expensive to prepare such data manually. Therefore, at present, the text (Pseudo Truth: PT) automatically generated using the LVCSR system is often used. However, since field data for obtaining a PT includes the above-described stretched voice, it is difficult to generate an accurate PT. As a result, it is difficult to construct an LVCSR system with high recognition accuracy.

  Patent Documents 1 and 2 and Non-Patent Documents 1 to 4 exist as conventional techniques for constructing a speech recognition system capable of processing speech of an utterance style different from normal.

  Patent Document 1 aims to create a language model and an acoustic model that can realize robust speech recognition for an emphasized utterance in a language in which a syllable break may come after any phoneme as in English. . Patent Document 1 discloses a preparation unit for preparing a language model having a syllable boundary in a predetermined language, and a phoneme adjacent to the syllable boundary for each of the syllable boundaries of the language model prepared by the preparation unit. The phone model description rewriting means for rewriting the description into a predetermined format that allows the insertion of a short pause is disclosed. Patent Document 1 also refers to a means for preparing an acoustic model representing a pose between syllables and an acoustic model representing the pose, and for each of a plurality of phoneme models, Disclosed is an acoustic model conversion device including a phoneme model rewriting unit for rewriting a phoneme model description so as to add a corresponding new state and a route that reaches the terminal by bypassing the new state. To do.

  Patent Document 2 provides a speech recognition device and a speech recognition method capable of reliably recognizing a recognition vocabulary that is likely to cause utterance breaks or utterance errors in a user's utterance input in a recognition dictionary of a speech recognition device. With the goal. Patent Document 2 determines a recognition dictionary that stores a plurality of vocabularies as recognition vocabulary and whether or not the recognition vocabulary stored in the recognition dictionary can be divided into a plurality of vocabularies. An analysis unit that divides a simple recognition vocabulary into a plurality of partial recognition vocabularies, and for the partial recognition vocabulary, a predetermined phoneme is added, replaced with another vocabulary, a corresponding partial recognition vocabulary is deleted, and these are recombined A derivation recognition vocabulary generation unit that generates a derivation recognition vocabulary, a voice input unit that receives input of voice data, a voice detection unit that detects a voice section of voice data received by the voice input unit, and the voice detection unit A speech recognition unit comprising: a speech recognition unit configured to perform speech recognition processing using speech data in the detected speech section using a recognition vocabulary stored in the recognition dictionary and a derivative recognition vocabulary generated by the derivative recognition vocabulary generation unit; It discloses an apparatus.

  Non-Patent Document 1 describes a technique for introducing a new pronunciation into a vocabulary by generating and referring to a phoneme confusion matrix and a technique for learning an acoustic model based on a corpus that collects corrected utterances in order to tackle the problem of hyper articulation. Disclosed are a technology for switching models according to detection of Hyper articulation, and a technology that uses articulation vector models that have articulation characteristics as elements.

  Non-Patent Document 2 adopts a multi-model approach to tackle the problem of Hyper articulation and combines a decoding result using a model specialized for correction speech and a decoding result using a standard model. Disclose.

  Non-Patent Document 3 discloses a technology in which an infant's pronunciation is added to the pronunciation dictionary, and when the infant's utterance is inputted, the standard utterance expression is output through the utterance dictionary. Further, it is disclosed that words are automatically added to the pronunciation dictionary using a pronunciation pattern table, and patterns are added in order of appearance frequency and relative frequency.

  Non-Patent Document 4 discloses the construction of an acoustic model / language model specialized for infant utterance recognition in order to improve the recognition accuracy of child speech recognition. Non-Patent Document 4 also discloses a technique for automatically adding a word to the pronunciation dictionary using a pronunciation pattern table in which sounds changing between standard utterances and infant expressions are recorded.

  However, the methods proposed by Patent Literature 1 and Non-Patent Literatures 1 and 2 are methods for newly constructing a language model and an acoustic model specialized for a specific utterance style. A change is necessary, which is not preferable in terms of cost and cost. On the other hand, the methods proposed by Patent Document 2 and Non-Patent Documents 3 and 4 are changes to the recognition / pronunciation dictionary in speech recognition, and are preferable because they are relatively easy changes.

  However, since Patent Document 2 targets recognition vocabulary that is likely to cause utterance breaks and utterance errors, and Non-Patent Documents 3 and 4 target infant utterances, the duration of phonemes and syllables is extended. It cannot be applied as it is to an extended utterance. Non-patent document 5 is a squirrel as a reference document for explaining the discriminative learning used in the embodiment of the present invention.

JP 2006-243213 A JP 2011-27971 A

Hagen Soltau, “Compensating Hyperarticulation for Automatic Speech Recognition,” Karlsruhe Institute of Technology 2005. Shigeki Matsuda et al., “Speech Recognition System Robust to Noise and Speaking Styles,” ICSLP 2004. Izumi Shindo, 4 others, “Study on real environment speech recognition considering infant's peculiar pronunciation variation”, Acoustical Society of Japan Proceedings, September 2006. Izumi Shindo, 3 others, “Development and Evaluation of Infant Speech Recognition Unit in Public Guidance System”, IPSJ Research Report. HI, Human Interface Study Group Report 2007 (11), PP. 103-108. Xiaodong He, Li Deng, ”Discriminative Learning Speech Recognition: Theory and Practice,” Morgan and Claypool Publishers, August 12, 2008.

  The present invention has been made in view of the above-described problems in the prior art, and an expansion method for extending a pronunciation dictionary so that speech data including utterances in which the duration of phonemes and syllables is extended can be correctly recognized, An object is to provide an extended system and an extended program. Another object of the present invention is to provide a learning method, a learning system, and a learning program for learning an acoustic model using an expanded pronunciation dictionary.

  The present invention provides a method for expanding a pronunciation dictionary including a plurality of vocabularies having the following characteristics as recognition vocabulary in order to solve the above-described problems of the prior art. In the method of expanding the pronunciation dictionary of the present invention, (a) a computer reads out the pronunciation of each vocabulary from the pronunciation dictionary, and (b) the computer stretches a resonance sound included in the read out pronunciation of the vocabulary. A step of generating a new pronunciation (hereinafter referred to as “stretched pronunciation”), and (c) the computer determines whether or not a vocabulary having the same pronunciation as the stretched pronunciation exists in the pronunciation dictionary And (d) expanding the pronunciation dictionary using the stretched pronunciation in response to a determination that the computer does not exist.

  Preferably, when the resonance is a vowel, the resonance is stretched by inserting a corresponding long vowel after the short vowel or repeating the same short vowel. Instead of or together with this, when the resonance sound is a nasal sound, the resonance sound is stretched by repeating the nasal sound.

  Preferably, in the step (b), the computer reads out the pronunciation on the condition that the appearance frequency exceeds a predetermined value corresponding to each of both the notation and the pronunciation. A process for generating a stretched pronunciation is performed.

  More preferably, in the above step (a), the vocabulary from which the pronunciation is read is a vocabulary excluding a syllable vocabulary.

  Preferably, the method of expanding the pronunciation dictionary includes a step of deleting, from the pronunciation dictionary, the extended pronunciation having a low accuracy rate based on a result of speech recognition performed by the computer using the expanded pronunciation dictionary.

  In the above, the present invention has been described as a method of expanding the pronunciation dictionary. However, the present invention provides a pronunciation dictionary expansion program for causing a computer to execute each step of such a pronunciation dictionary expansion method, and a pronunciation dictionary expansion system realized by installing the pronunciation dictionary expansion program in a computer. It can also be grasped.

  The present invention also provides a method for learning an acoustic model having the following features in order to solve the above-mentioned problems of the prior art. According to the acoustic model learning method of the present invention, (a) a computer reads the pronunciation of each vocabulary from the pronunciation dictionary, and (b) the computer stretches a resonance sound included in the read pronunciation of the vocabulary. A step of generating a new pronunciation (hereinafter referred to as “stretched pronunciation”); and (c) the computer uses the stretched pronunciation on the condition that no vocabulary having the same pronunciation as the stretched pronunciation exists in the pronunciation dictionary. (D) the computer acquires speech data recognition results recognized using the extended pronunciation dictionary; and (e) the computer And learning the acoustic model using the recognition result of the voice data as learning data.

  Preferably, the learning in the step (e) is identification learning, and the step (e) includes a step in which the computer executes alignment on the recognition result of the voice data without using the stretched pronunciation.

  More preferably, the step (d) includes the step of the computer creating a list of utterances recognized using stretched pronunciation, and in the step (e), the computer refers to the list of utterances. And weighting the learning data.

  The present invention has been described above as an acoustic model learning method. However, the present invention is grasped as an acoustic model learning program that causes a computer to execute each step of such an acoustic model learning method, and an acoustic model learning system that is realized by installing the acoustic model learning program in the computer. You can also

  In the pronunciation dictionary expansion method having the above-described configuration, for each vocabulary included in the pronunciation dictionary, a resonance sound included in the pronunciation is stretched to generate a new pronunciation, i.e., a stretched pronunciation. Vocabulary with the same pronunciation is added to the pronunciation dictionary on condition that the pronunciation dictionary does not exist. According to such an expansion method, the pronunciation dictionary can be expanded without causing a new problem of erroneous recognition due to the expansion of the pronunciation dictionary.

  In addition, by recognizing speech using the pronunciation dictionary expanded by the above-described method, it is possible to correctly recognize speech data including utterances in which the duration of phonemes and syllables is extended. Moreover, in the acoustic model learning method having the above-described configuration, the acoustic model is learned using the recognition result thus correctly recognized as learning data. A model can be constructed. Other effects of the present invention will be understood from the description of each embodiment.

It is a figure which shows an example of the hardware constitutions of the information processing apparatus suitable for implement | achieving the pronunciation dictionary expansion system which concerns on this embodiment. It is a figure which shows an example of the functional block diagram of the pronunciation dictionary expansion system which concerns on this embodiment. FIG. 3A shows an example of the pronunciation dictionary. FIG. 3B is a diagram for explaining the configuration of one entry in the pronunciation dictionary. FIG. 3C is a diagram illustrating an example of the language model. It is a figure explaining the example of the word by which the addition of the extended pronunciation to a pronunciation dictionary is suppressed. It is a flowchart which shows an example of the flow of the expansion process of the pronunciation dictionary which concerns on this embodiment. It is a figure which shows an example of the functional block diagram of the learning system of the acoustic model which concerns on this embodiment. FIG. 7A is a diagram for explaining an example of alignment by stretching pronunciation. FIG. 7B is a diagram for explaining an example of alignment by normal pronunciation. It is a flowchart which shows an example of the flow of the learning process of the acoustic model which concerns on this embodiment. FIG. 9A is a diagram showing an example of a first experimental result of speech recognition using the present invention. FIG. 9B is a diagram showing an example of a second experimental result of speech recognition using the present invention. It is a figure which shows the relationship between the size suppression effect and error reduction rate of the pronunciation dictionary for every expansion pronunciation addition suppression option.

  DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, modes for carrying out the invention will be described in detail with reference to the drawings. However, the following embodiments do not limit the invention according to the claims, and are described in the embodiments. Not all combinations of features are essential to the solution of the invention. Note that the same numbers are assigned to the same elements throughout the description of the embodiment.

  FIG. 1 shows an exemplary hardware configuration of a computer 100 for implementing the present invention. The computer 100 includes a CPU 102 and a main memory 104, which are connected to a bus 106. CPU 102 is preferably based on a 32-bit or 64-bit architecture. The CPU 102 includes, for example, Intel's Core (TM) i series, Core (TM) 2 series, Atom (TM) series, Xeon (R) series, Pentium (R) series or Celeron (R) series, AMD (Advanced Micro Devices) A series, Phenom (TM) series, Athlon (TM) series, Turion (TM) series or Sempron (TM), or Power (TM) series of International Business Machines Corporation sell.

  A display 110, for example, a liquid crystal display (LCD) can be connected to the bus 106 via a display controller 108. The liquid crystal display (LCD) may be, for example, a touch panel display or a floating touch display. The display 110 can be used to display information output by software running on the computer 100 with an appropriate graphic interface.

  Optionally, a storage device 114, such as a hard disk drive, and a drive 116, such as a CD, DVD or BD drive, may be connected to the bus 106 via, for example, a SATA or IDE controller 112.

  A keyboard 120 and a mouse 122 can optionally be connected to the bus 106 via a peripheral device controller 118, for example, via a keyboard / mouse controller or a USB bus.

  The storage device 114 includes an operating system, for example, a Windows (registered trademark) OS, a UNIX (registered trademark), a MacOS (registered trademark), and a Java (registered trademark) processing environment such as J2EE, a Java (registered trademark) application, A program that provides a Java (registered trademark) virtual machine (VM), a Java (registered trademark) runtime (JIT) compiler, a computer program according to this embodiment, and other programs and data are loaded into the main memory 104 It can be stored as possible.

  The storage device 114 may be built in the computer 100, connected via a cable so that the computer 100 can be accessed, or wired or wireless so that the computer 100 can be accessed. It may be connected via a network.

  The drive 116 may be used to install a computer program, such as an operating system or application, from the CD-ROM, DVD-ROM or BD 117 to the storage device 114 as required. Note that the computer program can be compressed, or divided into a plurality of pieces and recorded on a plurality of media.

  The communication interface 126 follows, for example, the Ethernet (registered trademark) protocol. The communication interface 126 is connected to the bus 106 via the communication controller 124, plays a role of connecting the computer 100 to the communication line 128 by wire or wirelessly, and corresponds to the TCP / IP communication protocol of the communication function of the operating system of the computer 100. Providing a network interface layer. The communication line is, for example, a wired LAN environment based on the wired LAN connection standard, or a wireless LAN environment based on the wireless LAN connection standard, for example, a Wi-Fi wireless LAN environment such as IEEE802.11a / b / g / n, or a mobile phone network environment. (Eg, 3G or 4G (including LTE) environments).

  The computer 100 can receive data from other computers via the communication line 128 and store the data on the storage device 114.

  From the above description, it will be easily understood that the computer 100 is realized by an information processing apparatus such as a normal personal computer, a workstation, a main frame, or a combination thereof. In addition, the component demonstrated above is an illustration, All the components are not necessarily an essential component of this invention. Similarly, it goes without saying that the computer 100 for carrying out the present invention may include other components such as a speaker.

  FIG. 2 shows an example of a functional block diagram of the pronunciation dictionary expansion system 200 according to the present embodiment. The pronunciation dictionary expansion system 200 generates, for each word satisfying a predetermined condition included in the pronunciation dictionary, a new pronunciation by extending the resonance sound included in the pronunciation, that is, a stretched pronunciation, and the generated stretched pronunciation is pronounced. Extend the pronunciation dictionary by adding to the dictionary. The pronunciation dictionary expansion system 200 includes a pronunciation dictionary storage unit 202, a reading unit 204, a frequency confirmation unit 206, a language model learning corpus 208, a PT data storage unit 210, and a new pronunciation generation unit 212. Unit 214, candidate list storage unit 216, and verification unit 218. The pronunciation dictionary storage unit 202, the language model learning corpus 208, the PT data storage unit 210, the candidate list storage unit 216, and the extended pronunciation dictionary storage unit 226 included in the verification unit 218 are physically the same storage device. Or a plurality of storage devices. Each component will be described below.

  The pronunciation dictionary storage unit 202 stores a pronunciation dictionary used in the LVCSR system. The pronunciation dictionary is a dictionary that defines a vocabulary (word) to be recognized and its pronunciation (phonological information). A dictionary 300 shown in FIG. 3A is an example of a Japanese pronunciation dictionary. As shown in FIG. 3A, one entry in the pronunciation dictionary is a pair of kana-kanji spelling and pronunciation, which is called a token in this specification.

  A token will be described with reference to FIG. The token “Kyoto / ky.o.O.t.o” 300 shown in FIG. 3B includes the notation “Kyoto” 312 and the pronunciation “ky.o.O.t.o” 314. The pronunciation 314 is a series of phonemes, and is composed of a combination of vowels, consonants, and nasal sounds. Taking the pronunciation “ky.oOto” 314 as an example, the phoneme “ky” 316a and the phoneme “t” 316d are consonants, the phonemes “o” 316b and 316e are short vowels, and the phoneme “O” 316c is a long vowel. It is. As shown in FIG. 3C, in the language model 320, tokens having the same notation share the occurrence probability. For example, the occurrence probabilities of the token “sushi / s.u.sh.i” and the token “sushi / z.u.sh.i” are both 0.0012.

  The reading unit 204 reads the pronunciation of each word from the pronunciation dictionary stored in the pronunciation dictionary storage unit 202. However, the words from which the reading unit 204 reads out the pronunciation are the remaining words excluding the one mora (one syllable) word. The pronunciation dictionary includes 1-mora words such as “teeth”, “he”, and “wo”. If such a one-mora word is subjected to a predetermined process described below to generate an extended pronunciation and add it to the pronunciation dictionary, the adverse effect of the addition is greater than the benefit of the addition, and the recognition accuracy is greatly degraded. For this reason, the reading unit 204 reads the remaining words from the pronunciation dictionary excluding one mora (one syllable) word.

  The frequency confirmation unit 206 confirms that the appearance frequency of each word read by the reading unit 204 exceeds a predetermined threshold corresponding to both the notation and the pronunciation thereof. More specifically, the frequency confirmation unit 206 counts the number of appearances in the language model learning corpus for each word notation, and determines whether the appearance frequency exceeds a first predetermined threshold. When the language model is constructed based on the corpus for language model learning, the frequency confirmation unit 206 may make the determination with reference to the occurrence probability of the language model. A language model learning corpus or language model is prepared in the language model storage unit 208 in advance.

  The frequency confirmation unit 206 also counts the number of occurrences of the pronunciation in PT data and the total number of occurrences of the notation (including the number of occurrences of tokens with different pronunciations in the same notation) for each word, It is determined whether the ratio of the number of occurrences of pronunciation to the number exceeds a second predetermined threshold. The PT data is a pair of PT, which is a result of automatic recognition of user voice, and corresponding voice data. The PT data is prepared in the PT data storage unit 210 in advance. Finally, the frequency confirmation unit 206 generates a new pronunciation for a word to be described later for a word whose appearance frequency exceeds the first predetermined threshold for notation and whose appearance frequency exceeds the second predetermined threshold for pronunciation. The data is passed to the generation unit 212.

  The filtering process by the frequency confirmation unit 206 is for the following reason. That is, the present invention attempts to construct an LVCSR system that is robust to speech data including utterances with extended phoneme and syllable duration by extending the pronunciation dictionary. However, as the number of words to be recognized increases, the calculation time at runtime increases, and the recognition rate tends to decrease. Therefore, the extended pronunciation added to the pronunciation dictionary should be limited to those expected to improve the recognition rate. Here, since the utterance style that speaks slowly and carefully is a special utterance style that is different from normal, the frequency that a word is uttered in that special utterance style is lower than the frequency that is uttered in the normal utterance style . Accordingly, at least a normal utterance style, that is, a token included in the pronunciation dictionary before expansion, whose appearance frequency is high is set as a processing target of the new pronunciation generation unit 212 described later. Even if the frequency of appearance is high, if the frequency of pronunciation is low, the harmful effect of the addition is greater than the profit of the addition. Therefore, the frequency confirmation unit 206 passes a token whose appearance frequency exceeds a predetermined threshold corresponding to both the notation and the pronunciation to the new pronunciation generation unit 212 for processing to be described later.

  With reference to FIG. 4, an example of a word for which the addition of the extended pronunciation to the pronunciation dictionary is suppressed will be described. The horizontal axis of the table shown in FIG. 4 is the relative appearance frequency of pronunciation in PT data, and the vertical axis is the language model probability. Since words occurring in the region 406 have a low occurrence probability in the language model, and words existing in the region 408 have a low relative appearance frequency of pronunciation with respect to the same notation, each is expanded to a pronunciation pronunciation dictionary. It is a word that is suppressed from adding. For example, the notation “sushi” is frequently pronounced as “s.u.sh.i”, but is rarely pronounced as “z.u.sh.i”. For this reason, the language probabilities of the token “sushi / sush.i” and the token “sushi / zush.i” are equally high, but the frequency confirmation unit 206 processes only the token “sushi / sush.i”. To the new pronunciation generation unit 212. It should be noted that the token “rain / ama” indicated by reference number 402 and the token “outside / soto” indicated by reference number 403 are words that are prevented from being added with an extended pronunciation by the same pronunciation confirmation unit 214 described later. is there. Further, the token “Large / o.o” indicated by reference numeral 404 is a word that is prevented from being added with an extended pronunciation by processing performed by the verification unit 218 described later.

  The new pronunciation generation unit 212 generates a stretched pronunciation by stretching the resonance sound included in the pronunciation of the token passed by the frequency confirmation unit 206. Then, the new pronunciation generation unit 212 pairs the generated extended pronunciation with the same notation as the original token and passes it to the same pronunciation confirmation unit 214 described later as a new token. Here, when the resonance sound included in the token pronunciation is a vowel, the resonance sound is stretched by inserting a corresponding long vowel after the short vowel or repeating the same short vowel after the short vowel. It's okay. When the resonance sound included in the pronunciation of the token is a nasal sound, the resonance sound may be stretched by repeating the nasal sound.

  The expansion of the resonance sound will be explained using the token “sushi / s.u.sh.i” as an example. In the pronunciation “s.u.sh.i”, “u” and “i” are short vowels. Therefore, the corresponding long vowel “U” is inserted after the short vowel “u”, and the corresponding long vowel “I” is inserted after the short vowel “i”. Then, the extended pronunciation based on the pronunciation “s.u.sh.i” becomes “s.u.U.sh.i.I”. The token “Ninja / n.i.N.j.a” will be described as an example. In the pronunciation “n.i.N.j.a”, “i” and “a” are short vowels, and “N” is a nasal sound. Therefore, the corresponding long vowel is inserted after the short vowels “i” and “a”, and the same nasal sound is inserted after the nasal sound “N”. Then, the extended pronunciation based on the pronunciation “n.i.N.j.a” becomes “n.i.I.N.N.j.a.a”. Taking the token “Kyoto / ky.oOto” as an example, in the pronunciation “ky.oOto”, the first “o” and the second “o” are short vowels, but after the first “o” The long vowel “O” already exists. Therefore, nothing is performed on the first short vowel “o”, and the long vowel “O” is inserted only after the second “o”. Then, the extended pronunciation based on the pronunciation “ky.o.O.t.o” becomes “ky.o.O.t.o.O”.

  The same pronunciation confirmation unit 214 determines whether or not a token having the same pronunciation is present in the pronunciation dictionary for the extended pronunciation generated by the new pronunciation generation unit 212. Then, the same pronunciation confirmation unit 214 extracts only the extended pronunciations determined to have no tokens having the same pronunciation in the pronunciation dictionary from the extended pronunciations generated by the new pronunciation generation unit 212, and stores them in the pronunciation dictionary. The candidate list is stored in the candidate list storage unit 216 as an additional candidate.

  The above processing by the same pronunciation confirmation unit 214 is to prevent erroneous recognition of the LVCSR system at runtime. In other words, if a token with the same pronunciation as the extended pronunciation exists in the pronunciation dictionary, adding that extended pronunciation to the pronunciation dictionary will result in a misrecognition despite the correct pronunciation at runtime. Side effects will occur. For example, it is assumed that a stretched pronunciation “Y.u.U.m.e.E” is generated for the token “dream / Y.u.m.e”. At this time, if the token “famous / YuUmeE” exists in the pronunciation dictionary, the LVCSR system may output “dream” even though voice data is input as “famous / YuUmeE”. End up. In order to prevent such a situation, the above function of the same pronunciation confirmation unit 214 is important.

  The verification unit 218 verifies the validity of the extended pronunciation of the additional candidate stored in the candidate list storage unit 216, and includes a decoding unit 220, a removal unit 222, a test data storage unit 224, and an extended pronunciation dictionary storage. Part 226. The test data storage unit 224 stores a pair of test voice data and correct text data. The extended pronunciation dictionary storage unit 226 stores a copy of the pronunciation dictionary stored in the pronunciation dictionary storage unit 202 as an initial value.

  The decoding unit 220 receives the test voice data read from the test data storage unit 224 as an input, and uses the pronunciation dictionary stored in the pronunciation dictionary storage unit 202 and the extended pronunciation of additional candidates stored in the candidate list storage unit 216. Referring to the above, the audio data is decoded. The decoding of the audio data is a known method and is not the gist of the present invention, so that the description is omitted.

  The removal unit 222 receives the decoding result from the decoding unit 220, compares the decoding result with the correct text data stored in the test data storage unit 224, and puts either a correct or incorrect mark on the decoding result. Put on. The removal unit 222 also calculates the ratio of the number of correct answers to the total number of decoded extended pronunciations for each additional candidate extended pronunciation. Then, the removal unit 222 extracts only the extended pronunciation in which the calculated ratio exceeds a predetermined threshold from the expanded pronunciations of the additional candidates stored in the candidate list storage unit 216, and the pronunciation dictionary stored in the extended pronunciation dictionary storage unit 226 Add to.

  Note that the processing performed by the frequency check unit 206 and the verification unit 218 described above is optional, and configurations excluding these processing are also included in the technical scope of the present invention. In order to simplify the description of the evaluation experiment to be described later, the function of the new pronunciation generation unit 212 is A1, the function of the same pronunciation storage unit 214 is A2, the function of the frequency confirmation unit 206 is the processing function for the notation A3, The processing function for pronunciation is named A4, and the function of the verification unit 218 is named A5.

  Next, an example of a pronunciation dictionary expansion process performed by the pronunciation dictionary expansion system 200 according to the present embodiment will be described with reference to FIG. The pronunciation dictionary expansion process starts at step 500, and the pronunciation dictionary expansion system 200 reads a token from the pronunciation dictionary. Subsequently, the pronunciation dictionary expansion system 200 determines whether or not the appearance frequency in the language model learning corpus for the read token notation exceeds a predetermined threshold A (step 502). When the appearance frequency of the notation of the read token is equal to or lower than the predetermined threshold A (step 502: NO), the process returns to step 500, and the pronunciation dictionary expansion system 200 reads the next token from the pronunciation dictionary.

  On the other hand, when the appearance frequency of the read token notation exceeds the predetermined threshold A (step 502: YES), the process proceeds to step 504, and the pronunciation dictionary expansion system 200 generates the appearance frequency in the PT data for the pronunciation of the read token. Whether or not exceeds a predetermined threshold B. When the appearance frequency of pronunciation of the read token is less than or equal to the predetermined threshold B (step 504: NO), the process returns to step 500, and the pronunciation dictionary expansion system 200 reads the next token from the pronunciation dictionary.

  On the other hand, when the appearance frequency of the pronunciation of the read token exceeds the predetermined threshold B (step 504: YES), the process proceeds to step 506, and the pronunciation dictionary expansion system 200 generates the resonance sound included in the pronunciation of the read token. Stretching pronunciation is generated by stretching. Subsequently, the pronunciation dictionary expansion system 200 determines whether or not a token having the same pronunciation as the generated extended pronunciation is included in the pronunciation dictionary (step 508). When a token having the same pronunciation as the generated extended pronunciation is included in the pronunciation dictionary (step 508: YES), the process returns to step 500, and the pronunciation dictionary expansion system 200 reads the next token from the pronunciation dictionary.

  On the other hand, if a token having the same pronunciation as the generated extended pronunciation is not included in the pronunciation dictionary (step 508: NO), the process proceeds to step 510, and the pronunciation dictionary expansion system 200 transfers the generated extended pronunciation to the pronunciation dictionary. Add to the candidate list to add. Subsequently, the pronunciation dictionary expansion system 200 determines whether or not there is an unexamined token in the pronunciation dictionary (step 512). If an unexamined token remains in the pronunciation dictionary (step 512: NO), the process returns to step 500, and the pronunciation dictionary expansion system 200 reads the next token from the pronunciation dictionary.

  On the other hand, if there are no unexamined tokens remaining in the pronunciation dictionary (step 512: NO), the process proceeds to step 514, and the pronunciation dictionary expansion system 200 decodes the test data by referring to the candidate list, and decodes the test data. The correct answer rate for the extended pronunciation is calculated by comparing the result with the correct text data (step 514). Subsequently, the pronunciation dictionary expansion system 200 expands the pronunciation dictionary by adding the extended pronunciation whose calculated correct answer rate exceeds the predetermined threshold C to the pronunciation dictionary (step 516). Then, the process ends.

  FIG. 6 shows an example of a functional block diagram of the acoustic model learning system according to the present embodiment. The acoustic model learning system 600 generates PT data by automatically recognizing speech data with reference to the extended pronunciation dictionary expanded by the above-described method, and learns the acoustic model by identification learning using the generated PT data as learning data. To do. The acoustic model learning system 600 includes an audio data storage unit 602, a decoder 604, a first recognition result storage unit 606, an alignment execution unit 608, a second recognition result storage unit 610, a weighting execution unit 612, 3 recognition result storage part 614 and learning part 616 are included. The pronunciation dictionary storage unit 202 and the extended pronunciation dictionary storage unit 226 shown in FIG. 6 are the same as the pronunciation dictionary storage unit 202 and the extended pronunciation dictionary storage unit 226 shown in FIG. The pronunciation dictionary storage unit 202, the extended pronunciation dictionary storage unit 226, and the first to third recognition result storage units 606, 610, and 614 may be physically the same storage device, or may be a plurality of storage devices. There may be. Each component will be described below.

  The voice data storage unit 602 stores voice data (field data) of the target domain.

  The decoder 604 reads the audio data from the audio data storage unit 602, decodes the read audio data with reference to the extended pronunciation dictionary stored in the extended pronunciation dictionary storage unit 226. At this time, the decoder 604 creates a list of decoding results decoded with reference to the extended pronunciation, and passes it to the weighting execution unit 612 described later. The decoder 604 stores the decoding result, that is, PT data in the first recognition result storage unit 606. The decoding of the audio data is a known method as described above, and is not the gist of the present invention, so the description is omitted.

  In speech data decoding using a conventional pronunciation dictionary, utterances with extended phoneme and syllable duration are not recognized correctly, so the quality of PT data was maintained by discarding such utterances in advance. . However, the acoustic model constructed based on the PT data thus generated cannot correctly recognize the utterances that are slowly and carefully uttered. Therefore, in the acoustic model learning system 600 according to the present embodiment, by using the extended pronunciation dictionary, it is possible to correctly recognize an utterance in which the duration of phonemes and syllables is extended, and solve the above problem.

  The alignment execution unit 608 refers to the pronunciation dictionary stored in the pronunciation dictionary storage unit 202 and aligns PT data stored in the first recognition result storage unit 606. The alignment execution unit 608 stores the alignment result in the second recognition result storage unit 610. Normally, PT data is aligned using a pronunciation dictionary referred to when PT data is acquired. However, the learning of the acoustic model by the learning unit 616, which will be described later, uses identification learning, and here, alignment is performed with reference to a normal pronunciation dictionary instead of an extended pronunciation dictionary. That is, if extended pronunciation is included in the pronunciation dictionary during identification learning, learning for identifying even such a pair is performed even though both normal pronunciation and extended pronunciation are correct. . Therefore, the alignment execution unit 608 executes alignment using a normal pronunciation dictionary in order to prevent such waste.

  FIGS. 7A and 7B show the alignment of the decoding result “sushi”. FIG. 7A shows an alignment using the extended pronunciation “s.u.U.sh.i.I”. FIG. 7B shows alignment by normal pronunciation “s.u.sh.i”. “SIL” indicates silence. In order to obtain an alignment using normal pronunciation as shown in FIG. 7B, in addition to referring to the normal pronunciation dictionary as described above, an extended pronunciation dictionary is removed from the extended pronunciation dictionary and the alignment program is run. The recognition result obtained by referring to the extended pronunciation dictionary (for example, the sentence “sushi (suUsh.iI) (o) eat (taberu)”) is obtained by referring to the normal pronunciation dictionary ( You may run the alignment program after converting “sushi (sush.i) to (o) eat (taberu)”). Instead, by referring to the alignment of the stretched pronunciation and returning the stretched pronunciation portion to the original pronunciation by combining (in the example of FIG. 7A, “uU” is combined to the original pronunciation “u ”May be combined with“ iI ”to obtain the original pronunciation“ i ”), and alignment by normal pronunciation may be acquired.

  The weighting execution unit 612 weights the alignment result read from the second recognition result storage unit 610 based on the decoding result list received from the decoder 604. Then, the weighting execution unit 612 stores the weighted result in the third recognition result storage unit 614. More specifically, the weighting execution unit 612 weights a word or sentence recognized based on the stretched pronunciation in the alignment result. As described above, since the alignment is performed using normal pronunciation, the weighting execution unit 612 identifies the recognized word or sentence with reference to the extended pronunciation based on the decoding result list.

  The learning unit 616 reads the weighted alignment result from the third recognition result storage unit 614 and learns the acoustic model by identification learning. In discriminative learning, correct words (correct phoneme strings) and incorrect words (incorrect phoneme strings, opposite candidates) are used, so that the difference between the correct and incorrect phoneme string scores increases. The model parameters are estimated and the model is learned. As the conflict candidate, the N best candidate for speech recognition is used, or the conflict candidate is extracted from the word lattice obtained from the decoder. Here, if the pronunciation dictionary contains a stretched pronunciation, the stretched pronunciation is similar to the corresponding normal pronunciation, so a pair of normal pronunciation and the corresponding extended pronunciation always appears as a correct phoneme sequence and an incorrect answer phoneme sequence. As a result, the learning time for identification is increased. Therefore, in the present invention, as described above, the decoding results are aligned using a normal pronunciation dictionary. The learning unit 616 outputs an acoustic model as a learning result. For further details of discriminative learning, see Non-Patent Document 5, for example.

  Next, an example of the flow of acoustic model learning processing by the acoustic model learning system 600 will be described with reference to FIG. The acoustic model learning process starts in step 800, and the acoustic model learning system 600 recognizes the recognition result of the speech data recognized with reference to the extended pronunciation dictionary and the word or sentence recognized with reference to the extended pronunciation. Get list and. Subsequently, the acoustic model learning system 600 performs alignment on the recognition result using a normal pronunciation dictionary (step 802).

  Subsequently, the acoustic model learning system 600 weights the alignment result based on the word or sentence list acquired in Step 800 (Step 804). Subsequently, the acoustic model learning system 600 identifies and learns the acoustic model using the weighted alignment result as learning data (step 806). Then, the process ends.

Next, a speech recognition evaluation experiment using the extended utterance dictionary and acoustic model proposed by the present invention will be described with reference to FIGS. In the evaluation experiment described with reference to FIGS. 9A and 9B, two types of speech data were used: test set 1 composed of a short utterance centered on a noun phrase and test set 2 including a long sentence. . In the evaluation experiment of FIG. 9A, the acoustic model used is a conventional acoustic model, that is, an acoustic model constructed by discriminative learning based on PT data generated using a normal pronunciation dictionary. There are the following five variations of the extended utterance dictionary evaluated.
Variation 0 (original voc): Normal pronunciation dictionary variation 1 (A1 + A2): Extended pronunciation dictionary variation 2 (A1 + A2 + A3) with extended pronunciation not present in the pronunciation dictionary: Stretched pronunciation not present in the pronunciation dictionary and notation Extended pronunciation dictionary variation 3 (A1 + A2 + A3 + A4) with extended pronunciation of vocabulary with high appearance frequency: Extended pronunciation with extended pronunciation that does not exist in the pronunciation dictionary and has a high occurrence frequency of notation and pronunciation Pronunciation dictionary variation 4 (A1 + A2 + A3 + A4 + A5): Extended pronunciation dictionary in which extended pronunciation with a low correct answer rate is removed from the extended pronunciation dictionary of variation 4

  In the above experiment, the kana error reduction rate and the character error reduction rate were measured. Here, the kana error rate (KER: Katakana Error Rate) is a recognition rate evaluation scale in which correct data and hypothesis data are translated into a katakana character string (katakana sequence) representing pronunciation and matched. The character error rate (CER) is a recognition rate evaluation scale mainly used for evaluating OCR character recognition and Kanji conversion, and is a method of matching on a character basis. In this evaluation experiment, the reduction rate of each error rate was obtained based on the recognition result using the normal pronunciation dictionary of variation 0.

  As can be seen from the table in FIG. 9A, the error reduction rate is high for the extended pronunciation dictionaries of all variations 1 to 4 including the extended pronunciation dictionary (variations 2 and 3) in which the increase in the dictionary size is suppressed. Has been achieved.

  In the evaluation experiment of FIG. 9B, two acoustic models were prepared: the conventional acoustic model described above and the acoustic model reconstructed by the acoustic model learning method according to the present embodiment described above. As the pronunciation dictionary, the extended pronunciation dictionary of variation 4 described above was used. In this case, a combination of a normal pronunciation dictionary and a conventional acoustic model is used as a reference.

  As can be seen from the table of FIG. 9B, a sufficient error reduction rate is achieved even by using the acoustic model reconstructed by the acoustic model learning method according to the present embodiment. In addition to the reconstructed acoustic model, a higher error reduction rate is achieved when the extended pronunciation dictionary of variation 4 is used.

  FIG. 10 is a table showing the relationship between the number of pronunciations in the dictionary and the error reduction rate for each of the extended pronunciation dictionaries of variations 1 to 4 described above. From the table shown in FIG. 10, by prohibiting the addition of extended pronunciation of vocabulary with low occurrence frequency of pronunciation and notation, the increase in the number of pronunciations can be significantly suppressed without substantially affecting the error reduction rate. I understand.

  As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the range as described in the said embodiment. It will be apparent to those skilled in the art that various modifications or improvements can be added to the above-described embodiments. Accordingly, it is a matter of course that embodiments with such changes or improvements are also included in the technical scope of the present invention.

  In addition, the execution order of each process such as operations, procedures, steps, and stages in the apparatus, system, program, and method shown in the claims, the specification, and the drawings is particularly “before”, “ It should be noted that “preceding” is not specified, and the output of the previous process can be realized in any order unless it is used in the subsequent process. Also, even when the output of the previous process is used in the subsequent process, it may be possible that another process enters between the previous process and the subsequent process, or another process may be inserted in between. Note that it may be possible to change the previous processing to be performed immediately before the subsequent processing even though it is described in the above. Even though the operation flow in the claims, the description, and the drawings is described using “first,” “next,” “follow,” etc. for convenience, it is essential to carry out in this order. It does not necessarily mean that.

Claims (13)

A method for extending a pronunciation dictionary including a plurality of vocabulary as recognition vocabulary by a computer,
Before Symbol computer, a step of reading the pronunciation of each vocabulary from the pronunciation dictionary,
Before SL computer, the vocabulary read the pronunciation, for its representation, the occurrence frequency in the corpus language model exceeds a first predetermined value, and its pronunciation, a pair of audio data and their automatic recognition result A step of generating a new pronunciation (hereinafter referred to as “stretched pronunciation”) by extending a short vowel or nose included in the pronunciation of the vocabulary on the condition that the frequency of occurrence in the data exceeds a second predetermined value When,
Before SL computer, determining whether present in the stretching pronunciation and the same vocabulary the pronunciation dictionary with pronunciation,
A step of pre-Symbol computer, in response to a determination that no, expanding the sound dictionary using said stretching sound,
Including methods. 2. The pronunciation dictionary expansion method according to claim 1, wherein the extension of the short vowel is performed by inserting a corresponding long vowel after the short vowel or repeating the same short vowel. 2. The pronunciation dictionary expansion method according to claim 1 , wherein the expansion of the nasal sound is performed by repeating the nasal sound. The pronunciation dictionary expansion method according to claim 1, wherein in the reading step , the vocabulary from which the pronunciation is read is a vocabulary excluding a syllable vocabulary.   The pronunciation dictionary expansion method according to claim 1, further comprising the step of deleting, from the pronunciation dictionary, a stretched pronunciation having a low accuracy rate based on a result of speech recognition performed using the expanded pronunciation dictionary. The computer, expansion program pronunciation dictionary to execute the steps of the expansion process of the pronunciation dictionary according to any one of claims 1 to 5. Comprising means adapted to perform the steps of the expansion process of the pronunciation dictionary according to any one of claims 1 to 5, expansion system pronunciation dictionary. Using the extended pronunciation dictionary by expansion methods Pronunciation dictionary according to any one of claims 1 to 5, a computer, a method of learning an acoustic model,
The computer obtaining a recognition result of speech data that has been speech-recognized using the extended pronunciation dictionary;
The computer, the recognition result of the speech data as the training data, and a step of learning an acoustic model,
Learning in the learning step is identification learning, and the learning step includes a step in which the computer executes alignment on a recognition result of the speech data without using the stretched pronunciation. Method. A method of learning an acoustic model by a computer,
Before Symbol computer, a step of reading the pronunciation of each vocabulary from the pronunciation dictionary,
Before SL computer, the vocabulary read the pronunciation, for its representation, the occurrence frequency in the corpus language model exceeds a first predetermined value, and its pronunciation, a pair of audio data and their automatic recognition result A step of generating a new pronunciation (hereinafter referred to as “stretched pronunciation”) by extending a short vowel or nose included in the pronunciation of the vocabulary on the condition that the frequency of occurrence in the data exceeds a second predetermined value When,
Before SL computer, the steps of vocabulary having the stretching pronunciation and same pronunciation on condition that does not exist in the sound dictionary, expanding the sound dictionary using said stretching sound,
Before SL computer were speech recognition using the extended the pronunciation dictionary, and obtaining a recognition result of the speech data,
Before SL computer, a recognition result of the speech data as the training data, comprising the steps of learning the acoustic model,
Learning method of acoustic model including Learning in the step of the learning is discriminative training, said step of learning, the computer, the alignment of the recognition result of the speech data includes performing without using the stretching pronunciation vocabulary, claim 9 The learning method of the acoustic model as described in 2. The obtaining step includes a step in which the computer creates a list of utterances recognized using the stretched pronunciation, and in the learning step , the computer weights learning data with reference to the list. The method for learning an acoustic model according to claim 10 , comprising the step of: The computer, acoustic model learning program for executing the steps of the method of learning the acoustic model according to any one of claims 9 to 11. Comprising means adapted to perform the steps of the method of learning the acoustic model according to any one of claims 9 to 11, the acoustic model learning system.

Images