JP2008129527A - Sound model generating device, method and program, and its recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a sound model generating technique in which recognition performance of an important keyword to be recognized is improved. <P>SOLUTION: A learning section 13 calculates a sufficient amount of statistics from a learning data which is read from a learning data recording section 11, and a base sound model. An enhancement learning data selection section 22 extracts the learning data including the same key word as or similar keyword to a predetermined keyword from the learning data recording section 11, and makes the extracted data an enhancement learning data. An enhancement learning section 24 calculates an enhancement sufficient amount of statistics from the enhancement learning data and the base sound model. An enhancement model combining section 26 generates the enhancement sound model from the sufficient amount of statistics and a value in which weight is multiplied by the enhancement sufficient statistic. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an acoustic model generation apparatus, method, program, and recording medium for creating a probability model that represents the acoustic features of each phoneme for an acoustic feature parameter sequence calculated from an input speech signal.

  In conventional speech recognition, an acoustic model is created by modeling with a Hidden Markov Model (hereinafter referred to as “HMM”) for each category of speech units such as phonemes, syllables, and words constituting the recognition result candidate. The method has high recognition performance and has become the mainstream of current speech recognition technology. The HMM is detailed in Non-Patent Document 1, for example.

With reference to FIG. 3, the example of the acoustic model production | generation apparatus by a prior art is demonstrated. FIG. 3 is a diagram illustrating a functional configuration of the acoustic model generation device 9 according to the related art. FIG. 4 is a diagram illustrating a processing procedure of a conventional acoustic model generation method.
The acoustic model generation device 9 records a learning data recording unit 11 in which learning data including voice data and a label in which phonemes are associated with the voice data is recorded, and a base acoustic model that is an acoustic model before learning is recorded. Base acoustic model recording unit 12, learning unit 13, sufficient statistic recording unit 14 in which sufficient statistic necessary for calculating acoustic parameters is recorded, model synthesis unit 15, and acoustic model after learning Is recorded from the acoustic model recording unit 16.

The learning unit 13 accumulates statistics from the learning data read from the learning data recording unit 11 and the base acoustic model read from the base acoustic model recording unit 12, and calculates sufficient statistics (step S91). The calculated sufficient statistics are stored in the sufficient statistics recording unit 14. The sufficient statistics are detailed in Non-Patent Document 2, for example.
The model synthesis unit 15 generates an acoustic model after learning using the sufficient statistics read from the sufficient statistics recording unit 14 (step S92). The generated acoustic model is recorded in the acoustic model recording unit 16. The processing of the learning unit 13 and the processing of the model synthesis unit 15 are detailed in Non-Patent Document 2, for example.
Seiichi Nakagawa, “Speech recognition using probabilistic models”, edited by IEICE, Corona, July 1988 Lawrence Rabiner and Biing-Hwang Juang, written by Sadahiro Furui, “Basics of Speech Recognition (Part 2)”, NTT Advanced Technology, 1995

  In the acoustic model generation device 9 of the prior art, an acoustic model is generated so that a word having a phoneme sequence having a high appearance frequency in the learning data is easily recognized. In other words, a word having a phoneme sequence with a low appearance frequency in the learning data is harder to be recognized by the acoustic model. For this reason, when the phoneme series of the important word / keyword to be recognized is included in the learning data, there is a problem that the recognition performance of the important word / keyword is lowered.

  In the present invention, sufficient statistics are calculated from the learning data read from the learning data recording means and the base acoustic model. Learning data including a keyword that is the same as and / or similar to a predetermined keyword is extracted from the learning data recording means and used as reinforcement learning data. A reinforcement sufficient statistic is calculated from the reinforcement learning data and the base acoustic model. A reinforced acoustic model is generated from the sufficient statistic and the weighted reinforced sufficient statistic.

  The recognition performance of the predetermined keyword can be improved by generating a reinforcement acoustic model by weighting the reinforcement sufficient statistics calculated from the learning data including the same and / or similar keyword as the predetermined keyword. In addition, this reduces the number of words that are erroneously recognized, thereby improving speech recognition performance.

With reference to FIG. 1 and FIG. 2, the acoustic model production | generation apparatus 1 by this invention is demonstrated. FIG. 1 is a diagram illustrating a functional configuration of the acoustic model generation device 1. FIG. 2 is a diagram illustrating processing of the acoustic model generation device 1.
As illustrated in FIG. 1, the acoustic model generation apparatus 1 records a learning data recording unit 11, a base acoustic model recording unit 12, a learning unit 13, and sufficient statistics that are statistics necessary for calculating acoustic parameters. Sufficient statistics recording unit 14, reinforcement keyword list recording unit 21, reinforcement learning data selection unit 22, reinforcement learning data recording unit 23, reinforcement learning unit 24, reinforcement sufficient statistics recording unit 25, reinforcement model synthesis unit 26, learning It is comprised from the reinforcement | strengthening acoustic model recording part 27 by which the acoustic model which is a later acoustic model is recorded.

<Step S1>
The learning data recording unit 11 stores in advance a plurality of learning data composed of voice data and a label in which phonemes are associated with the voice data. In the base acoustic model, a base acoustic model that is an acoustic model before learning is stored in advance.
The learning unit 13 calculates a sufficient statistic from the learning data read from the learning data recording unit 11 and the base acoustic model read from the base acoustic model, and stores it in the sufficient statistic recording unit 14 (step S1).

とする。このとき、音響モデルパラメータである、時刻ｔ＝１に状態ｉに存在すると期待される頻度（回数）π_ｊ、状態遷移確率ａ_ｉｊ、状態ｊで特徴ベクトルｖ_ｋが観測される確率ｂ_ｊ（ｋ）、状態ｉのｍ番目の混合分布に対する混合重み係数ｃ_ｊｍ、状態ｉのｍ番目の混合分布の平均ベクトルμ_ｊｍ、共分散行列Ｕ_ｊｍは、例えば、それぞれ次のように定義される。

ここで、プライムはベクトルの転置を表し、γ_ｔ（ｉ，ｋ）は、観測された特徴量ｏ_ｔを考慮したときに、時刻ｔに状態ｉのｍ番目の混合要素に存在する確率である。 Hereinafter, sufficient statistics will be described. Sufficient statistics are parameters that characterize the HMM and include statistics related to transition probabilities, mixing weights of mixed normal distribution, average, statistics related to variance, and the like. Specifically, when an acoustic model λ and an observation sequence O are given, the probability ξ _t (i, j) that exists in state i at time t and in state j at time t + 1, and acoustic model λ And an observation sequence O, and the state is composed of N states, the probability γ _t (i) existing in the state i at time t is

And At this time, the frequency (number of times) π _j that is expected to exist in the state i at time t = 1, the state transition probability a _ij , and the probability b _j (where the feature vector v _k is observed in the state j, which are acoustic model parameters. k), the mixture weight coefficient c _jm for the m-th mixture distribution in state i, the mean vector μ _jm of the m-th mixture distribution in state i, and the covariance matrix U _jm are defined as follows, for example.

Here, the prime denotes the transpose of a vector, gamma _{t (i,} k), when considering the observed feature quantity o _t, is the probability that exists m th mixture component of the state i at time t .

At this time, the values defined by the mathematical formulas described in the numerator and denominator of the above formula are sufficient statistics. That is, for a _ij , Σ _{t = 1} ^T−1 ξ _t (i, j) and Σ _{t = 1} ^T−1 γ _t (i) are sufficient statistics. For b _j (k), _{Σt = 1 (st. Ot = Vk)} ^T γ _t (j) and Σ _{t = 1} ^T γ _t (j) are sufficient statistics. The same applies to c _im , μ _im , and U _jk . For π _i , γ ₁ (i) is a sufficient statistic. These sufficient statistics can be estimated using the learning data read from the learning data recording unit 11 and using an existing algorithm such as a Baum-Welch algorithm, for example. The Baum-Welch algorithm is detailed in Non-Patent Document 2.
As will be described later, a sufficient statistic is calculated for each learning data recorded in the learning data recording unit 11.

<Step S2>
The strengthened keyword list recording unit 21 records a list of predetermined keywords. The predetermined keyword is, for example, a keyword whose recognition performance is to be improved or a misrecognition keyword, and is registered in advance prior to the process of step S2. Here, in the present invention, the keyword is not limited to one word, but also means a segment composed of a plurality of words and a key phrase.

  The reinforcement learning data selection unit 22 extracts learning data including the same and / or similar keyword as the keyword read from the reinforcement keyword list recording unit 21 from the learning data recording unit 11 and creates reinforcement learning data (step) S2). The created reinforcement learning data is recorded in the reinforcement learning data recording unit 23.

  For example, when a part of certain learning data includes a keyword that is the same as and / or similar to the predetermined keyword, the reinforcement learning data selection unit 22 is the same as the predetermined keyword in the learning data and / or The learning data of the part related to the similar keyword is extracted and used as reinforcement learning data. In addition, phrases, sentences, utterances, and utterances containing keywords that are the same and / or similar to predetermined keywords may be extracted and used as reinforcement learning data. Further, for example, when learning data is created for each utterance / utterance, the entire learning data including the same and / or similar keyword as the predetermined keyword is extracted to create reinforcement learning data. Also good.

  The “same and / or similar keyword” means any one keyword of the same keyword and similar keyword, or both the same keyword and similar keyword. That is, “the same and / or similar keyword as the keyword read from the enhanced keyword list recording unit 21” refers to the same keyword as the keyword read from the enhanced keyword list recording unit 21 and the keyword read from the enhanced keyword list recording unit 21. This means either one of the similar keywords, or the same keyword as the keyword read from the enhanced keyword list recording unit 21 and the keyword similar to the keyword read from the enhanced keyword list recording unit 21.

  When the learning keyword does not include the same keyword as the predetermined keyword, or when the learning data includes a small number of the same keywords as the predetermined keyword, Learning data including similar keywords may be extracted. As a result, it is possible to secure an amount of an enhanced sufficient statistic, which will be described later, and improve recognition performance. Of course, learning data including a keyword similar to the predetermined keyword may be extracted even when the number of the same keywords as the predetermined keyword included in the learning data is large.

  Note that the same keyword as the predetermined keyword is not limited to a keyword having a common name and concept, but may include a keyword having a common name but a different concept. For example, “clouds” and “蜘蛛” have the same name as “spider” but have different concepts. If there is a small amount of learning data that includes the keyword “蜘蛛” that has the same concept as “cloud” as the predetermined keyword, the keyword “cloud” that has the same name but different concept Learning data including “蜘蛛” may be extracted. As described above, by extracting learning data including keywords having different ideas but having common names, it is possible to secure an amount of a sufficient reinforcement statistic described later and improve recognition performance.

Note that the similarity determination as to whether or not the keyword is similar to a predetermined keyword can be performed as follows, for example. And phonemes constituting the predetermined keyword, by comparing the phonemes constituting the keywords to be analogy determination, it is determined to be similar if the ratio of the matching phoneme is the threshold value z ₁ or more. In such a case, the phoneme sequence conversion unit 31 of the reinforcement learning data selection unit 22 converts each keyword read from the reinforcement keyword list recording unit 21 into a phoneme sequence. The reinforcement learning data selection unit 22 compares the phonemes constituting the predetermined keyword with the phonemes constituting the keyword that is the target of similarity determination, and is similar to the case where the proportion of matching phonemes is equal to or greater than the threshold z _1. Judge that

Further, the reinforcement learning data selection unit 22 obtains phoneme similarity distances in terms of phonology for a predetermined keyword and a keyword to be compared, and whether the reciprocal of the sum of the distances is greater than or equal to a threshold z ₂ The analogy may be determined by In such a case, the phoneme sequence conversion unit 31 converts a predetermined keyword into a phoneme sequence, and the distance calculation unit 32 of the reinforcement learning data selection unit 22 calculates a phoneme similarity distance in phonological theory.

Further, in the base acoustic model read out from the base acoustic model recording unit 12 by the reinforcement learning data selection unit 22, the inter-distribution distance between the phonemes constituting the predetermined keyword and the phonemes constituting the keyword to be compared is determined. determined may determine analogy depending on whether the inverse of the sum of the distance is the threshold value z ₃ or more. That is, the distance between states is obtained by obtaining the sum of the distances between distributions, the distance between phonemes is obtained by obtaining the sum of the distances between states, and the analogy is judged from the distance between the phonemes. In such a case, the phoneme sequence conversion unit 31 converts a predetermined keyword read from the enhanced keyword list recording unit 21 into a phoneme sequence, and the distance calculation unit 32 ′ calculates the inter-distribution distance.

<Step S3>
The reinforcement learning unit 24 calculates a sufficient reinforcement statistic from the reinforcement learning data read from the reinforcement learning data recording unit 23 (step S3). The calculated enhanced sufficient statistic is stored in the enhanced sufficient statistic recording unit 25.
The reinforcement sufficient statistic is a sufficient statistic calculated based on the reinforcement learning data read from the reinforcement learning data recording unit 23. That is, the sufficient statistic and the enhanced sufficient statistic have the same definition expressed by mathematical formulas, but the sufficient statistic is calculated based on the learning data read from the learning data recording unit 11, whereas The reinforcement sufficient statistic is different in that it is calculated based on the reinforcement learning data read from the reinforcement learning data recording unit 23. The method for calculating the enhanced sufficient statistics is the same as the method for calculating the sufficient statistics described in step S1.

と表現される。 <Step S4>
The reinforced model synthesizing unit 26 generates a reinforced acoustic model from the sufficient statistics read from the sufficient statistic recording unit 14 and the weighted reinforced sufficient statistics read from the reinforced sufficient statistic recording unit 25 ( Step S4). The generated enhanced acoustic model is stored in the enhanced acoustic model recording unit 27.
It is assumed that G sufficient statistics are obtained from a plurality of learning data in the process of step S1. At this time, the sufficient statistics are g = 1,.

It is expressed.

と表現される。 Further, it is assumed that H reinforcement sufficient statistics are obtained from a plurality of reinforcement learning data in the process of step S3. At this time, the strengthening sufficient statistics are set as h = 1,.

It is expressed.

At this time, the reinforcement model synthesis unit 26 generates a reinforcement acoustic model by calculating each acoustic parameter using the following equation.

Here, w ^h (h = 1,..., H) is a weight to be applied to the reinforcement sufficient statistics, and is a number of 1 or more. The weights w ^h (h = 1,..., H) may be the same value or different values. For example, the weight applied to the sufficient statistics calculated from the learning data related to the same keyword as the predetermined keyword is larger than the weight applied to the enhanced sufficient statistics calculated from the learning data related to the keyword similar to the predetermined keyword. can do. In this way, an acoustic model with further improved recognition performance can be generated by assigning a greater weight to more reliable data.

≪Modifications etc.≫
As shown by a broken line in FIG. 1, an evaluation unit 41 that evaluates the recognition performance of the generated acoustic model, a weight determination unit 40 that calculates an appropriate weight to be applied to the enhanced sufficient statistics, and development data that approximates the evaluation target data A recorded development data recording unit 42 may be provided to determine an appropriate weight to be applied to the enhanced sufficient statistics.

  The acoustic model generation device 1 generates a reinforced acoustic model based on a plurality of predetermined weights and stores the reinforced acoustic model in the reinforced acoustic model recording unit 27. The evaluation unit 41 checks the recognition accuracy of the development data approximated to the evaluation target data read out from the development data recording unit 42 for each generated enhanced acoustic model, and the weight determination unit 40 uses the evaluation value for the recognition accuracy. Output to. The weight determination unit 40 determines the weight corresponding to the reinforced acoustic model with the best recognition accuracy. That is, the weight determination unit 40 selects a weight used for generating the enhanced acoustic model that has obtained the highest evaluation value, and sets the weight in the enhanced model synthesis unit 26. Information about the weight determined by the weight determination unit 40 is output to the strengthened model synthesis unit 26.

  Moreover, the threshold value determination part 43 can be provided and the appropriate value about the threshold value used by the process of the reinforcement learning data selection part 22 can be determined. A method for determining an appropriate threshold is the same as the method for determining an appropriate weight described above. That is, for each of the enhanced acoustic models generated based on a plurality of predetermined threshold values, the threshold value used to check the recognition accuracy of the development data and generate the enhanced acoustic model with the best recognition accuracy. Is set as a threshold used in the reinforcement learning data selection unit 22. In FIG. 1, the asterisk “*” output from the threshold determination unit 43 and input to the reinforcement learning data selection unit 22 indicates that the information about the threshold determined by the threshold determination unit 43 is the reinforcement learning data selection unit 22. It means that it is input to.

In addition, the process of step S2 and S3 may be performed before the process of step S1, and the process of step S1 and the process of step S2 and S3 may be performed in parallel.
The processing function of the acoustic model generation device 1 can be realized by a computer. In this case, the content of the processing function of the acoustic model generation device 1 is described by a program. Then, by executing this program on a computer as shown in FIG. 5, the processing function of the acoustic model generation device 1 is realized on the computer.

  The program describing the processing contents can be recorded on a computer-readable recording medium. As the computer-readable recording medium, for example, any recording medium such as a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory may be used. Specifically, for example, as a magnetic recording device, a hard disk device, a flexible disk, a magnetic tape or the like, and as an optical disk, a DVD (Digital Versatile Disc), a DVD-RAM (Random Access Memory), a CD-ROM (Compact Disc Read Only). Memory), CD-R (Recordable) / RW (ReWritable), etc., magneto-optical recording medium, MO (Magneto-Optical disc), etc., semiconductor memory, EEP-ROM (Electronically Erasable and Programmable-Read Only Memory), etc. Can be used.

  The program is distributed by selling, transferring, or lending a portable recording medium such as a DVD or CD-ROM in which the program is recorded. Furthermore, the program may be distributed by storing the program in a storage device of the server computer and transferring the program from the server computer to another computer via a network.

  A computer that executes such a program first stores, for example, a program recorded on a portable recording medium or a program transferred from a server computer in its own storage device. When executing the process, the computer reads a program stored in its own recording medium and executes a process according to the read program. As another execution form of the program, the computer may directly read the program from a portable recording medium and execute processing according to the program, and the program is transferred from the server computer to the computer. Each time, the processing according to the received program may be executed sequentially. Also, the program is not transferred from the server computer to the computer, and the above-described processing is executed by a so-called ASP (Application Service Provider) type service that realizes the processing function only by the execution instruction and result acquisition. It is good. Note that the program in this embodiment includes information that is provided for processing by an electronic computer and that conforms to the program (data that is not a direct command to the computer but has a property that defines the processing of the computer).

In this embodiment, the acoustic model generation apparatus 1 is configured by executing a predetermined program on a computer. However, at least a part of these processing contents may be realized by hardware. .
In addition to the above embodiments, the acoustic model generation device, method, program, and recording medium according to the present invention are not limited to the above-described embodiments, and can be appropriately changed without departing from the spirit of the present invention. It is.

[Experimental example]
From all the learning data 45264 utterances, 2718 utterances containing 6 keywords that were misrecognized from the speech recognition results in the development data are selected, and the weight of the reinforcement sufficient statistic is multiplied by 50 and integrated with the base sufficient statistic ( The keyword reproduction rate was improved from 82.4% to 83.6% (error reduction rate 6.5%) by using the enhanced acoustic model created by merging.

The figure which illustrates the function structure of the acoustic model production | generation apparatus by this invention. The figure which illustrates the process of the acoustic model production | generation apparatus by this invention. The figure which illustrates the function structure of the acoustic model production | generation apparatus by a prior art. The figure which illustrates the process of the acoustic model production | generation apparatus by a prior art. The figure which illustrates the functional structure in the case of making a computer perform the acoustic model production | generation apparatus by this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Acoustic model production | generation apparatus 9 Acoustic model production | generation apparatus 11 Learning data recording part 12 Base acoustic model recording part 13 Learning part 14 Sufficient statistics recording part 15 Model synthetic | combination part 16 Acoustic model recording part 21 Reinforcement keyword list recording part 22 Reinforcement learning data selection Unit 23 Reinforcement learning data recording unit 24 Reinforcement learning unit 25 Reinforcement sufficient statistics recording unit 26 Reinforcement model synthesis unit 27 Reinforcement acoustic model recording unit 31 Phoneme sequence conversion unit 32 Distance calculation unit 40 Determination unit 41 Evaluation unit 42 Development data recording unit 43 Threshold determination unit

Claims (10)

Learning means for calculating sufficient statistics from the learning data read from the learning data recording means and the base acoustic model;
Reinforcement learning data selection means that extracts learning data including the same and / or similar keywords as the predetermined keyword from the learning data recording means, and uses it as reinforcement learning data;
Reinforcement learning means for calculating reinforcement sufficient statistics from the reinforcement learning data and the base acoustic model,
Enhanced model synthesis means for generating a reinforced acoustic model from the sufficient statistic and the weighted reinforced sufficient statistic;
An acoustic model generation device having: The acoustic model generation device according to claim 1,
The reinforcement learning data selection means is:
The predetermined keyword is converted into a phoneme sequence, and the same and / or similar keyword as the predetermined keyword is determined by comparing whether the converted phoneme sequence matches another phoneme sequence for each phoneme. A means for extracting learning data including
An acoustic model generation device characterized by that. The acoustic model generation device according to claim 1,
The reinforcement learning data selection means is:
A means for extracting learning data including a keyword that is the same as and / or similar to the predetermined keyword by obtaining a similar distance between phonemes in phonology for the predetermined keyword and a keyword to be compared;
An acoustic model generation device characterized by that. The acoustic model generation device according to claim 1,
The reinforcement learning data selection means is:
By converting the predetermined keyword into a phoneme sequence and calculating the sum of the distance between distributions for each phoneme in the base acoustic model of the converted phoneme sequence and other phoneme sequences, It is a means for extracting learning data containing similar keywords.
An acoustic model generation device characterized by that. In the acoustic model generation device according to any one of claims 1 to 4,
The same keyword as the predetermined keyword includes keywords that have different ideas but have common names.
An acoustic model generation device characterized by that. In the acoustic model generation device according to any one of claims 1 to 5,
The weight applied to the reinforcement sufficient statistic calculated from the learning data including the same keyword as the predetermined keyword in the reinforcing model synthesizing means is calculated from the learning data including a keyword similar to the predetermined keyword. Is larger than the weight applied to the statistic
An acoustic model generation device characterized by that. In the acoustic model generation device according to any one of claims 1 to 6,
7. The recognition accuracy of the data approximating the evaluation target data is checked for each of the enhanced acoustic models generated by the acoustic model generation device according to claim 1 based on a plurality of weights, and the best recognition accuracy is obtained. A weight determination means for setting the weight corresponding to the enhanced acoustic model obtained as a weight to be applied to the sufficient reinforcement statistic.
An acoustic model generation device characterized by that. A learning step for calculating sufficient statistics from the learning data read from the learning data recording means and the base acoustic model;
Reinforcement learning data selection step of extracting learning data including the same and / or similar keyword as a predetermined keyword from the learning data recording means and using it as reinforcement learning data;
A reinforcement learning step of calculating a reinforcement sufficient statistic from the reinforcement learning data and the base acoustic model;
A reinforcement model synthesis step for generating a reinforcement acoustic model from the sufficient statistics and the weighting of the reinforcement sufficient statistics;
An acoustic model generation method comprising:   The acoustic model generation program for functioning a computer as each means of the acoustic model generation apparatus in any one of Claims 1-7.   A computer-readable recording medium on which the acoustic model generation program according to claim 9 is recorded.

Images