JP4861912B2 - Probability calculation apparatus and computer program for incorporating knowledge sources - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a probability calculating apparatus capable of robustly calculating a probability of a phoneme of a speech signal by using available training data. <P>SOLUTION: The probability calculating apparatus 516 calculates a probability of each of phonemes in a speech signal by using a statistical acoustic model and knowledge sources. The statistical acoustic model and the knowledge sources have a causal dependency represented by a Bayesian network (BN). The BN corresponds to a junction tree including cluster nodes and separator nodes. The apparatus 516 includes: a storage device 520 for local acoustic models R3, C1 and L3; a module for calculating observation data for each of frames; right, center and left context calculating devices 570, 572 and 574 for calculating a local probability of each of the phonemes causing the observation data by using the local acoustic models R3, C1 and L3; and a PDF calculating device 576 for calculating a probability of each of the phonemes as a function of local probabilities. <P>COPYRIGHT: (C)2009,JPO&amp;INPIT

Description

  The present invention relates to probability calculation in speech recognition, and more particularly to probability calculation in speech recognition incorporating one or more knowledge sources.

  Information technology continues to grow and is becoming increasingly influential in many aspects of everyday life. Voice communication between humans and information processing devices such as interactive systems is also becoming increasingly important. One of the basic technologies for realizing a voice-oriented interface is automatic speech recognition (ASR). For the past 40 years, many researchers have been studying the area of ASR. The goal is to develop intelligent information processing devices that can automatically recognize natural spoken words spoken by humans. However, extracting a linguistic message behind a complex acoustic signal is not an easy process because there are many sources of variation in the signal.

  Several approaches have been developed to address this issue. These approaches to ASR are generally classified into two types: “knowledge base” and “corpus base”.

  The former is mainly based on human ability to interpret spectrograms or other visual representations of speech signals and uses knowledge-based rules. However, it is difficult to foresee all cases where these rules depend on each other, so a rule necessarily conflicts with other rules, such as completely contradicting other rules in explaining the same phenomenon. Resulting in.

  In contrast, the latter approach is usually based on the modeling of speech signals using well-defined statistical algorithms that can automatically extract knowledge from the data. This modeling approach gives promising results and shows better performance than the former knowledge-based approach. This is why many of the current ASR systems use statistical data driven methods based on Hidden Markov Models (HMMs). State-of-the-art ASR systems have achieved very high performance under controlled conditions.

  Despite significant advances in this area, there are still many challenges to overcome before the ASR system is widely used in everyday life and fully displays its potential. For example, in the presence of unexpected acoustic changes, ASR systems perform much worse than human listeners. Simply relying on a statistical model and ignoring most of the additional knowledge available can only reach a limited level of success. Many researchers are aware of this problem and have made various attempts to better integrate knowledge bases and statistical approaches.

So far, Non-Patent Document 1 has proposed research that enables the incorporation of acoustic phoneme knowledge sources using neural networks for the purpose of rescoring. The Large-Vocabulary Speech Recognition (LVCSR) system disclosed in Non-Patent Documents 2 and 3 can also improve acoustic models by incorporating long-term simultaneous articulation effects such as quinphone / pentaphone. succeeded in. Some researchers have recently attempted to use graphical tools such as Bayesian Networks (BN). BN can be thought of as a universal HMM, and can easily incorporate additional knowledge such as articulation features, subband correlation, or style of speech in addition to speech spectrum information (non-patented). Reference 4).
JP 2007-052166 A J. et al. Li, Y. Cao, and C.I. -H. Lee, “Knowledge Source Integration for Re-Scoring Candidates in Automatic Speech Recognition,” ICASSP Proceedings, Philadelphia, USA, 2005, pages 837-840 (J. Li, Y. Tsao, and C.-H. Lee, “A study on knowledge source integration for candidate recycling in automatic speech recognition,” in Proc. ICASSP, Philadelphia, USA, 2005, pp. 37, 2008. C. Netty, G. Potamianos, J.A. Rutin, I.D. Matthews, H.C. Grotin, D.C. Vergili, J.A. Sison, A. Mashari and J.H. Shu, "Hearing-Visual Speech Recognition", Technical Report, CSLP John Hopkins University, Baltimore, USA, 2000 (C. Neti, G. Potamianos, J. Luettin, I. Mattews, H. Grotin, D. Vergiri, (J. Sison, A. Mashari, and J. Zhou, “Audio-visual speech recognition,” Tech. Rep., CSLP John Hopkins University, Baltimore, USA, 2000.) A. Laurier, D.C. Hindle, M.M. Riley and R.C. Sprote, “AT & T LVCSR-2000 System”, Speech Transcription Workshop, University of Maryland, USA, 2000 (A. Ljolje, D. Hindle, M. Riley, and R. Sproat, “The AT & T LVCSR-2000 system. , "In Speech Transcribion Workshop, University of Maryland, USA, 2000.) K. Daudi, D.D. Fore and C.I. Antoine, “A New Trial of Multi-Band Speech Recognition Based on Probabilistic Graph Model”, ICSLP Proceedings, Beijing, China, pp. 329-332, 2000 (K. Daoudi, D. Fohr, and C. Antoine, “ A new approach for multi-band speech recognition based on probabilistic graphical models, “in Proc. ICSLP, Beijing, China, pp. 329-332, 2000.) K. Markov and S.M. Nakamura, “Forward-Backward Training of Hybrid HMM / BN Acoustic Model”, ICLSP Proceedings, 621-624, 2006 (K. Markov and S. Nakamura, “Forward-backwards training of hybrid HMM / BN acoustics” , "In Proc. ICSLP, pp. 621-624, 2006.) J. et al. J. et al. Odel, “Use of Context in Large Vocabulary Speech Recognition”, Doctoral Dissertation, University of Cambridge, Cambridge, UK, 1995 (JJ Odell, The Use of Large Vocabulary Speech Recognition, Ph. D. thesis University, Cambridge) , Cambridge, UK, 1995.) Ji. Min, P.M. O. Boyle, M.C. Owens, and F.M. J. et al. Smith, “A Bayesian approach to building a triphone model for continuous speech recognition”, IEEE Speech and Acoustic Processing Transactions, Vol. 7, No. 6, pp. 678-684, November 1999 (Ji. Ming, P O. Boyle, M. Owens, and F. J. Smith, “A Bayesian approach for building triphone models for continous spike recognition,” IE Transport. , November 1999.) S. Sakti, S. Nakamura and K. Markov, “Improvement of acoustic model accuracy by incorporating wide-range phoneme context based on Bayesian framework”, IEICE Information & System Transaction, Volume E89-D, No. 3, pages 946-953, 2006 (S. Sakti, S. Nakamura, and K. Markov, “Improving acoustic model precision by inducing a wide phonetic context based on a Bayesian framework, 94 & E. ) T.A. Gitzhiro, T .; Matsui and S. Nakamura, “Automatic Generation of Non-uniform HMM Topology Based on MDL Criteria”, IEICE Information & System Transactions, E87-D Volume 8, No. 2121-2129, 2004 (T. Jitsuhiro, T. Matsui, and S. J. Nakamura, “Automatic generation of non-uniform HMM topologies based on the MDL criterion,” IEICE Trans. Inf. & Syst., Vol E87-D, no.

  However, it was often not possible to develop such complex models and achieve the best performance. This is especially true when there are insufficient resources to properly train the parameters of the model, that is, when there is insufficient resources such as the amount of training data and available memory space. As a result, the resolution of the input space is lost due to non-robust estimation and an increase in the number of unknown patterns. In addition, decoding using large models is also cumbersome and sometimes impossible. The best way to do this is to select a simple form of model that can be reliably estimated using the available training data.

  Therefore, one of the objects of the present invention is to provide a probability calculation device capable of robustly calculating the probability of phonemes of speech signals using available training data.

  Another object of the present invention is to provide a probability calculation device capable of calculating the probability of phonemes of a speech signal robustly and with high reliability using training data whose data may be sparse. It is.

  A first aspect of the invention uses, for each predefined set of phonemes present in a given segment of a speech signal, a statistical acoustic model for the speech signal and one or more knowledge sources. It relates to a probability calculation device for calculating probabilities. A segment includes multiple frames of an audio signal. The acoustic model and one or more knowledge sources have a causal relationship represented by a Bayesian network. A Bayesian network corresponds to a junction tree that includes multiple cluster nodes and one or more separator nodes. The apparatus includes means for storing a plurality of local acoustic models corresponding to cluster nodes and one or more separator nodes, means for calculating observation data predefined for each of the frames, Using local acoustic models, local probability calculation means for calculating the local probability of generating observation data for each phoneme, and the probability of generating observation data for each phoneme And a probability calculation means for calculating as a predetermined function of the local probability calculated by the statistical probability calculation means.

  The probability of generating observational data for each local phoneme is calculated by a predefined function of local probability. Local probabilities for each phoneme are calculated using multiple local acoustic models. Because local models are smaller than models that incorporate one or more knowledge sources, less computation is required, less training data is required to train the model, and probability calculations are more robust and reliable. The nature becomes high.

  Preferably, the predetermined function is

で定義され，Ｄは観測データであり，Ｍは音響モデルであり，Ｎは正の整数であり，Ｋ_ｉは１以上の知識源であり，ただし，Ｐ（Ｄ｜Ｋ_ｉ，Ｍ）（ｉ＝１〜Ｎ）及びＰ（Ｄ｜Ｍ）は局部的確率計算手段により計算された局部的確率である．

D is observation data, M is an acoustic model, N is a positive integer, K _i is a knowledge source of 1 or more, provided that P (D | K _i , M) (i = 1 to N) and P (D | M) are local probabilities calculated by the local probability calculating means.

  More preferably, model M is a monophone acoustic model and the one or more knowledge sources include a preceding triphone context unit and a subsequent triphone context unit.

  More preferably, the model M is a monophone acoustic model trained with additional knowledge sources, the one or more knowledge sources including a preceding triphone context unit and a subsequent triphone context unit.

  Additional knowledge sources include accent knowledge, or gender knowledge, or both accent knowledge and gender knowledge.

  A second aspect of the present invention, when executed on a computer, causes the computer to perform statistics for the speech signal for each predefined set of phonemes present in a given segment of the speech signal. This invention relates to a computer program that functions as a probability calculation device for calculating probabilities using an acoustic model and one or more knowledge sources. A segment includes multiple frames of an audio signal. The acoustic model and one or more knowledge sources have a causal relationship represented by a Bayesian network. A Bayesian network corresponds to a junction tree that includes multiple cluster nodes and one or more separator nodes. The computer program is for the computer to calculate means for storing a plurality of local acoustic models corresponding to cluster nodes and one or more separator nodes, and to calculate observation data predefined for each of the frames. The local probability calculation means for calculating the local probability of generating the observation data for each phoneme, and the observation data for each of the phonemes using the local acoustic model Let the probability function as a probability calculation means for calculating the probability as a predetermined function of the local probability calculated by the local probability calculation means.

1. Introduction In this section, we discuss the application of the framework proposed in this application to the problem of incorporating it into wide-area phoneme knowledge information, which often involves the difficulties of data sparseness and memory constraints. First, we show how additional knowledge sources are incorporated into the distribution of HMM states. Next, we show how additional knowledge sources can be incorporated into HMM phoneme modeling. Both approaches have been experimentally demonstrated by large vocabulary continuous speech recognition experiments using English speech data containing two types of accents.

  First, a general framework for incorporating additional knowledge sources is described in the next section. Next, Section 3 outlines the conventional HMM acoustic model. Sections 4 and 5 show how this framework can be used to incorporate additional knowledge sources at the HMM state and phoneme model level. This includes application to the problem of embedding global phoneme context information. Details of the experiment, including results and discussion, are given in Section 6. Finally, the conclusion is described in Section 7.

2. General framework for incorporating knowledge sources In a statistical corpus-based approach, given observation data D, model M is trained. One important issue of interest is computing the likelihood P (D | M) that predicts the data that can be expected given specific knowledge about the model.

  The probability density function P (D | M) is a conditional probability table (CPT) (when D is discrete) or a continuous function such as a Gaussian distribution (when D is continuous). ). In this case, the output probability for a given data d and model parameter m is simply calculated as follows:

その後，付加的知識源をこのモデルに組込む必要があると仮定する．ここでは，どのようにして付加的知識源が組込まれるかを考慮することが必要である．この考慮の手順はいくつかのステップを含み，その概略を図１に示す．

Then assume that additional knowledge sources need to be incorporated into the model. Here, it is necessary to consider how additional knowledge sources are incorporated. This consideration procedure includes several steps, the outline of which is shown in Fig. 1.

  Referring to FIG. 1, this procedure defines a step (step 50) in which a causal relationship between an information source, a model, and data is defined using BN, and whether or not BN inference can be performed directly. A step of determining (step 52), a step of directly executing BN inference when it is determined that direct BN inference is possible (step 54), and a case of determining that direct BN inference is impossible The method includes a step (step 56) of decomposing a network related to a set of linked clusters using a junction tree algorithm, which will be described later, and a step 58 of performing inference on the junction tree obtained in step 56.

  The details of the procedure are described below.

A. Defining the causal relationship between information sources Start with a simple case where the causal relationship between D and M is explained using BN. One example of a BN is a BN 70 including a node 72 and a node 74, the outline of which is shown in FIG. Here, the node M72 is a discrete variable indicated by a square node, and the node D74 is a continuous variable indicated by an elliptical node.

  The joint probability function of BN is factored as follows.

ただし，Ｐａ（Ｚ_ｋ）はＢＮ変数Ｚ_ｋの親を示す．このことから，図２（Ａ）より以下の式を得る．

However, Pa (Z _k ) indicates the parent of the BN variable Z _k . From this, the following equation is obtained from Fig. 2 (A).

このため，データに関する知識に基づき，Ｄ，Ｍ及びＫの間の条件に関する依存性を単純に定義し，付加的な知識ＫをＰ（Ｄ，Ｍ）に組込み，同時確率モデルを同様の方法により表す．例えば，Ｄ，Ｍ及びＫの間の条件に関する依存性を，図２（Ｂ）に概略を示すＢＮにより表すことができる．図２（Ｂ）では，ＢＮ８０はノード７２及び７４と，付加的ノードＫ７６とを含む．ここでＢＮ同時確率関数は以下のようになる．

For this reason, based on the knowledge about data, the dependency regarding the condition between D, M, and K is simply defined, the additional knowledge K is incorporated into P (D, M), and the joint probability model is constructed in the same way. To express. For example, the dependency on the condition among D, M and K can be represented by BN as outlined in FIG. In FIG. 2B, the BN 80 includes nodes 72 and 74 and an additional node K76. Here, the BN joint probability function is as follows.

さらに詳細な例を考える．ここまで，Ｋ_１，Ｋ_２，…，Ｋ_Ｎ知識源があると仮定していた．ここでは，これらすべてが条件に関する依存性が無いと仮定している．図３に，Ｄ，Ｍ及びＫ_１，Ｋ_２，…，Ｋ_Ｎの間の条件に関する依存性の構造の２つの例の概要を示す．

Consider a more detailed example. So far we have assumed that there are K ₁ , K ₂ ,..., K _N knowledge sources. Here, it is assumed that all of these have no dependency on conditions. FIG. 3 outlines two examples of dependency structures for the conditions between D, M and K ₁ , K ₂ ,..., K _N.

3A, a network 90 includes nodes 72 and 74, and nodes 92, 94,..., 96 (nodes K _{1 to} K _N ). Nodes K _{1 to} K _N have a parent node 72 and a child node 74. The network 100 shown in FIG. 3B includes nodes 72 and 74 and nodes 92,..., And 96 (nodes K _{1 to} K _N ). Of the nodes K _{1 to} K _N , the nodes 92 and 96 have only child nodes 74, while the other nodes have a parent node 72 and child nodes 74.

  For this reason, the joint probability density function for BN shown in FIG.

もし，図３（Ｂ）（Ｋ_１及びＫ_Ｎを参照）に示すように，Ｍからの因果関係の影響を何ら受けないあるＫ_ｉがある場合，同時確率密度関数は以下の式で示されるようになる．

As shown in FIG. 3B (see K ₁ and K _N ), if there is a certain K _i that is not affected by any causal relationship from M, the joint probability density function is given by It becomes like this.

ここで分かるように，条件に関する独立性の仮定が異なると，確率関数の分解の仕方も異なってくる（式（５）及び（６）を参照）．

As can be seen here, different assumptions of independence with respect to conditions result in different ways of decomposing probability functions (see equations (5) and (6)).

B. Direct inference in Bayesian networks The most important concern in inference is to calculate the conditional probability P (D | K ₁ ,..., K _N , M) in the global. When the form of this probability density function allows direct BN inference, the following two cases can be considered.

  1) All variables can be observed.

  In this case, the probability density function is simply calculated by equation (1).

２）付加的な知識源Ｋ_１，…，Ｋ_Ｎのような，いくつかの変数が観測できないか，または隠されている．

2) Some variables such as additional knowledge sources K ₁ ,..., K _N are not observable or hidden.

In this case, the probability density function and formula (5), all _{K i} of for all possible _{K i: k i1, k i2} , ..., is calculated by the marginalization about _{k iM.}

ただし単純化のために，＜Ｄ＝ｄ＞，＜Ｍ＝ｍ＞，及び＜Ｋ_ｉ＝ｋ_ｉｊ＞の変わりに，ｄ，ｍ，及びｋ_ｉｊを用いている．

However, for simplicity, d, m, and k _ij are used _instead of <D = d>, <M = m>, and <K _i = k _ij >.

However, the calculation of the overall conditional probability P (D | K ₁ ,..., K _N , M) may not be straightforward due to too many variables and / or computational complexity. In such cases, it is necessary to break the directed graph into a cluster of variables so that appropriate calculations can be performed on them. This can be done with the junction tree algorithm described in the next subsection.

C. Consider the simple case of only the incorporation of two additional knowledge source of the junction tree decomposition K ₁ and K _2. The causal relationship between D, M, K ₁ and K ₂ is indicated by BN110 shown in FIG. BN 110 includes nodes 112, 114, 116 and 118, denoted by M, D, K ₁ and K ₂ respectively. Here, nodes M, K ₁ and K ₂ are discrete variables indicated by square nodes, and node D is a continuous variable indicated by elliptical nodes.

  Then, the following graph transformation is performed to obtain a junction tree.

  1) Assemble an undirected graph from BN110 by joining parents (adding links between all variable pairs with common children) and removing the link orientation. In the case of FIG. 4A, a link is provided between the nodes 116 and 118. The resulting graph is called a “moral graph”.

  2) To form a graph consisting of triangles (triangular graph), an arc is selectively added to the moral graph. If there is no "cordless cycle", the graph is said to be triangulated. A chord is an edge that connects two non-contiguous vertices in a cycle with a length greater than 3.

  3) Form a subset including Pa (A) ∪A for all variables A with Pa (A) ≠ 0 in the triangulate graph. This is called a cluster or clique.

  4) Build a junction tree with clusters / creeks as nodes. In this case, each link between two cliques is labeled with a non-empty common set of separators between these cliques.

FIG. 4B shows an outline of the moral and triangulated graph 130 corresponding to the BN 110 shown in FIG. Graph 130 includes an additional link 120 between nodes 116 and 118. However, from this triangulated graph, only one cluster / clique consisting of the entire set of D, M, K ₁ and K ₂ variables can be obtained, and no further decomposition is possible. Fortunately, since K ₁ and K ₂ are assumed to be independent, by reversing some arrows, BN 140 shown in FIG. 4C equivalent to BN 110 can be obtained. This is possible because P (X, Y) can be decomposed into P (X | Y) P (Y) and P (Y | X) P (X), and the two are equivalent to each other.

FIG. 4D shows an outline of the moral and triangulate graph 150 corresponding to the BN 140. As a result, a cluster / clique can be identified, and a junction tree 160 whose outline is shown in FIG. 4 (E) can be obtained. Here, the cluster set is represented by ellipse nodes 164 and 166, and the separator set is represented by a square node 162.

  From the above, the BN joint probability distribution is defined as follows by dividing the product of all cluster potentials (probabilities) by the product of the separator potentials.

ただし，Ｕはグラフにおける全ての変数を示す「世界」を，φ_Ｃｉはクラスタポテンシャル（クラスタＣｉにおける確率）を，φ_Ｓｉはセパレータポテンシャル（セパレータＳｉにおける確率）を示す．このため，同時確率関数，Ｐ（Ｄ，Ｋ_１，Ｋ_２，Ｍ）は図４（Ｅ）によれば以下のようになる．

However, U represents the “world” indicating all variables in the graph, φ _Ci represents the cluster potential (probability in cluster Ci), and φ _Si represents the separator potential (probability in separator Si). Therefore, the joint probability function, P (D, K ₁ , K ₂ , M) is as follows according to FIG.

ただし，Ｐ（Ｄ，Ｋ_１，Ｍ）とＰ（Ｄ，Ｋ_２，Ｍ）とはクラスタポテンシャルであり，Ｐ（Ｄ，Ｍ）はセパレータポテンシャルである．

However, P (D, K ₁ , M) and P (D, K ₂ , M) are cluster potentials, and P (D, M) is a separator potential.

Based on similar assumptions and considerations, a BN topology similar to BN90 shown in FIG. 3 (A) can be written as in FIG. 5 (A). FIG. 5B shows a junction tree 180 corresponding to this. In FIG. 5B, N clusters 164, 166, ... 170 of variables {(D, K1, M), (D, K2, M), ... (D, KN, M)}, and N-1 There are separators {D, M} (nodes 162, 168, etc.). Therefore, the joint probability function obtained by equation (5) can be decomposed by the following equation.

これは，同時確率関数Ｐ（Ｄ，Ｋ_１，…，Ｋ_Ｎ，Ｍ）を，ある付加的な知識Ｋ_１，Ｋ_２，…，Ｋ_Ｎが与えられた場合の観測データＤの確率に対応するいくつかの局部的な同時確率関数Ｐ（Ｄ，Ｋ_１，Ｍ），…，Ｐ（Ｄ，Ｋ_Ｎ，Ｍ）を合成したものとして表す新しい表記方法を示す．

This joint probability function _{P (D, K 1, ...} , K N, M) , and some additional knowledge _{K 1,} K 2, ..., corresponds to the probability of the observed data D _{when K N} is given A new notation is shown that represents a combination of several local joint probability functions P (D, K ₁ , M), ..., P (D, K _N , M).

D. Junction Tree Inference By using chain rules, we obtain the following equations for all P (D, K _i , M).

このため，式（１１）は以下のようになる．

Therefore, Equation (11) is as follows.

この式（１１）を式（５）と比較すると，

When this equation (11) is compared with equation (5),

であることが分かり，これは，Ｐ（Ｄ｜Ｋ_１，…，Ｋ_Ｎ，Ｍ）が，特定の付加的な知識Ｋ_１，Ｋ_２，…，Ｋ_Ｎが与えられた場合の観測データＤの確率に対応する別々の項に分解可能であることを示す．

It know it is, this _{is, P (D | K 1,} ..., K N, M) is, certain additional knowledge _{K 1,} K 2, ..., observed data D _{when K N} is given This shows that it can be decomposed into separate terms corresponding to the probabilities of.

Defining, estimating, and observing some simple P (D | K _i , M) is one but complicated P (D | K ₁ ,..., K _N , M ) Is much easier.

Therefore, the output probability in the inference for the case where data d, model parameter m, and additional knowledge sources k _1j ,..., K _Nj are given is calculated as follows.

３．従来のＨＭＭ音響モデル
従来のＨＭＭに関連して，いくつかの表記を定義する．トライフォンコンテキスト／ａ⁻，ａ，ａ^＋／のＨＭＭ音声モデルをλ，ＨＭＭ状態変数をＱと表記する．Ｘは観測変数であり，Ｘ_ｓ＝Ｘ_ｔ，…，Ｘ_ｔ＋ｍは長さｍの観測データセグメントである．図６に，標準的なＨＭＭ１９０の構造の概要を示す．ここでは，
１）短時間スペクトル特性はガウス分布２１０，２１２，及び２１４の混合によりモデル化される．

3. Conventional HMM Acoustic Model Several notations are defined in relation to the conventional HMM. The HMM speech model of triphone context / a ⁻ , a, a ⁺ / is denoted by λ, and the HMM state variable is denoted by Q. X is an observation variable, and X _s = X _t ,..., X _{t + m} is an observation data segment of length m. Figure 6 outlines the structure of a standard HMM 190. here,
1) Short-time spectral characteristics are modeled by a mixture of Gaussian distributions 210, 212, and 214.

  2) Temporal speech features are dominated by HMM state transitions 216, 218, 220, 222 and 224 between states 200, 202 and 204.

The HMM state output probability p (x _t | q _i ) is normally calculated from the state probability density function (PDF) P (X | Q) by the following equation.

ただし，ｂ_ｍは状態ｑ_ｉのｍ番目の混合分布の混合重みであり，Ｎ（・）は平均ベクトルμ_ｍと共分散行列Σ_ｍとを持つガウス関数である．ＨＭＭセグメントの尤度Ｐ（Ｘ_ｓ｜λ）は，観測結果と状態シーケンスとの同時確率を，全ての状態シーケンスに対してとることにより（合計尤度），又は最も確からしい状態シーケンスのみに対してとることにより（Ｖｉｔｅｒｂｉ Ｐａｔｈ），計算される．

Here, b _m is a mixture weight of the m-th mixture distribution of the state q _i , and N (•) is a Gaussian function having an average vector μ _m and a covariance matrix Σ _m . The likelihood P (X _s | λ) of the HMM segment is obtained by taking the joint probability of the observation result and the state sequence for all the state sequences (total likelihood) or only for the most probable state sequence. It is calculated by taking (Viterbi Path).

4). Incorporating knowledge sources at the HMM state level A. General Consideration Model M is a given triphone HMM state Q, and D is an observed variable X that follows the theoretical framework described in Section 2.

1) Definition of causal relationship The structure of this topology is the same as that shown in FIG. 2 (A), and the triphone HMM state PDF is represented here by the same BN joint probability function as in equation (3).

単純に式（５）に従えば，以下のようになる．

If you simply follow equation (5), it will be as follows.

これにより，追加の知識源Ｋ_１，Ｋ_２，…，Ｋ_ＮをＨＭＭ状態分類Ｐ（Ｘ，Ｑ）に組込む（すべてのＫ_１，Ｋ_２，…，Ｋ_Ｎが独立した所定のＱであると仮定されている．）．

This incorporates additional knowledge sources K ₁ , K ₂ ,..., K _N into the HMM state classification P (X, Q) (all K ₁ , K ₂ ,..., K _N are independent predetermined Qs). Is assumed.).

2) Reasoning The main concern is the calculation of the HMM state output probability P (X | K ₁ ,..., K _N , Q), which can be easily modeled by a Gaussian function. For this reason, the state output can be obtained directly. Assuming that all additional knowledge sources K ₁ ,..., K _N are hidden as shown in section 2-B, the state output probabilities are all for 1 ≦ i ≦ N. The possible K _i : k _i1 , k _i2 ,..., K _iM can be obtained in the same way as in Eq.

ここで，ｐ（ｋ_ｉ１｜ｑ_ｔ）…ｐ（ｋ_Ｎｊ｜ｑ_ｔ）の項を，ガウス成分ｐ（ｘ_ｔ｜ｋ_ｉ１，…，ｋ_Ｎｊ，ｑ_ｔ）の混合重み係数として扱えば，式（１９）もまた，式（１６）の従来のＨＭＭの状態出力確率と等価であることが分かる．式（１９）はガウス混合分布を表すので，ＨＭＭを基にした既存のデコーダを，何らかの修正をする必要なく用いて認識を行なうことができる．さらに，ＢＮは状態出力の尤度を推論するために使用されるのみであるので，ＨＭＭを基にしたトライフォン音響モデルのトポロジをそのまま維持し，ＨＭＭ状態遷移が依然として時間的な音声特性により支配されるようにできる．このアプローチはまた，ハイブリッドＨＭＭ／ＢＮモデル化フレームワークとして知られ，非特許文献５に記載されている．以後，状態レベルで付加的知識を組込んで得られるモデルを，ＨＭＭ／ＢＮモデルと呼ぶ．

Here, if the term of p (k _i1 | q _t )... P (k _Nj | q _t ) is treated as a mixture weight coefficient of the Gaussian component p (x _t | k _i1 ,..., K _Nj , q _t ), Equation (19) is also equivalent to the state output probability of the conventional HMM of Equation (16). Since Equation (19) represents a Gaussian mixture distribution, recognition can be performed using an existing decoder based on the HMM without any modification. Furthermore, since BN is only used to infer the likelihood of state output, the topology of the triphone acoustic model based on HMM is maintained as it is, and the HMM state transition is still governed by temporal speech characteristics. Can be done. This approach is also known as a hybrid HMM / BN modeling framework and is described in [5]. Hereinafter, a model obtained by incorporating additional knowledge at the state level is called an HMM / BN model.

  Parameter learning of this model can be adopted from normal training of the HMM / BN model described in Non-Patent Document 5. This is based on a backward-forward algorithm. In this algorithm, each training iteration consists of BN training and updating of the HMM transition probability. BN training is done using standard statistical methods. Maximum likelihood (ML) parameter estimation is applied if all variables are observable during training, and if some variables are hidden, the parameters are standard expectation-maximization (EM) algorithms Is estimated.

B. Incorporating global phoneme context information The most widely used acoustic unit in an ASR system is still a triphone that still contains the last preceding phoneme context and the following phoneme context. Triphones have been found to be an effective choice, but wide-range phoneme context is considered more appropriate to capture simultaneous articulation effects over longer periods. However, wide-area phoneme context has problems of data sparseness and memory constraints.

  Here we explain how to apply the framework described in the previous section to the problem of incorporating wide-area phoneme knowledge information.

Conventional, triphone context ^{/ a -, a, a +} / a HMM is, the ^{λ, / a -, a -} , a, a +, it is necessary to extend the ^{a ++} / penta von context such as Suppose. Therefore, based on this approach, by inserting two variables in BN, the two previous and subsequent contexts, C _L (/ a ⁻⁻ ) and C _R (/ a ⁺⁺ /), are assigned to the triphone state PDF. Incorporate.

A triphone HMM state Q, the observed data X, and dependence for the two conditions between additional variables C _L and C _R are described by BN topology outlined in Figure 7. This is called a BN-C topology.

Referring to FIG. 7, Bayesian network 240 includes nodes 250, 252, 254 and 256, these Q, X, by _{C L,} and _{C R,} respectively shown. The node _CL represents the previous context (/ a ⁻⁻ ), and the node _CR represents the second subsequent context (/ a ⁺⁺ /).

  The HMM state PDF is currently indicated by the BN joint probability. This is decomposed as follows according to equation (18).

ただし，Ｘは２つ前のコンテキストＣ_Ｌ及び２つ後のコンテキストＣ_Ｒの両方に依存する．Ｘは連続の変数であり，Ｃ_Ｌ，Ｃ_Ｒ及びＱは離散的変数であるので，Ｐ（Ｘ｜Ｃ_Ｌ，Ｃ_Ｒ，Ｑ）はガウス関数でモデル化され，各々のＰ（Ｃ_Ｌ｜Ｑ）又はＰ（Ｃ_Ｒ｜Ｑ）はＣＰＴにより表される．

However, X is dependent on both the previous two contexts C _L and two after the context C _R. Since X is a continuous variable, and C _L , C _R and Q are discrete variables, P (X | C _L , C _R , Q) is modeled by a Gaussian function, and each P (C _L | Q) or P (C _R | Q) is represented by CPT.

The state output probability can be obtained from P (X | C _L , C _R , Q). Additional context variable C _L and C _R are, assuming that the formula can not be obtained at the time of recognition as (19) (hidden),

となり，ｐ（ｃ_ｌ｜ｑ_ｉ）ｐ（ｃ_ｒ｜ｑ_ｉ）の項を，ガウス成分の混合重み係数ｐ（ｘ_ｔ｜ｃ_ｌ，ｃ_ｒ，ｑ_ｉ）として扱えば，式（１９）は式（１６）の従来のＨＭＭの状態出力確率と等価である．したがって，ここで，ガウスＰＤＦはｃｌ，ｃｒ及びｑｉの全ての組合せに対しトレーニングされる．

The term | _{(q i} c r), mixing weight coefficients of the Gaussian components _p | _{next, p (c l q i)} p (x t | c l, c r, q i) be handled as the formula (19) Is equivalent to the state output probability of the conventional HMM in Eq. (16). Thus, here, the Gaussian PDF is trained for all combinations of cl, cr and qi.

The Pentaphone BN can also be extended with other additional variables such as sex information or accent information using this framework. FIG. 8 shows an example of a dependency structure regarding conditions between the triphone HMM state Q, the observation data X, the two additional variables _CL and _CR, and the variable G and / or the accent variable A regarding sex. Here are some.

  The BN topology is indicated by reference numeral 270 in FIG. 8A by extending BN-C with an additional variable G related to gender indicated by node 272. This is called BN-CG. When the BN-C is extended using the additional accent variable A indicated by the node 292, the BN topology is indicated by the reference number 290 in FIG. 8B, which is called BN-CA. The BN topology 310 of FIG. 8C is extended using both accent and gender variables indicated by nodes 292 and 272, respectively, and is called BN-CGA.

  The HMM state PDF for the BN-CGA example (see FIG. 8C) is expressed as follows.

ただし，Ｘは，アクセントＡ，性別Ｇ，２つ前のコンテキストＣ_Ｌ，及び２つ後のコンテキストＣ_Ｒに依存する．この状態出力確率はまた，式（２１）と同様の方法によりＰ（Ｘ｜Ｃ_Ｌ，Ｃ_Ｒ，Ｑ，Ａ，Ｇ）から得ることができる．

However, X is accented A, depends sex G, two previous context _{C L,} and two after the context _{C R.} This state output probability can also be obtained from P (X | C _L , C _R , Q, A, G) in the same manner as in equation (21).

ここで，ｐ（ａ）ｐ（ｇ）ｐ（ｃ_ｌ｜ｑ_ｉ）ｐ（ｃ_ｒ｜ｑ_ｉ）の項を，ガウス成分の混合重み係数ｐ（ｘ｜ｃ_ｌ，ｃ_ｒ，ｑ_ｉ，ａ，ｇ）として扱えば，各ガウスＰＤＦはｃ_ｌ，ｃ_ｒ，ｑ_ｉ，ａ，及びｇの各組合せに対しトレーニングされる．

Here, the term of p (a) p (g) p (c ₁ | q _i ) p (c _r | q _i ) is used as the Gaussian component mixing weighting coefficient p (x | c ₁ , c _r , q _i a, be handled as g), each Gaussian PDF is _{c l, c r, q i} , is trained for each combination of a, and g.

Both notations (Equations (21) and (23)) show a mixture of Gaussian distributions used in the standard triphone HMM acoustic model. Therefore, recognition can be performed using an existing HMM-based decoder without any modification. Parameter learning of the provided model is performed as described in the previous section. Since all variables including the triphone state Q, the accent A, the sex G, the second previous context (C _L ), the second subsequent context (C _R ), and the variable X are observable in training, ML Parameter estimation is used.

If the amount of training data is insufficient to reliably estimate all model parameters, the number of parameters can be reduced by clustering techniques such as knowledge-based or data-driven clustering. For example, for each value _c l / _{c r} phoneme context _C L / _{C R} after two previous / by the equation (21) and (23), the corresponding Gaussian components are present.

9 for BN330 only _{C R} is added, an overview of the observation space 344. _{C R} in FIG. 9 is indicated by node 342, has various values / b / a after two contexts, / p /, ..., / z / a. The various values of this variable correspond to various Gaussian distributions 350, 352,. If a set of 44 phonemes (including silence) is used for English ASR, the phoneme context C before / after 2 may have 44 values (C = c ₁ , c ₂ ,..., C ₄₄ ). It means that there is sex. Therefore, the total number of Gaussian distributions for each state of the BN-C topology (see FIG. 7) can be 44 ² = 1936. The topology of BN-CG, BN-CA and BN-CGA will be even more. If the amount of training data is insufficient to reliably estimate the increased model parameters, the overall performance will be significantly reduced. For this reason, it is preferable to reduce the number of Gaussian distributions. There are two methods that can be used to reduce the number of Gaussian distributions. One is to use knowledge-based phoneme classes. The other is data-driven clustering. These methods can be applied to any Bayesian network.

  Here, phoneme contexts are classified based on the main differences in articulation, and the parameter size is reduced. Table 1 gives examples of knowledge-based phoneme classes taken from Non-Patent Document 6.

ＨＭＭ／ＢＮアプローチに基づくペンタフォンの可能性についての，さらなる詳細及び議論は特許文献１に示されている．

Further details and discussion on the possibility of a pentaphone based on the HMM / BN approach is given in US Pat.

5. Incorporating knowledge sources at the phoneme model level A. General Consideration According to the theoretical framework described in Section 2, model M is again the HMM phoneme model λ and D is the segment X _s .

1) Definition of causal relationship The structure of the topology is the same as that shown in Fig. 2 (A), and the probability function of the HMM phoneme unit is represented by a BN joint probability function similar to equation (3) this time.

追加の知識源Ｋ_１，Ｋ_２，…，Ｋ_ＮをＨＭＭ音素モデルＰ（Ｘ_ｓ，λ）に組込むためには（所与のλに対し，全てのＫ_１，Ｋ_２，…，Ｋ_Ｎが独立と仮定する．），簡易に式（５）に従い，次の式を得る．

To incorporate additional knowledge sources K ₁ , K ₂ ,..., K _N into the HMM phoneme model P (X _s , λ) (for a given λ, all K ₁ , K ₂ ,..., K _N ), Simply follow equation (5) to obtain the following equation.

２）推論
ここでの最大の関心事は，与えられた入力セグメントＸ_ｓに対するＰ（Ｘ_ｓ｜Ｋ_１，…，Ｋ_ｎ，λ）を計算することである．しかし，条件付ＰＤＦに対する単純な形式の関数を得るのは困難である．なぜなら，この式には，持続時間が変化するＨＭＭモデルλ，及びセグメントＸ_ｓが関係しているからである．このためここで，セクション２−Ｃで述べたジャンクションツリーアルゴリズムにより，Ｐ（Ｘ_ｓ｜Ｋ_１，…，Ｋ_Ｎ，λ）を分解する必要がある．これは式（１４）に従い以下のように分解される．

2) Inference The biggest concern here is to calculate P (X _s | K ₁ ,..., K _n , λ) for a given input segment X _s . However, it is difficult to obtain a simple form of function for conditional PDF. This is because the HMM model λ whose duration changes and the segment X _s are related to this equation. For this reason, it is necessary to decompose P (X _s | K ₁ ,..., K _N , λ) by the junction tree algorithm described in section 2-C. This is decomposed as follows according to equation (14).

この式は，いくつかの，より複雑さの少ない依存関係，すなわち，特定の追加の知識Ｋ_１，Ｋ_２，…，Ｋ_Ｎが与えられた場合のセグメント観測データＸｓの尤度に対応するＰ（Ｘ_ｓ｜Ｋ_１，λ），…，Ｐ（Ｘ_ｓ｜Ｋ_Ｎ，λ）によって，音素のＨＭＭ尤度Ｐ（Ｘ_ｓ｜Ｋ_１，Ｋ_２，…，Ｋ_Ｎ，λ）を表す新しい方法である．

This equation represents some less complex dependencies, ie P corresponding to the likelihood of the segment observation data Xs given certain additional knowledge K ₁ , K ₂ ,..., K _N. (X _s | K ₁ , λ),..., P (X _s | K _N , λ) represents a phoneme HMM likelihood P (X _s | K ₁ , K ₂ ,..., K _N , λ) It is a method.

B. Incorporation of wide-area phoneme context information We will apply the approach described in the previous section to the same problem as in the case of incorporating wide-area phoneme knowledge information. In incorporation of the wide area phoneme knowledge information, triphone context ^{/ a -, a, a +} / a, penta von context ^{/ a -, a -, a} , a +, extended to ^{a ++} /. Structurally, the conventional triphone context unit model of the HMM is described as a model 370 shown in FIG. 10A, and the pentaphone context unit model is described as a model 372 shown in FIG. 10B.

The two previous contexts C _L / a ⁻⁻ and the second subsequent context / a ⁺⁺ / are added to the probability function P (X _s | λ). X _s, lambda, dependence on a condition of _{C L} and _{C R} is described by BN similar to that shown in FIG. 4 (A). The junction tree finally obtained by decomposition is also the same as that shown in FIG. Figure 4 is a HMM phoneme model λ of M is now in (E), D is a segment _{X s.} From this, the conditional probability function is defined as follows according to equation (26).

λが，トライフォン／ａ⁻，ａ，ａ^＋／，２つ前のコンテキストＣ_Ｌ／ａ⁻⁻／，及び，２つ後のコンテキストＣ_Ｒ／ａ^＋＋／と関連付けられていることから，以下のように書ける．

Since λ is associated with the triphones / a ⁻ , a, a ⁺ /, the second previous context C _L / a ⁻ /, and the second subsequent context C _R / a ^{+ +} / Can be written as

この式（２８）は以下のようになる．

This equation (28) is as follows.

これはペンタフォンモデルが，ｐ（Ｘｓ｜［ａ⁻⁻，ａ⁻，ａ，ａ^＋］），ｐ（Ｘｓ｜［ａ⁻，ａ，ａ^＋，ａ^＋＋］），及びｐ（Ｘｓ｜［ａ⁻，ａ，ａ^＋］）により構成できることを示す．これら構成要素は，左／先行テトラフォンコンテキスト，右／後続テトラフォンコンテキスト，及び中央トライフォンコンテキストというユニットが与えられた場合の，セグメントＸｓの尤度に対応する．

This is because the pentaphone model is represented by p (Xs | [a ⁻⁻ , a ⁻ , a, a ⁺ ]), p (Xs | [a ⁻ , a, a ⁺ , a ⁺⁺ ]), and p (Xs | [ a ^-, a, show that can be configured by ^a +]). These components correspond to the likelihood of segment Xs given the units left / preceding tetraphone context, right / following tetraphone context, and central triphone context.

However, it is also difficult to create a tetraphone model for [a ⁻⁻ , a ⁻ , a, a ⁺ ], [a ⁻ , a, a ⁺ , a ⁺⁺ ] because the data exists only sparsely. is there.

Alternatively, using equation (28), lambda is monophones / a / a as shown, and two before and after the context _{C L} and _{C R} and the ^{/ a -, a} - ^/ and / ^{a +} , A ⁺⁺ / respectively. As a result, the following equation is obtained.

この式は，ペンタフォンコンテキスト／ａ⁻⁻，ａ⁻，ａ，ａ^＋，ａ^＋＋／が，ｐ（Ｘｓ｜［ａ⁻⁻，ａ⁻，ａ，］），ｐ（Ｘｓ｜［ａ，ａ^＋，ａ^＋＋］），及びｐ（Ｘｓ｜［ａ］）により構成されることを示し，これら構成要素は，左／先行テトラフォンコンテキスト（Ｌ３），右／先行テトラフォンコンテキスト（Ｒ３），及び中央トライフォンコンテキスト（Ｃ１）のユニットが与えられたときの，観測データＸｓの尤度に対応する．この構成をＣ１Ｌ３Ｒ３と呼び，その構造を図１０（Ｃ）に示す．

This formula indicates that the pentaphone context / a ⁻⁻ , a ⁻ , a, a ⁺ , a ⁺⁺ / is p (Xs | [a ⁻⁻ , a ⁻ , a,]), p (Xs | [a, a ⁺ , A ⁺⁺ ]), and p (Xs | [a]), these components are left / preceding tetraphone context (L3), right / preceding tetraphone context (R3), and Corresponds to the likelihood of the observed data Xs when the unit of the central triphone context (C1) is given. This configuration is called C1L3R3, and its structure is shown in FIG.

  Referring to FIG. 10C, Bayesian pentaphone context unit C1L3R3 374 includes left / preceding triphone context unit (L3) 380, right / following triphone context unit (R3) 382, and monophone unit (C1). (Not shown).

As can be seen in this figure, the number of context units to be estimated is reduced from N ⁵ to (2N ³ + N) without impairing the range covered by the context. N is the number of phonemes. If a set of 44 phonemes is used for the English ASR, the total number of contexts that need to be estimated with the pentaphone model is 44 ⁵ ≈165,000,000 context units. In a configuration using a triphone context unit, this complexity is reduced to about 170,000 units.

Analysis of equations (29) and (30) shows that equation (27) can also be used as a starting point for deriving other configurations of the HMM phoneme model. λ is monophone unit / a /, C _L and C _R are each context unit / a ^- preceding / and / a + ^/ a, and assuming that a subsequent context units, in Non-Patent Document 7 A factorization similar to that proposed is obtained. This is known as a Bayesian triphone.

ここでは，トライフォンモデルがモノフォン及びバイフォンモデルから構築されている．以後，同様の方法で構成された全てのモデルも，ベイズモデルと呼ばれる．

Here, the triphone model is built from the monophone and biphone models. Hereinafter, all models constructed in the same way are also called Bayesian models.

  An extension of the Bayesian triphone, called the Bayesian wide-area phoneme context model, is also described in Non-Patent Document 8, which is the previous research paper of the present inventor. With this approach, it is possible to model a wide-range phoneme context from a model with less dependency on the context, simply based on Bayes' law. However, difficulties arise when it is necessary to incorporate various types of knowledge sources.

  In contrast, the unified framework here gives us a better way to incorporate different types of knowledge sources. For example, C1L3R3 can be further extended with other additional knowledge variables such as gender or accent information. C1L3R3 can be extended with gender information alone (C1L3R3-G), accent information alone (C1L3R3-A), or both gender and accent information (C1L3R3-AG).

In the case of C1L3R3-AG, the BN topology, the moral and triangulated graphs, and the corresponding junction tree are as shown in FIG. Referring to FIG. 11A, the BN topology 400 includes nodes 410, 412, 414, 416, 418, and 420, indicated by λ, X _s , C _L , C _R , G, and A, respectively. Referring to FIG. 11B, a moral and triangulate graph 430 corresponding to the BN topology 400 includes nodes 410, 412, 414, 416, 418 and 420, nodes 418 and 420, nodes 410 and 418, and Includes three additional links 422, 424, and 426 connecting nodes 410 and 420, respectively. Referring to FIG. 11C, junction trees 450 corresponding to the graph of FIG. 11B are indicated by “X _s λAG”, “X _s C _L λ”, and “X _s C _R λ”, respectively. is includes a cluster node 460 and 464, and 474, _{"X s} λ", and indicated _{"X s} λ" respectively, and a separator nodes 462 and 472.

  In this case, the conditional probability function is obtained as follows.

したがって，λ，Ｃ_Ｌ及びＣ_Ｒに対するＣ１Ｌ３Ｒ３の設定に従えば，Ｃ１Ｌ３Ｒ３−ＡＧのペンタフォン尤度は以下のようになる．

Thus, lambda, according to the setting of C1L3R3 for _{C L} and _{C R,} penta von likelihood of C1L3R3-AG is as follows.

これは，Ｐ（Ｘｓ｜［ａ⁻⁻，ａ⁻，ａ，ａ^＋，ａ^＋＋］，Ａ，Ｇ）を，Ｐ（Ｘｓ｜［ａ］，Ａ，Ｇ），Ｐ（Ｘｓ｜［ａ⁻⁻，ａ⁻，ａ］，Ａ，Ｇ），及びＰ（Ｘｓ｜［ａ，ａ^＋，ａ^＋＋］，Ａ，Ｇ）に因数分解することにより，単純化できることを示している．

This means that P (Xs | [a ⁻⁻ , a ⁻ , a, a ⁺ , a ⁺⁺ ], A, G) is changed to P (Xs | [a], A, G), P (Xs | [a ^{− -, a -, a],} a, G), and ^{P (Xs | [a, a} +, a ++], a, by factoring in G), and show that it is possible simplified.

  In order to realize the ASR system with the proposed pentaphone model, a special decoder that can operate with several models is required. This can be avoided if the proposed pentaphone model is applied to rescoring the N-best list generated by a standard triphone-based HMM system.

  FIG. 12 shows the overall structure of the ASR system 500 according to the first embodiment of the present invention. Referring to FIG. 12, ASR system 500 receives standard waveform 512 for receiving speech waveform data 510, decoding the speech, and outputting an N best list of hypotheses of input speech, and 530, 532, 534. And a model storage device 520 for storing the pentaphone models C1L3R3, C1L3R3-A, C1L3R3-G, and C1L3R3-AG, and models 530, 532, 534, respectively, in response to human operation. And the selector 522 for selecting one of 536 and N-best of the hypothesis from the standard decoder 512 are re-scored using the model selected by the selector 522, and the N-best hypothesis And a hypothesis selection module 516 for outputting one indicating the highest score.

FIG. 13 shows details of the hypothesis selection module 516. Referring to FIG. 13, the hypothesis selection module 516 reads a hypothesis from the memory 550 for storing the N best hypotheses one by one, and subsequent re-scoring the feature parameters of the separated phonemes. A read and supply module 552 for supplying the functional units for the left to right in order, and five shift memories 554, 556, 558, 560 and 562 for receiving these characteristic parameters in the shift memory 554 Including. When the characteristic parameter is shifted shift memory 554,556,558,560 and 562, shift memory 554,556,558,560, and ^562, a ^+, a ++, a, a ^- and ^{a -} relative Each feature parameter is stored.

The hypothesis selection module 516 further uses the R3 model and the feature parameters stored in the shift memories 554, 556, and 558 to calculate a right context for calculating the probability P (Xs | [a, a ⁺ , a ⁺⁺ ]). Using the device 570, the C1 model, and the feature vector stored in the shift memory 558, the central context calculation device 572 for calculating the probability P (Xs | [a]), the L3 model and the shift memories 558, 560 , 562 and the left context calculator 574 for calculating the probability P (Xs | [a ⁻ , a ⁻ , a]) from the memory 550 by the read and supply module 552. For each segmentation of the read hypothesis, the probability P (Xs | [a ⁻⁻ , a ⁻ , a, ⁺ , a ⁺⁺ ]).

  The hypothesis selection module 516 further rescores each hypothesis stored in the memory 550 by multiplying the probabilities of the segments of each hypothesis and re-scores for storing the score in the memory 550 in association with the corresponding hypothesis. A ring module 578, and a sort and selection module 580 for sorting hypotheses in memory 550 in descending order of scores and outputting the hypothesis with the highest score.

  N-best recognition at the word level is performed by standard decoder 512 for all utterances of test data using a conventional HMM acoustic model and standard Viterbi decoding. All N-best hypotheses include an acoustic score for all phonemes, a language model (LM) score, and a Viterbi partition. Then, for each phoneme segment of each hypothesis, re-scoring is performed in the hypothesis selection module 516 using the proposed pentaphone model.

  Referring to FIG. 13, memory 550 stores N best hypotheses. The read and supply module 552 reads the first hypothesis from the memory 550 and outputs phoneme segments (feature parameters) in the hypothesis to the shift memory 554 from left to right (from the head to the end).

Shift memories 554 to 562 shift phoneme segments. For each set of phoneme segments stored in the shift memories 554, 556 and 558, the right context calculator 570 uses the R3 model to calculate the probability P (Xs | [a, a ⁺ , a ⁺⁺ ]). . For each phoneme segment stored in the shift memory 558, the central context calculator 572 calculates the probability P (Xs | [a]) using the C1 model. For each set of phoneme segments stored in shift memories 558, 560, and 562, left context calculator 574 calculates probability P (Xs | [a ⁻ , a ⁻ , a]) using the L3 model. Do it. The calculated probability is given to the PDF calculator 576. The PDF calculator 576 calculates the pentaphone context probability P (Xs | [a ⁻⁻ , a ⁻ , a, ⁺ , a ⁺⁺ ]) according to the equation (30), and gives the probability to the re-scoring module 578. .

  For each phoneme segment stored in the shift memory 558, the read and supply module 552 informs the rescoring module 578 when to read the output of the PDF calculator 576. In response, rescoring module 578 reads the output of PDF calculator 576 and stores the value. At the end of the hypothesis, the read and supply module 552 signals the rescoring module 578. In response, the rescoring module 578 calculates the hypothesis score by multiplying the probabilities of all the phoneme segments in the provision. When the calculation is completed, the rescoring module 578 stores the score (pentaphone score) in the memory 550 in association with the hypothesis to be processed.

  Once the pentaphone score has been calculated for all hypotheses stored in memory 550, the read and feed module signals the sort and select module 580. In response, the sort and select module 580 reads all hypotheses stored in the memory 550 along with the corresponding pentaphone and LM score, combines the pentaphone and LM score into a new score, and the new The hypotheses are rearranged in descending order of the scores, and the one having the highest score is selected from the rearranged hypotheses, and the new hypothesis 518 is output.

  Figure 14 shows an example of hypothesis rescoring.

  Some phonemic contexts may not have appeared during training. For such contexts, the proposed pentaphone context model cannot produce output probabilities during recognition. To deal with this problem, we simply assign a small number as the output probability. Because this rescoring involves output probabilities from the preceding, following, and central models, flooring is applied to all elemental models.

If the amount of training data is insufficient, the parameter estimation is less reliable, even for the proposed pentaphone model, and the state output is also less reliable. Deletion interpolation was used to improve the reliability of the model, but as a result, if a model that seems to be more precise actually lacks reliability, it can return to a more reliable model. This concept is related to interpolating between two separately trained models, one of which is trained more reliably than the other. However, instead of interpolating the two models, we applied this approach to the incorporation of two phoneme likelihoods. However, here we propose a Bayesian penta von model of phoneme likelihood _{P (X s | λ bay P} enta) is a more precise, triphone of the likelihood _{P (X s} | λ triphn) is more reliable The higher one. For this reason, the phoneme likelihood P (X _s | λ) is given by

ただし，αはここで提案したペンタフォンモデルのＨＭＭ音素尤度の重みを表し，（１−α）はトライフォンモデルのＨＭＭ音素尤度の重みを表す．トレーニングデータの量が十分に多ければ，Ｐ（Ｘ_ｓ｜λ_ｂａｙＰ_ｅｎｔａ）はより信頼性が高くなり，αは１．０に近づく．十分でなければ，αは０．０に近づき，より信頼性の高いモデルＰ（Ｘ_ｓ｜λ_{ｔｒｉｐｈｎ}）に戻る．

Here, α represents the weight of the HMM phoneme likelihood of the pentaphone model proposed here, and (1-α) represents the weight of the HMM phoneme likelihood of the triphone model. If the amount of training data is large enough, P (X _s | λ _{Bay Penta} ) becomes more reliable and α approaches 1.0. If not, α approaches 0.0 and returns to the more reliable model P (X _s | λ _triphn ).

  At the beginning / end of the utterance, all left / right contexts are filled with silence. Assuming that there is no long silence between adjacent words, the last phoneme context of the previous word also affects the first phoneme context of the current word. This rescoring mechanism thus behaves similarly for all segments within and between words (crossword model).

  The score calculated as described above is then combined with the LM score corresponding to the current hypothesis. From N best, the hypothesis that achieves the highest utterance score is selected as the new recognition output.

6). Experiment An accented English speech corpus prepared by the applicant (ATR) was used for this experiment. The material of the sentence is based on the basic domain of expressions used in travel. The utterance database consists of American (US) and Australian (AUS) English accents, each accented by about 45,000 utterances (44 utterance hours) by 100 speakers (50 men and 50 women). It consists of 90% of this data, that is, 40,000 utterances (20,000 utterances by 40 male and female speakers) were used as training data. For evaluation, 200 utterances by 20 different speakers (10 males and 10 females) were randomly selected from the remaining 10% mixture of accent data (US and AUS). The bigram and trigram language models were trained with about 150,000 travel statements. The pronunciation dictionary that was available consisted of 37,000 words and was based on US pronunciation.

16kHz sampling frequency, a frame length of 20 ms, a frame shift of 10 ms, and 12-order MFCC (Mel-Frequency Cepstrum Coefficients: Mel frequency cepstrum), use the 25-dimensional feature parameters consisting ΔMFCC and Δ log power It was. For all phonemes, three states were used as the initial HMM. Using a continuous state splitting (SSS) training algorithm, a triphone acoustic model having a state coupled HMnet topology was obtained. The number of state bindings is automatically determined by the algorithm because the SSS algorithm used here is based on a Minimum Description Length (MDL) optimization criterion. Details of MDL-SSS are described in other documents (Non-patent Document 9). SSS topology training was performed using all training data. The total number of states was 2,126, and four types of Gaussian mixture components per state were obtained, that is, models with 5, 10, 15, and 20 Gaussian mixture components per state.

  It is also possible to incorporate additional knowledge such as gender and accent into the traditional triphone acoustic model (AM) by training gender and / or accent dependent AM. Only built-in training procedures with predetermined accent or gender training data were performed to ensure that the structure corresponding to the topology for all models was the same. Thus, in total, one single triphone AM (no additional knowledge), two accent-dependent triphones AM (for both US and AUS), and two gender-dependent triphones AM ( And 4 accents and gender-dependent triphones AM (for men and women in US and men and women in AUS).

  The performance of these five mixed component baseline models per state was plotted in the graph of FIG. The baseline of triphone without additional knowledge achieved a word accuracy rate of 83.60%. However, only the gender-dependent model could improve the performance slightly. The performance of other models was only reduced. In particular, the accent / gender dependency model decreased to a word correct rate of 82.11%. This is probably because the amount of training data was particularly small compared to other baseline models.

A. Performance when incorporating knowledge sources at the HMM state level The same amount of training for all accent data labeled with phoneme class context variables, as described in Section 4-B, for the proposed pentaphone model We trained using the data. The model's state topology, total number of states, and transition probabilities are all the same as the triphone HMM baseline. Therefore, they all have similar complexity in terms of the number of parameters. The main difference is that in the state probability distribution, each Gaussian distribution is clearly conditioned by C _L or C _R. In contrast, all Gaussian components at the HMM baseline were learned by the EM algorithm without a “significant” interpretation of the mixed index. Some phonemes context class C _L or C _R is absent by the grammar rules, or not appear in the training data, the result, after training, the Gaussian distribution of state per average of about 50 was obtained. By reducing the size of the pentaphone model to correspond to 5, 10, 15, and 20 mixed components per state using data-driven clustering techniques, the reliability of the estimated parameters is avoided and the total number of Gaussian distributions is reduced. Being exactly the same makes it possible to compare performance with the baseline system.

  First, we evaluated the performance of the pentaphone model BN-C, BN-CG, BN-CA or BN-CGA using the same test data as the baseline. The results of all four models with the same number of mixed components on average, five per state, are plotted in FIG.

  As can be seen, recognition was improved by using all BN types and changing the state probability distribution to incorporate various types of knowledge sources. However, when the gender and accent variables were incorporated, the recognition rate of the proposed model did not improve further. This problem may also be related to the limited training data for each accent or gender-dependent model. That is why the best performance is 85.03% when using BN-C.

  We evaluated this with a set of matching accent tests. This test data is 200 utterances randomly selected from each accent (US and AUS) to investigate further what the effects that can be achieved with BN-C. Table 2 summarizes the results obtained using different numbers of mixed component models.

これからわかるように，ここで提案したペンタフォンモデルは，同じ数のパラメータの範囲ではベースラインよりも良い性能を示す．ＵＳのペンタフォンＨＭＭ／ＢＮの最良の性能はガウス混合分布数が１０の時に得られ，これによってＷＥＲ（Ｗｏｒｄ Ｅｒｒｏｒ Ｒａｔｅ：単語誤り率）が相対的に約８％削減し，ＡＵＳのペンタフォンの最良の性能はガウス混合分布数が２０の時に得られ，ＷＥＲが相対的に約１１％削減した．一致しないアクセントのテストの組でもこれらペンタフォンモデルの性能を評価した．例えば，ＵＳ発声でトレーニングされたモデルをＡＵＳ発声のテストデータでテストし，その逆も行なった．１５個の混合成分のモデルを用いて得られた結果をテーブル３に要約する．一致時と不一致時との比較を簡単にするため，テーブル３には一致するアクセントの評価から得た結果も含ませてある．一致しないアクセントに対するペンタフォンモデルでも，標準的なＨＭＭトライフォンモデルに比べ，依然として一貫して性能が優れていることが分かる．

As can be seen, the proposed pentaphone model performs better than the baseline for the same number of parameters. The best performance of US pentaphone HMM / BN is obtained when the number of Gaussian mixture distributions is 10, which reduces the WER (Word Error Rate) by about 8% and The best performance was obtained when the number of Gaussian mixture distributions was 20, and the WER was relatively reduced by about 11%. The performance of these pentaphone models was also evaluated in a test set of inconsistent accents. For example, a model trained with US utterances was tested with AUS utterance test data and vice versa. Table 3 summarizes the results obtained using the 15 mixed component model. To simplify the comparison between coincidence and non-coincidence, Table 3 also includes the results obtained from the evaluation of matching accents. It can be seen that the Pentaphone model for inconsistent accents still has consistently better performance than the standard HMM triphone model.

B. Performance when incorporating knowledge sources at the HMM phoneme model level In Non-Patent Document 8, we investigated several methods for decomposing the pentaphone model and found that the best method was the C1L3R3 configuration. Here we describe an additional experiment using only the C1L3R3 model.

  All components of all accented pentaphone models were trained separately using the same amount of training data and the same SSS training algorithm. The total number of states is 3,360 (total of C1: 132 state, L3: 1,746 state, R3: 1,782 state), and the number of four types of Gaussian mixture components per state, ie, 5, 10, 15 and 20 The number of Gaussian mixture components was obtained. A built-in training procedure was then performed for Pentaphone C1L3R3-A, C1L3R3-G, and C1L3R3-AG with specific accent or gender training data.

最初に，付加的知識源の組込みが複数のアクセント付のテストデータに対しどんな効果を有するかを評価した．５つの混合成分を有する，提案に係るペンタフォンＣ１Ｌ３Ｒ３，Ｃ１Ｌ３Ｒ３−Ａ，Ｃ１Ｌ３Ｒ３−Ｇ，及びＣ１Ｌ３Ｒ３−ＡＧに対する結果を図１７に要約する．１０ベストリストと，削除補間のための０．３の重みパラメータαを用いて再スコアリングが行なわれた．ここから分かるように，組込んだ知識源が多いほど，性能もよくなった．提案に係るペンタフォンＣ１Ｌ３Ｒ３モデルは，ベースラインに対して性能が向上し，達成された最高性能は，アクセントＡ，性別Ｇ，先行コンテキストＣ_Ｌ，及び後続コンテキストＣ_Ｒという付加的知識を組込んだＣ１Ｌ３Ｒ３−ＡＧによる，８４．３８％という単語正解率である．性別及びアクセントが組込まれた時には，ペンタフォンＨＭＭ／ＢＮに対する場合と同様，性能の低下はなかったが，これは恐らく削除補間法を使用したことによるものである．

First, we evaluated the effect of incorporating additional knowledge sources on test data with multiple accents. The results for the proposed pentaphones C1L3R3, C1L3R3-A, C1L3R3-G, and C1L3R3-AG with five mixed components are summarized in FIG. Rescoring was performed using 10 best lists and a weight parameter α of 0.3 for deletion interpolation. As you can see, the more knowledge sources incorporated, the better the performance. Penta von C1L3R3 model according to the proposed maximum performance capability is improved versus baseline was achieved, incorporating accents A, sex G, prior context C _L, and the additional knowledge that subsequent context C _R The correct word rate is 84.38% according to C1L3R3-AG. When gender and accent were incorporated, there was no performance degradation, as was the case with Pentaphone HMM / BN, probably due to the use of the deletion interpolation method.

  Next, we investigated the details of the performance of C1L3R3-AG for all test data with accents using the N best (N = 10) list. The weight parameter α for the interpolation deletion method was the same (0.3). Here, both the relative improvement (Rel-Imp) used in Non-Patent Document 1 and the relative improvement (Rel-Resc-Imp) for rescoring were calculated as follows.

ただし，Ｎベストリストの上限はＮベスト認識結果である．

However, the upper limit of the N best list is the N best recognition result.

  Table 4 summarizes the results obtained with various numbers of mixed component models. As can be seen, the proposed pentaphone model has consistently improved the performance of the ASR system. The highest Rel-Resc-Imp was obtained for 15 mixed models for both US and AUS accents (37.92% for US model and 38.04% for AUS model).

  We also evaluated the performance of the proposed pentaphone C1L3R3-AG model against the mismatched accent test set. Table 5 summarizes the results obtained using the model with 15 mixed components. Table 5 also includes the results from evaluations for matching accents to simplify comparisons between matches and mismatches. It can be seen that the proposed pentaphone C1L3R3-AG model shows consistently better performance than the standard triphone model for mismatched accents.

C. Comparison of various models Finally, the performance of the proposed model using a 2,202 state number conventional pentaphone HMM model trained using MDL-SSS from nothing. An additional experiment was conducted to investigate whether the height of the was mainly brought about by the wide phoneme context. A gender and accent dependent pentaphone model was also obtained using a built-in procedure with training data for a specific accent or gender. These were realized by re-scoring the N best list, as in Bayes Pentaphone.

  The results for all models with five mixed components per state are plotted in FIG. As can be seen, the proposed Pentaphone C1L3R3 model improves performance compared to the baseline, which is better than simply re-scoring with a conventional Pentaphone HMM. This is because a model having a total of 2,202 states was obtained as a result of training a conventional pentaphone model using the MDL-SSS algorithm when a certain amount of training data was given. This is probably because it is not so different from the total number of states in the triphone HMM. The resolution of the context was reduced because there seem to be too many different pentaphone contexts sharing the same Gaussian distribution component. Therefore, by improving the context resolution and performance by approximating the pentaphone model using a combination of several context-independent models, we were able to promote the improvement of context resolution and performance. The highest performance obtained was 85.03% of correct word rate by BN-C.

7). Conclusion A general framework for incorporating additional knowledge sources into HMMs based on statistical acoustic models is described. The implementation of this framework was presented by incorporating wide-area phoneme context information into the triphone HMM. This was first done at the state level of the HMM using BN. Even if additional knowledge sources are hidden during recognition, this approach allows the standard decoding system to be used without modification. Next, wide-area phoneme context acoustic modeling was built at the HMM phoneme model level by building with several other models with narrower contexts. Because this composite technique resulted in a reduction in the number of context units to be estimated, it was only necessary to estimate a model with less context dependency, so the context resolution was significantly improved.

These global context model configurations were applied to the post-processing stage by N-score re-scoring. Experimental results show that the phoneme context model created by the proposed framework improves the word accuracy rate compared to the standard triphone model. Two previous context C _L, additional knowledge that the two after the context C _R, is suitable for incorporation in the HMM state level, while the additional knowledge that accent A and sex G, HMM phoneme model level It was more suitable for incorporation in.

  As mentioned above, the present invention relates to a method and apparatus for incorporating additional knowledge sources in a unified way. These methods and devices use the Bayesian network framework to easily integrate all additional knowledge sources from any domain. The advantages of this graphical model framework are: (1) it enables learning of probabilistic relationships between information sources, and (2) the joint probability density function is linked to each other locally. It is easy to decompose into a set of conditional probability density functions. Since the model is in a simplified form, it is possible in this way to build a model with a limited amount of data and estimate it reliably.

  This framework represents a general approach. In other words, this framework can be applied to many existing acoustic model modeling problems, each with a model-based likelihood function.

Realization by computer The above-described embodiment can be realized by a computer system and a computer program executed on the system. FIG. 19 shows the external appearance of the computer system 650 used in these embodiments, and FIG. 20 is a block diagram of the computer system 650. The computer system 650 shown here is merely exemplary, and various other configurations can be used.

  Referring to FIG. 19, a computer system 650 includes a computer 660, a monitor 662, a keyboard 666, a mouse 668, a speaker 692, and a microphone 690. Further, the computer 660 includes a DVD (Digital Versatile Disc) drive 670 and a semiconductor memory port 672.

  Referring to FIG. 20, a computer 660 further includes a bus 686 connected to DVD 670 and semiconductor memory port 672, a CPU (Central Processing Unit) 676 for executing a computer program for realizing the above-described device, and computer 660. ROM (Read-Only Memory) 678 for storing the boot-up program, RAM (Random Access Memory) 680 for providing a work area used by the CPU 676 and a storage area for programs executed by the CPU 676, audio data, and sound Hard disk drive 674 for storing data, language model and lexicon required for speech recognition, and connection of computer 660 to network 652 And a network interface (I / F) 696 for providing all of them are connected to the bus 686.

  The software that realizes the system according to the above-described embodiment is distributed in the form of an object code stored in a storage medium such as a DVD 682 or a semiconductor memory 684, and is connected to a computer via a reading device such as a DVD drive 670 or a semiconductor memory port 672. 660 and stored in the hard disk drive 674. When the CPU 676 executes the program, the program is read from the hard disk drive 674 and stored in the RAM 680. An instruction is fetched from an address specified by a program counter (not shown), and the instruction is executed. The CPU 676 reads data to be processed from the hard disk drive 674 and stores the processing result in the hard disk drive 674 as well. The speaker 692 and the microphone 690 are used for speech recognition and speech synthesis.

  The general operation of computer system 650 is well known and will not be described in detail here.

  Regarding software distribution methods, software does not necessarily have to be fixed on a storage medium. For example, the software may be distributed from another computer connected to the network 652. A part of the software may be stored in the hard disk 674, and the remaining part of the software may be taken into the hard disk 674 via the network and integrated at the time of execution.

  Typically, modern computers utilize general-purpose functions provided by a computer operating system (OS) and execute these functions in a controlled manner according to the desired purpose. Therefore, even if it is a program that does not include general-purpose functions that can be provided by the OS or a third party and only specifies a combination of execution order of general functions, a control structure that achieves a desired purpose as a whole Obviously, the program is included in the scope of the present invention as long as it has.

The embodiment disclosed this time is merely an example, and the present invention is not limited to the embodiment described above. The scope of the present invention is indicated by each claim in the claims after taking into account the description of the detailed description of the invention, and all modifications within the meaning and scope equivalent to the wording described therein are included. Including.

It shows the general procedure for incorporating additional knowledge sources into an acoustic model. It is a figure which shows various BN topologies. FIG. 2 shows some examples of various BN topologies. It is a figure which shows BN topology, the corresponding triangulate graph, and the junction tree obtained from one of the triangulation graph. It is a figure which shows the same BN topology as BN shown to FIG. 3 (A), and the junction tree corresponding to this. FIG. 6 is a diagram illustrating a conventional HMM acoustic model at a Gaussian mixture distribution density used to model triphones / a ⁺ , a, a ⁻ /. Penta von Context ^{/ a -, a -, a} , a +, a diagram showing a BN-C topology for modeling ^{a ++} /. It is a figure which shows the topology of BN-CG, BN-CA, and BN-CGA. It is a figure which shows the example of observation space modeling by BN. It is a figure which shows the conventional triphone model, the conventional pentaphone model, and the Bayes pentaphone model configuration C1L3R3. It is a figure which shows BN topology, the moral and triangulate graph corresponding to this, and the junction tree corresponding to this. 1 is a diagram showing an overall structure of an ASR system 500 according to an embodiment of the present invention. It is a block diagram which shows the detail of the hypothesis selection module 516. FIG. It is a figure which shows the example of the mechanism of N best rescoring which concerns on this Embodiment. It is a figure which shows the value of the recognition word correct rate of the triphone baseline model used in the experiment. It is a figure which shows the value of the recognition word correct rate of the pentaphone HMM / BN model using various BN topologies. It is a figure which shows the value of the recognition word correct rate of various Bayes pentaphone models. It is a figure which shows the value of the recognition word correct rate of the triphone HMM baseline of various systems, the pentaphone HMM baseline, the pentaphone HMM baseline, and the pentaphone model which concerns on embodiment of this invention. It is a figure which shows the external appearance of the computer system 650. FIG. 1 is a block diagram showing a computer system 650. FIG.

Explanation of symbols

70, 80, 90, 100, 110, 140, 240, 330 Bayesian network 130, 150 Moral and triangulated graph 160, 180, 450 Junction tree 190 HMM
270, 290, 310, 400, 430 BN topology 164, 166, 170, 460, 464, 474 Cluster set 162, 168, 462, 472 Separator set 500 ASR system 510 Speech waveform data 512 Standard decoder 514 N Best List 516 Hypothesis selection module 530 C1L3R3 Pentaphone model 532 C1L3R3-A Pentaphone model 534 C1L3R3-G Pentaphone model 536 C1L3R3-AG Pentaphone model 550 Memory 552 Read and supply module 554, 556, 558, 560, 526 Shift memory 570 Right context calculator 572 Central context calculator 574 Left context calculator 576 Probability density function calculator 578 Rescoring Jules 580 sorting and selection module

Claims (5)

Probability calculation for calculating a probability for each predefined set of phonemes present in a given segment of the speech signal using a statistical acoustic model for the speech signal and one or more knowledge sources The apparatus, wherein the segment includes a plurality of frames of the audio signal, the acoustic model and the one or more knowledge sources have a causal relationship indicated by a Bayesian network, and the Bayesian network includes a plurality of cluster nodes. And a junction tree containing one or more separator nodes,
The device is
Means for storing a plurality of local acoustic models corresponding to the cluster nodes and one or more separator nodes;
Means for calculating predefined observation data for each of the frames;
Local probability calculating means for calculating a local probability of generating the observation data of each of the phonemes using the plurality of local acoustic models;
Wherein each phoneme, the probability of generating the observed data, look including a probability calculation means for calculating a predetermined function of the local probabilities calculated by the local probability calculation means,
The predetermined function is:

Defined by
Where D is the observed data, M is the acoustic model, N is a positive integer, Ki is a knowledge source of 1 or more,
P (D | Ki, M) (i = 1 to N) and P (D | M) are local probability calculated by the local probability calculating means . The model M is a monophone acoustic model,
The apparatus of claim 1 , wherein the one or more knowledge sources include a preceding triphone context unit and a subsequent triphone context unit. The model M is a monophonic acoustic model trained with additional knowledge sources,
The apparatus of claim 1 , wherein the one or more knowledge sources include a preceding triphone context unit and a subsequent triphone context unit. 4. The apparatus of claim 3 , wherein the additional knowledge source includes accent knowledge, or gender knowledge, or both accent knowledge and gender knowledge. When executed on a computer , the computer causes a statistical acoustic model and one or more knowledge for the speech signal for each of a predefined set of phonemes present in a given segment of the speech signal. A computer program that functions as a probability calculation device for calculating a probability using a source, wherein the segment includes a plurality of frames of the speech signal, and the acoustic model and the one or more knowledge sources are based on a Bayesian network. The Bayesian network corresponds to a junction tree that includes a plurality of cluster nodes and one or more separator nodes;
The computer program stores the computer,
Means for storing a plurality of local acoustic models corresponding to the cluster nodes and one or more separator nodes;
Means for calculating predefined observation data for each of the frames;
Local probability calculating means for calculating a local probability of generating the observation data of each of the phonemes using the plurality of local acoustic models;
Functioning as probability calculation means for calculating the probability of generating the observation data of each of the phonemes as a predetermined function of the local probability calculated by the local probability calculation means ;
The predetermined function is:

Defined by
Where D is the observed data, M is the acoustic model, N is a positive integer, Ki is a knowledge source of 1 or more,
P (D | Ki, M) (i = 1~N) and P (D | M) is Ru local probability der calculated by the local probability calculation unit, the computer program.

Images