JP2001312294A - Learning method of transducer transducing input symbol series into output symbol series, and computer-readable recording medium with stored learning program of transducer - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a learning method of a transducer, transducing an input symbol series into an output symbol series, which makes it possible to obtain a transducer taking into account not only the context of input symbols, but also the context of output symbols and a computer-readable recording medium with stored a learning program of the transducer. SOLUTION: This is a learning method of a transducer transducing the input symbol series into the output symbol series; and a group of input and output symbols which are previously made to correspond to each other is used as learning data and modeled as n-gram.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] 1. Field of the Invention [0002] The present invention relates to a transducer learning method for converting an input symbol string into an output symbol string and a computer-readable recording medium storing a transducer learning program.

[0002]

2. Description of the Related Art Various probability models required for speech recognition are as follows.
They were often generated based on independent formulas. For example, an acoustic model is often modeled as an HMM, and a language model is modeled as an n-gram. However, with the development of automata theory in recent years, these stochastic models have been replaced by weighted finite state transducers (weig
hted finite-state transducer, hereinafter WFST)
It has become clear that a model can be unified as (see Document 1).

Reference 1: Fernando Pereira and Michael
Riley, "Speech Recognition by Composition of Weig
hted Finite Automata ", In Emmanuel Roche and Yves
Schabes, editors, Finite-State Language Processin
g, pp.431-453, MIT Press, Cambridge, Massachusett
s, 1997.

[0004] WFST is a generalization of likelihood in the form of weights, and is a stochastic finite state transducer (probabi
listic finite-state transducer (hereinafter PFST) can be considered a special form. In addition, a plurality of WFSTs are converted into one W
It has been demonstrated that large-vocabulary continuous speech recognition can be performed efficiently and with high accuracy by expanding to FST (see Reference 2).

Reference 2: Mehryar Mohri, Michael Riley,
Don Hindle, Andrej Ljoljo and Fernando Pereira, "F
ull Expansion of Context-Dependent Networks in Lar
ge Vocabulary Speech Recognition ", In Proc. of the
International Conferenceon Acoustics, Speech, and
Signal Processing (ICASSP '98), 1998.

[0006] Against this background, a method of automatically learning a probability model necessary for speech recognition as WFST has recently been developed.
It has become one of the important research issues.

Until now, techniques for automatically learning a finite state transducer have been studied for the problems of morphological analysis and digit string conversion (see References 3 and 4). Reference 3: Emmanuel Roche and Yves Schabes, "Determin
istic Part-of-SpeechTagging with Finite-State Tran
sducers ", Computational Linguistics, Vol. 21, No.
2, pp.227-253, 1995.

Reference 4: Jose Oncina, Pedro Garcia and
Enrique Vidal, "Learning Subsequential Transducers
for Pattern Recognition Interpretation Tasks ", IE
EE Trans.Pattern Analysis and Machine Intelligenc
e, Vol. 15, No. 5, 1993.

However, since these methods do not learn a probabilistic model, they cannot be said to be very suitable for speech recognition. Further, as widely used in speech recognition, a method of constructing an environment-dependent HMM for each input (or output) symbol and modeling symbol string conversion is also conceivable. However, since the output symbols of the HMM assume independence, their ability as a transformation model is not very high. In the field of statistical machine translation, a method of automatically learning conversion between languages associated with a synchronization dependency tree has also been proposed (see Reference 5).

Reference 5: Hiyan Alshawi, Bangalore Srini
vas and Shona Douglas, "LearningDependency Transla
tion Models as Collections of Finite State Head Tr
ansducers ", Computational Linguistics, Vol. 26, 20
00.

[0011] However, the stochastic head transducer learned here cannot use the computation useful for speech recognition defined on the WFST as it is.

[0012]

According to the present invention, the conversion between a context-dependent input symbol string s _in and a context-dependent output symbol string s _out is performed by P (s _in , s _out ) or P (s _in | S
_out ) is proposed as a WFST that outputs automatically. This method models input / output symbol pairs (sets of input / output signals) associated in advance as variable-length n-grams (references 6, 7, and 8).
By making the order of m variable, the number of parameters can be optimized.

Document 6: Masahiko Haruno and Yuji Matsumoto, "Morphological Analysis Using Context Tree", IPSJ SIG Technical Report, 96-NL-
112, pp.31-36, 1996. Reference 7: Hinrich Schutze and Yoram Singer, "Part-of
-Speech Tagging Using a Variable Memory Markov Mod
el ", 32nd Annual Meeting of ACL, 1994. Reference 8: Marcelo J. Weinberger, Jorma J. Rissanen a
nd Meir Feder, "A Universal Finite Memory Source",
IEEE Trans. Information Theory, Vol. 41, No. 3, 19
95.

The invention is not limited to the context of the input symbol,
An object of the present invention is to provide a transducer learning method for converting an input symbol string into an output symbol string, and a computer-readable recording medium storing a transducer learning program, which can provide a transducer that also considers the context of output symbols.

[0015]

SUMMARY OF THE INVENTION The present invention relates to a transducer learning method for converting an input symbol string into an output symbol string, wherein a set of input / output symbols associated in advance is used as learning data. Ng the set of input / output symbols
It is characterized by being modeled as ram.

It is preferable to model the set of input / output symbols associated with each other as a variable-length n-gram using a context tree.

The present invention is a computer-readable recording medium storing a transducer learning program for converting an input symbol string into an output symbol string, using a set of input / output symbols associated in advance as learning data. A learning program for causing a computer to execute processing for modeling a sequence of a set of input / output symbols associated with each other as an n-gram is recorded.

[0018]

Embodiments of the present invention will be described below with reference to the drawings.

[1] Formulation of recognition problem based on WFST Here, the significance of automatically learning a WFST that outputs a conditional probability is clarified by formulating a recognition problem according to Reference 1.

First, the concept of semiring is introduced to express the weight (likelihood) of WFST. By using a half-ring, the likelihood used in speech recognition and the operation corresponding thereto can be expressed with good visibility.

A half ring is defined as <K, such as (i) to (iv).
+, .., _0K , _1K >. (i) sum holds bond law, commutative, with units 0 _K. (ii) the product is holds bond law, with identity element 1 _K. (iii) The product has a distribution rule for the sum. (iv) 0 _K is the zero element of the product.

For example, the probability value is expressed as <｛x∈R | 0 ≦ x ≦
1｝, +, .., 0, 1>. Although the maximum value max is used instead of + in the processing by the Viterbi approximation, also in this case, the probability value is expressed as <{x∈R | 0 ≦ x ≦ 1}, max,
0,1>. More often, the log of the probability value is used to convert the product to a sum, but again, <｛x∈R | x ≦ 1｝ ∪ ｛−∞｝, max, +,
−∞, 0>.

WFST converts an input symbol string into an output symbol string.
Is a finite state machine that converts to and returns weights. WFST
A is a set of 6 <Q_A, Σ^* _Ain, Σ^* _Aout, i_A, F
_A, E_A>. Q_AIs a finite set of states, Σ
_AinIs a finite set of input symbols, Σ _AoutIs a finite collection of output symbols
If i_A∈Q is the initial state, F_A: Q_A→ K is the final weight
Function, E_A⊆Q_A× Σ^* _Ain× Σ^* _Aout× K × Q_AIs trans
Represents a set of moves. Σ^*Is the Kleene closure of Σ, zero or more of Σ
Represents a set whose elements are the concatenation of the above elements. transition
K is a semi-ring representing a weight. F_AIs
It is a generalization of the concept of an ordinary final state, and has a normal meaning.
The final state is 1 if the final state_K, Otherwise 0
_KCan be expressed as a function that returns Also, only input symbols
For general automata that do not work,_AinAnd Σ_Aout
Can be expressed by WFST as
You.

[0024] The function used to refer to the components of the t∈E _A
Let src, dst, in, out, ω be t = (src (t), in (t), out (t), ω
(t), dst (t)). The WFST path p is a sequence of transitions p = t ₁ ,..., _Tm such that src (t _i ) = dst (t _i-1 ) with 1 <i ≦ m. The weight for the path p is w (p) = w (t ₁ )... W (t _m ), and the input symbol string is in
(p) = in (t ₁ )… in (t _m ), output symbol string is out (p) = out
(t ₁ )… out (t _m ), start point is src (p) = src (t ₁ ), end point is d
represented by st (p) = dst (t m). The weight W (u, v) for the input symbol string u and the output symbol string v is src (p) = i _A , in (p) =
Σ _p for all paths p such that u, out (p) = v
ω (p) • Defined by F _A (dst (p)).

From the WFST in which the input / output symbol string is Σ ^* ₀ × Σ ^* ₁ and Ｗ ^* ₁ × Σ ^* ₂ , Σ ^* ₀ ×
A composite WFST can be defined such that Σ ^* ₂ . The composition algorithm is similar to finding the intersection of the finite state automan (FSA), both of which have a joint rule. Here, for simplicity of description, only the composite A * B of WFST A in which the input / output symbol of the transition is Σ ₀ × Σ ₁ and WFST B in which the input / output symbol of the transition is Σ ₁ × Σ ₂ will be described. For details, see Reference 1.
checking ...

A * B satisfies the following conditions (i) to (ii) <
Q _A × Q _B , Σ ₀ , Σ ₂ , (i _A , i _B ), F _{A * B} , E
_{A * B} >.

(I) F _{A * B} (q, q ') = F _A (q) F _B
(Q ') (ii) ( q, x, y, k, r) ∈E A and (q', y,
z, k ′, r ′) {E _B } ((q, q ′), x, z, k
× k ', (r, r') ∈EA _{* B}

Using the concept of WFST and its synthesis, a probability model for speech recognition can be expressed with good visibility. P (s _{n + 1} | s ₀ ) is rewritten as shown in Equation 1 through the intermediate symbol sequence s _i in several stages. However, each s _i depends only on s _i-1 .

[0029]

(Equation 1)

A conditional probability P (s _i is given by WFST A _{i, i-1 where} s _i is an input symbol string and s _i-1 is an output symbol string.
| S _i-1 ) is modeled. At this time, the weight is <
｛X ∈R | 0 ≤x ≤1｝, +, ·, 0,1> or <｛x ∈
R | 0 ≦ x ≦ 1｝, max, ·, 0,1> Number 1
Is P (s) by A _{n + 1, n} * A _{n, n-1} *... A _{1,0
n + 1} | s ₀ ) can be modeled. Therefore, if o is a speech parameter sequence and ω is a symbol sequence to be recognized, a speech recognition problem is generally expressed as in Equation 2.

[0031]

(Equation 2)

After all, the search for the recognition candidate is performed by A _{O, Sn} *.
_{In s2, s1} * A _s1, ω * Aω _, ω, it can be formulated as a problem of searching for ω that maximizes the W (o, ω) value for a certain input sequence o.

What is important here is that if the models required for speech recognition can be modeled as WFSTs representing conditional probabilities, they can be freely combined by a synthesis operation that satisfies the coupling rule. Because of the usefulness of this combination operation, a general-purpose method for automatically learning a WFST model representing a conditional probability is desired. In fact, in the field of acoustic models, automatic learning of such a WFST has been performed using an environment-dependent discrete HMM.
However, since the output symbols of the HMM are assumed to be independent, it is desired to model the input and output in consideration of the environment in order to obtain a more general conversion model. Next, a method for automatically learning such a WFST will be described.

[2] Basic method of learning WFST

From another point of view, the transducer is Σ = Σ
It can also be seen as a finite state automan that uses the elements of _in × Σ _out as input symbols. The basic idea of the proposed automatic learning method is Σ = Σ _in × Σ _out (where ε ∈Σ _in , ε
_out ) are modeled as n-grams. That is, the input / output symbols of the learning data are associated in advance with at most one symbol using DP matching or the like, and the sequence of the set of the associated input / output symbols is represented by n-gr.
am is modeled as a stochastic finite state automan. In this way, the joint probability P (s _in , s _out ) (s _in ∈Σ
_in, it s _{out ∈Σ ou t)} can be learned WFST to output a.

Further, fixing n of n-gram is not optimal as the relation between the number of parameters and the performance. Therefore, in the present learning method, a method of modeling as a variable length n-gram using a context tree is adopted.

WFST of | (s _out s _in) [0037] conditional probability P
Are the WFST of P (s _in , s _out ) and the WF of 1 / P (s _out )
It is required as a composition of ST. However, the synthesis of this conditional probability WFST generally involves not only a huge calculation cost but also a large WFST to be synthesized. Therefore, in the following [3], a method of creating a context tree in consideration of the conditional probability will also be described.

The length of the context considered in the present method is the same for both the input symbol string and the output symbol string. Of course, a model that allows for different length contexts could be considered as an extension of this method. In this case, a model that does not consider the context of an input symbol string is equivalent to a left-environment-dependent discrete HMM. In the following, the correspondence between input and output symbols will be described as m: n (0 ≦ m ≦ 1, 0 ≦ n ≦ 1). However, the method itself is extended to the case where the maximum values of m and n are arbitrary. It is possible. The ε generated as a result of the association is treated in this specification as equivalent to a normal symbol. Naturally, a derivation method such as removing ε from the context of n-gram may be considered.

[3] Modeling Using Context Tree The context tree is a tree in which n-gram contexts are hierarchically managed. The branches of the context tree T are labeled with elements of Σ. Nodes are labeled with contextual labels, which are defined recursively as follows:

(A) The root node is labeled with ε representing an empty context. (B) The label σs obtained by adding the label σ of the branch from the parent to the child node before the label s of the parent node is added to the child node.

[0041] Each node _{s, Σ U P T (σ} | s) = 1
(Where U = σ∈Σ), the probability P _T (σ |
s) is defined. At this time, the probability of the character string ω = ω ₁ , ω ₂ ,..., Ω _n generated by the context tree T is given by Expression 3.

[0042]

(Equation 3)

Where s ₀ = ε, s _i (i> 0) is equal to ω ₁ ,
.., Ω _i suffixes match the label of the deepest node of T. As the estimation amount of P _T (σ | s), Equation 4 assuming the flooring value α was used.

[0044]

(Equation 4)

Here, n _T (σ | s) indicates the number of times σ appears after the context s. The pruning of the context tree in an optimal form can be performed using the gain function of Equation 5 (see Documents 6 and 8).

[0046]

(Equation 5)

This is because, instead of the parent node s, the child node σ
It shows the gain obtained when s is used. As shown in FIG. 1, using this gain function, it is possible to cut off a branch ahead of the deepest node s that satisfies Δ (s) ≧ C for a certain constant C.

Referring to FIG. 2, the context tree can be transformed into a stochastic finite state automan as follows.

(1) A parent node is sequentially copied so as to create a child node so as to satisfy the following conditions (i) to (ii). At this time, P _T (w _i | σs) of the child node s is equal to the parent node s
P _T (w _i | s). (i) For all prefixes of all leaf nodes, there is a node in the tree labeled with this. (ii) All sibling nodes are present for all nodes.

(2) The nodes of the context tree are made to correspond to the state of WFST.

(3) All leaf nodes s, all σ∈
For Σ, a leaf node s ′, which is a suffix of sσ, is uniquely determined. From s to s', the input / output symbol is σ and the weight is P
Create a transition such that _T (σ | s).

A WFST that outputs P _T (s _in , s _out ) such that s _in ∈Σ ^* _in , s _out ∈Σ ^* _OUT is
(Σ _in ∪ {ε}) was found to be auto-learning by a _{× (Σ out ∪ {ε}} ). Then, P _T (s _in
| S _out ) will be described.

Out: Σ ⁿ → (Σ _out ∪ ｛ε｝) ⁿ ,
Let と す^{る be} a mapping from ⁿ to its output symbol sequence.

(1) Assuming that s _train学習^* is learning data of T, T _out is learned using out (s _train ).

(2) Δ _T (s) ≧ C and Δ _Tout (out (s))
When ≧ C is not satisfied, both T and T _out are branched.

(3) Similarly, both T and T _out are extended to create a child node.

(4) A transition is created in the same manner, but P _T (σ | s) / P _Tout (out (σ) | out (s)) is used as the weight of the transition.
Is used.

Thus, the WFST obtained from T
And WFST obtained from T _out (where the weight of the transition is 1
/ P _T (s _out )) is regarded as a general symbol, and a WFST is obtained.

[4] Automatic learning of pronunciation deformation model

In the field of speech recognition, automatic learning of statistical models such as acoustic models based on HMMs and language models based on n-grams has achieved considerable success. However, the creation of pronunciation dictionaries that model the pronunciation of words usually still relies heavily on human language-dependent knowledge. Since the creation work is very labor-intensive, some automation is desired. This is a serious problem especially when trying to make the system multilingual.

As a first step toward this goal, an attempt is made to automatically learn a pronunciation transformation model for converting a phoneme string p into a phoneme string p ′ representing a typical pronunciation using the proposed method. Here, it is assumed that the pronunciation of a word can be modeled by a combination of a typical pronunciation dictionary automatically created from the kana of a word and a pronunciation transformation model from the representative pronunciation.

In fact, it has been confirmed that the recognition performance is improved by modeling the pronunciation deformation by a neural network and adding pronunciation to the pronunciation dictionary for each word (see Reference 9).

Reference 9: Toshiaki Fukada, Takayoshi Yo
shimura and Yoshinori Sagisaka, "Automatic Generati
on of Multiple Pronunciations based on Neural Netw
orks ", Speech Communication, Vol. 27, No. 1, pp. 63
-73, 1999.

[4-1] Content of Experiment Based on the formulation described in the above [1], an acoustic model,
We try to express the relationship between language models, pronunciation dictionaries, and pronunciation transformation models. When p is a phoneme sequence, p ′ is a phoneme sequence representing a representative pronunciation, and ω is a word sequence, the problem of continuous word recognition is expressed by Equation 6.

[0065]

(Equation 6)

Further, by defining the sum of the weights by max, the following equation (7) is obtained.

[0067]

(Equation 7)

P (o | p) is an acoustic model, and P (p |
p ′) represents a pronunciation transformation model, P (p ′ | ω) represents a representative pronunciation dictionary, and P (ω) represents a language model.

Here, the pronunciation transformation model P (p | p ')
For the evaluation of, two experiments represented by simplified equations 8 and 9 are performed.

[0070]

(Equation 8)

[0071]

(Equation 9)

The experiment represented by Expression 8 is an experiment in which a phonetic transcription is converted into a typical pronunciation. The experiment represented by Expression 9 is for speech recognition using speech as a representative pronunciation.

[4-2] Experimental Conditions Representative pronunciations were regularly generated from the pseudonym of each word. In actual pronunciation, long vowels are expressed by continuing two vowels, but in order to absorb fluctuations, typical pronunciations are represented by short vowels without distinguishing between long and short vowels.

Further, the signal does not cross the morpheme boundary / ou /,
/ Ei / was also represented by / o / and / e /, respectively. The actual pronunciation includes a special phoneme indicating a pause, but the typical pronunciation does not include this. DP matching was used for the correspondence between the two phoneme strings for the learning data.

FIG. 3 shows an example of phoneme correspondence of the utterance "Um, please give me an extra bet."
The lower part of FIG. 3 shows the phoneme sequence (input symbol sequence p) of the actual pronunciation,
The upper row shows a phoneme string (output symbol string p ′) representing a typical pronunciation.

In the ATR travel conversation database (see Reference 10), 532 utterances were set as a test set, and 6418 utterances not including the 532 utterances were set as a learning set. The test set and the learning set were created so that the utterance in which the misstatement exists does not include the correct word string because it does not match the speech data or the phoneme transcription.

Reference 10: Toshiyuki Takezawa, Tsuyoshi
Morimoto and Yoshinori Sagisaka, "Speech and Lang
uage Databases for Speech Translation Research in
ATR ", First International Workshop on EALREW (Orie
ntal COCOSDA), pp.148-155,1998.

P (p ') is a variable length ng using a context tree
_{ram (n ≦ 3, Δ T} ≦ 1000) was created as. P
(P | p ') was created using two types of learning data. One is `` correct representative phoneme sequence x phoneme transcription (phone
me transcription) ”, and the other is“ correct representative phoneme sequence × phoneme typewriter (phonetic type
writer) 's first-order recognition result ". In both cases, the pruning context tree in delta _T ≦ 1000.

In a phoneme typewriter, a phoneme is added to a language model.
Bigram was used. The language model and the acoustic model are learned using the test set including the 532 utterances described above. Therefore, it is the closed recognition result (phoneme recognition accuracy of 91.7%) that is used for learning P (p | p ′). Further, the recognition experiment of Expression 9 was performed by a method of rescoring the recognition result of the phoneme typewriter.

[4-3] Experimental Results Table 1 shows the experimental results.

[0081]

[Table 1]

The training condition (training cond.) Indicates whether a phoneme transcription (transcription) or a phoneme typewriter (typewrite) was used for model learning. The test condition (test cond.) Indicates whether it is a phonetic transcription transcription experiment (transcription) of the above equation (8) or a recognition experiment (typewriter) of the above equation (9).

In the first column, n represents the order of n-gram. The numbers in the table indicate the conversion error rate (Err
or Rate = 100 × (Ins + Del + Sub) / UtteranceLength)
It is. Even if the maximum value of n is increased beyond 2, the improvement is not large, but it has been confirmed that the performance of the model monotonically improves as n increases in any combination of learning conditions and experimental conditions. The case where n = 1 does not consider the context at all, and corresponds to a confusion matrix. In comparison, when n ≦ 3, a maximum improvement of 45% was obtained.

[5] Conclusion The WF is obtained from the input / output symbol pair string
We proposed a method to learn ST automatically. The model to be learned is configured as a variable-length n-gram using a context tree, so that the number of parameters can be optimized. Since the present model can consider not only the context of the input symbol string but also the context of the output symbol, the context-dependent discrete H
It is a more powerful conversion model than MM.

Experiments applying the present method to the problem of pronunciation deformation from typical pronunciation show that by considering variable-length contexts, it is possible to perform symbol string conversion that is impossible with a confusion matrix that does not consider contexts at all. Indicated.

The present invention can be used for various purposes other than speech recognition by separately developing a method of associating input / output symbol pairs suitable for the purpose. For example, it can be used for morphological analysis, data conversion with tags, and machine translation between very similar languages.

[0087]

According to the present invention, it is possible to obtain a transducer that takes into account not only the context of input symbols but also the context of output symbols.

[Brief description of the drawings]

FIG. 1 is a schematic diagram illustrating pruning of a context tree.

FIG. 2 is a schematic diagram for explaining expansion of a context tree and addition of transitions.

FIG. 3 is a schematic diagram showing an example of correspondence between input and output symbols as learning data.

Claims (3)

[Claims] 1. A learning method of a transducer for converting an input symbol string into an output symbol string, wherein a set of input / output symbols associated in advance is used as learning data, and a set of associated input / output symbols is n. A transducer learning method for converting an input symbol string into an output symbol string, characterized by being modeled as a gram. 2. The transducer learning method according to claim 1, wherein a set of input / output symbols associated with each other is modeled as a variable-length n-gram using a context tree. 3. A computer-readable recording medium recording a learning program for a transducer for converting an input symbol string into an output symbol string, wherein a set of input / output symbols associated in advance is used as learning data. The sequence of the set of input / output symbols
A computer-readable recording medium in which a learning program for causing a computer to execute a process for modeling as m is recorded.