JP2005257910A - Symbol string transduction method and voice recognition method using the symbol string transduction method, and symbol string transduction device and voice recognition device using the symbol string transduction device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To reduce computational quantity for performing the minimum cost search in a symbol string transduction method. <P>SOLUTION: A symbol string transduction method is disclosed for outputting an output symbol string corresponding to the state transition process of a WFST of a post stage in which the cumulative value of weights to state transitions which are to be applied respectively to the WFSTs of the preceding stage and the post stage of a symbol string transducing part become the minimum. A state transition process in which the cumulative weights become the minimum in possible state transition processes in the WFST of the post stage at the time when cumulative weights to hypothesis which expresses one state transition process of the WFST of the preceding stage and the output symbol string in the state transition process of the hypothesis are made the input symbol string of the WFST of the post stage is calculated while reading symbols one by one. The cumulative weights are corrected by adding the cumulative weights to the cumulative weights of the hypothesis and, at the point of time when all of input symbol strings are read, when the hypothesis in which the cumulative weights are the minimum and an output symbol string corresponding to the state transition process of the hypothesis are made the input symbol string of the WFST of the post stage, the output symbol string to the state transition process in which the cumulative weights become the minimum in possible state transition processes in the WFST of the post stage is made a transduced result. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention minimizes the cumulative value of the weights of conversion rules to be applied from a large number of output symbol strings that can be generated for an input symbol string by a symbol string conversion rule expressed by a weighted finite state converter. The present invention relates to a symbol string conversion method that makes it possible to efficiently find an output symbol string, a speech recognition method using the same, a symbol string conversion device, and a speech recognition device using the symbol string conversion device. is there.

(WFST: weighted finite state transducer)
Weighted Finite-State Transducer (WFST) is a method of expressing rules for converting a symbol string into another symbol string in a diagram of states and state transitions.
The weighted finite state converter is disclosed in Non-Patent Document 1, for example. Hereinafter, this weighted finite state transducer is referred to as WFST.
WFST is defined by a set of a state, a state transition indicating that the state can be transitioned from state to state, an input symbol accepted in the state transition, an output symbol output at that time, and a weight of the state transition Is done. WFST is a model that, when a certain input symbol string is given, repeats the state transition while outputting the output symbols according to the state transition that sequentially accepts the symbols of the input symbol string from the initial state, and terminates when the end state is reached. is there. Formally, WFST is defined by the following eight sets (Q, Σ, Δ, i, F, E, λ, ρ).
1. Q is a set of finite states.
2. Σ is a finite set of input symbols.
3. Δ is a finite set of output symbols.
4). iεQ is the initial state.
5). FεQ is a set of end states.
6). EεQ × Σ × Δ × Q is a set of state transitions in which an output symbol is output from the current state according to an input symbol to transition to the next state.
7). λ is the initial weight.
8). ρ (q) is the end weight of the end state q. qεF.

(Example of WFST)
An example of WFST is shown in FIG.
In FIG. 1, 10 indicates a state represented by a circle (“◯”), and the number in the circle represents the state number. Reference numeral 11 denotes an end state represented by a double circle (“◎”), and the number in the double circle is the number of the end state and the end weight accumulated at the end of the state transition. Is expressed as “(state number) / (end weight)”. Hereinafter, when the state is indicated using the state number, the state and the number are simply referred to as “state 0” or “state 3”.
Reference numeral 12 denotes a state transition represented by an arrow (“→”) connecting each state, and a symbol or a number given to each state transition is an input symbol, an output symbol, or a weight associated with the state transition. Is expressed as “(input symbol) :( output symbol) / (weight)”.
The WFST of FIG. 1 can also be defined by a table. In FIG. 2, each row represents one state transition, and the state number of the transition source, the state number of the transition destination, the input symbol, the output symbol, and the weight in the state transition are described. The final state (state 3 in FIG. 1) has a format in which the transition destination, the input symbol, and the output symbol are empty, and the weight (end weight) accumulated at the end of the state transition is entered. In general, the initial state of WFST is set to state 0, and the initial weight λ is often omitted. Therefore, in the present invention, the initial state is set to state 0, and the initial weight is omitted and not specified.

(Process of converting input symbol string a, a, b, c to output symbol string d, d, c, b)
The WFST of FIG. 1 can convert, for example, the input symbol string a, a, b, c into the output symbol string d, d, c, b, and the state transition process at this time uses a sequence of state numbers. In other words, 0, 0, 1, 3 and the cumulative value of weight (hereinafter referred to as “cumulative weight”) is 0.5 + 0.5 + 0.3 + 1 + 0.5 = 2.8. However, in the WFST of FIG. 1, there are two possible state transition processes of 0, 0, 1, 3 and 0, 0, 2, 3 for the input symbol strings a, a, b, and c. In general, when there is a possibility of multiple state transitions for an input symbol string (this is called non-determinism), the state transition process that minimizes the cumulative weight in the state transition process is selected, and the minimum cumulative weight is selected. An output symbol string corresponding to the state transition process is selected. Also in the example of FIG. 1, the state transition processes 0, 0, 1, 3 having the smallest cumulative weight are selected for the input symbol strings a, a, b, c, and the conversion results are d, d, c, b. And

ここで、Ｗ（Ｘ→Ｙ；Ａ）は、ＷＦＳＴ Ａによって記号列Ｘが記号列Ｙに変換されるときの状態遷移過程における累積重みを表す。この累積重みＷ（Ｘ→Ｙ；Ａ）の最小値Ｗ（Ｘ）を求めて、その最小値を与えるＹが記号列変換結果となる。
このＷ（Ｘ）を効率的に求めるには、一般にグラフのコスト最小探索の技術の一つである横型探索法を利用する。例えば、グラフの横型探索法の手順は、非特許文献２に開示されている。
ＷＦＳＴによる記号列変換は、入力記号列によって初期状態から終了状態に至るコスト（累積重み）最小の状態遷移過程を探し出すことによって行われる。 To obtain an output symbol string (that is, a symbol string conversion result) having a minimum cumulative weight when there is a certain WFST (this is A) and a symbol string X is given as an input symbol string to this A Therefore, it is necessary to calculate the minimum value W (X) of the next cumulative weight.

Here, W (X → Y; A) represents the cumulative weight in the state transition process when the symbol string X is converted to the symbol string Y by WFST A. The minimum value W (X) of the cumulative weight W (X → Y; A) is obtained, and Y giving the minimum value is the symbol string conversion result.
In order to obtain this W (X) efficiently, a horizontal search method, which is one of the techniques for searching a graph with a minimum cost, is generally used. For example, the procedure of the horizontal search method of the graph is disclosed in Non-Patent Document 2.
Symbol string conversion by WFST is performed by searching for a state transition process with a minimum cost (cumulative weight) from an initial state to an end state by an input symbol string.

(Symbol conversion using one WFST)
An example of symbol string conversion using one WFST is shown in FIG.
First, in the present specification, the “hypothesis” means that a certain symbol string is sequentially input and the state transition is repeated from the initial state in the WFST by the input symbol string for the input symbol string read up to the present time. Let us represent one possible state transition process.
The symbol string input unit 103 sequentially reads the symbols and sends them to the hypothesis developing unit 104. In the hypothesis developing unit 104, according to the symbol received from the symbol string input unit 103 and the WFST read from the WFST storage unit 101, the state transition of each hypothesis using the newly received set of hypotheses for the symbol string read so far By updating the process, a new hypothesis is generated and sent to the hypothesis narrowing unit 105. The hypothesis narrowing unit 105 narrows down the hypotheses by deleting hypotheses other than the hypothesis having the smallest cumulative weight among the hypotheses reaching the same state from the hypothesis set received from the hypothesis developing unit 104. If the input symbol string has been read to the end, the output symbol string corresponding to the hypothesis with the smallest cumulative weight is sent to the symbol string output unit 106. If the input symbol string has not been read to the end, the hypothesis is sent to the hypothesis developing unit 104. The symbol string output unit 106 outputs the output symbol string received from the hypothesis narrowing unit 105.

(Symbol string conversion procedure)
Next, an example of a procedure for converting a symbol string based on this embodiment will be described.
First, when a state transition with WFST is represented as e, n [e] is a transition destination state (next state), i [e] is an input symbol, o [e] is an output symbol, and w [e] is a weight. Define. Also, when a certain hypothesis is represented as h, a state in which s [h] is reached, W [h] is a cumulative weight in the state transition process, and O [h] is a symbol string output in the state transition process. To do.
In this procedure, hypotheses are managed using a list of hypotheses (hereinafter referred to as a hypothesis list). Hypotheses can be inserted into and extracted from the hypothesis list. However, when a hypothesis is inserted into the hypothesis list, if there is a hypothesis that reaches the same state in the hypothesis list, only the smaller cumulative weight is left in the hypothesis list to narrow down the hypotheses.

FIG. 4 shows a symbol string conversion procedure using WFST.
Hereinafter, the procedure of FIG. 4 will be described in comparison with an embodiment (FIG. 3) of symbol string conversion using WFST.
Starting from step S101, as a default setting, empty hypothesis lists H and H ′ are generated in step S102. In step S103, an initial hypothesis h is generated, and s [h] = 0 (initial state of WFST), W [h] = 0, O [h] = φ (empty symbol string) are set in the hypothesis list H. insert.
In step S104, the symbol string input unit 103 reads one symbol and substitutes the symbol for x. The following steps S105 to S108 are executed in the hypothesis developing unit 104.
In step S105, one hypothesis is extracted from the hypothesis list H and assigned to h, and a state transition list E having an input symbol equal to x is prepared from the state s [h].
In step S106, if E = φ (empty list), the process proceeds to S110. Otherwise, the process proceeds to S107, and one state transition is extracted from E and substituted into e.
In step S108, a new hypothesis f is generated, and s [f] = n [e], W [f] = W [h] + w [e], O [f] = O [h] · o [e] To do. Here, “·” represents an operation of connecting two symbols or symbol strings to form one symbol string.

Step S109 is executed by the hypothesis narrowing unit 105 to narrow down the hypotheses by inserting the hypothesis f into the hypothesis list H ′.
Returning from step S109 to S106, a hypothesis is developed for the next state transition.
In step S110, if H = φ (all hypotheses have been developed), the process proceeds to S111. Otherwise, return to S106 and develop the next hypothesis.
In step S111, all the elements of the newly generated hypothesis list H ′ are moved to the already empty H, and the process proceeds to S112.
In step S112, if there is a next input symbol in the symbol string input unit 103, the process returns to S104. Otherwise, it is determined that all the input symbol strings have been read, and the process proceeds to S113.
In step S113, after adding the end weight of the end state to the cumulative weight of the hypothesis reaching the end state in the hypothesis list H, the cumulative weight (W The hypothesis h that minimizes [h]) is selected, and the output symbol string O [h] is output as a symbol string conversion result in the symbol string output unit 106.
In step S114, the symbol string conversion procedure using WFST is terminated.

(Process of obtaining an output symbol string when input symbol strings a, a, b, and c are given to WFST)
The process of obtaining the output symbol string when the input symbol strings a, a, b, and c are given to the WFST of FIG. 1 according to this symbol string conversion procedure will be described in order. However, if there is a certain hypothesis (current state number s, output symbol string O, cumulative weight W), the hypothesis is represented as (s, O, W). A state transition (current state number s, next state number n, input symbol x, output symbol y, weight w) with WFST is represented as <s → n, x: y / w>.
Starting from S101, empty lists H and H ′ are created in S102.
In S103, the hypothesis (0, φ, 0) in the hypothesis list H is inserted.

Read symbol “a”
The symbol a is read in S104, and the hypothesis (0, φ, 0) is extracted from the hypothesis list H in S105. A state transition list E including a state transition <0 → 0, a: d / 0.5> whose input symbol is equal to a is created from the current state 0 of this hypothesis.
Since E = φ is not satisfied in S106, the process proceeds to S107, the state transition <0 → 0, a: d / 0.5> is extracted, and a new hypothesis (0, d, 0.5) is generated in S108. Insert into H '.
Returning to S106, since E = φ, the process proceeds to S110, and since H = φ, the process proceeds to S111. The element (0, d, 0.5) of H ′ is moved to H, and since the next input symbol exists in S112, the process returns to S104.
Subsequently, the symbol a is read in S104, and the hypothesis (0, d, 0.5) is extracted from the hypothesis list H in S105. A state transition list E including a state transition <0 → 0, a: d / 0.5> whose input symbol is equal to a is generated from the current state 0 of this hypothesis.
Since E = φ is not satisfied in S106, the process proceeds to S107, and the state transition <0 → 0, a: d / 0.5> is extracted from E. A new hypothesis (0, dd, 1) is generated in S108, and inserted into H ′ in S109.
Returning to S106, since E = φ, the process proceeds to S110, and since H = φ, the process proceeds to S111. The element (0, dd, 1) of H ′ is moved to H, and since the next input symbol exists in S112, the process returns to S104.

Reading the symbol “b” Subsequently, the symbol b is read in S104, and the hypothesis (0, dd, 1) is extracted from the hypothesis list H in S105. A state transition list E including state transitions <0 → 1, b: c / 0.3> and <0 → 2, b: b / 1> whose input symbol is equal to b is created from the current state 0 of this hypothesis.
Since E = φ is not satisfied in S106, the process proceeds to S107, and the state transition <0 → 1, b: c / 0.3> is extracted from E. A new hypothesis (1, ddc, 1.3) is generated in S108, and inserted into H ′ in S109.
Returning to S106, since E = φ is not established, the process proceeds to S107, and the state transition <0 → 2, b: b / 1> is extracted from E. A new hypothesis (2, ddb, 2) is generated in S108 and inserted into H ′ in S109.
Returning to S106, since E = φ, the process proceeds to S110, and since H = φ, the process proceeds to S111. Elements (1, ddc, 1.3) and (2, ddb, 2) of H ′ are moved to H, Since the next input symbol exists in S112, the process returns to S104.

Reading Symbol “c” Subsequently, the symbol c is read in S104, and the hypothesis (1, ddc, 1.3) is extracted from the hypothesis list H in S105. A state transition list E including a state transition <1 → 3, c: b / 1> whose input symbol is equal to c is created from the current state 1 of this hypothesis.
Since E = φ is not satisfied in S106, the process proceeds to S107, and the state transition <1 → 3, c: b / 1> is extracted from E. A new hypothesis (1, ddcb, 2.3) is generated in S108, and inserted into H ′ in S109.
Returning to S106, since E = φ, the process proceeds to S110, and since H ≠ φ, the process returns to S105, and the hypothesis (2, ddb, 2) is extracted from the hypothesis list H. A state transition list E including a state transition <2 → 3, c: a / 0.6> whose input symbol is equal to c is created from the current state 2 of this hypothesis.
Since E = φ is not satisfied in S106, the process proceeds to S107, and the state transition <2 → 3, c: a / 0.6> is extracted from E. A new hypothesis (3, ddba, 2.6) is generated in S108, and inserted into H ′ in S109. At this time, the hypothesis (3, ddcb, 2.3) is already included in H ′, and since the hypothesis (3, ddba, 2.6) has reached the same state 3, The small hypothesis (3, ddcb, 2.3) is left, and the hypothesis (3, ddba, 2.6) is deleted from the list.
Returning to S106, since E = φ, the process proceeds to S110, and since H = φ, the process proceeds to S111, the element H '(3, ddcb, 2.3) is moved to H, and the next input symbol does not exist in S112. Proceed to S113.
In S113, since the arrival state 3 of the hypothesis (3, ddcb, 2.3) in H is an end state, the end weight is added to (3, ddcb, 2.8), and this hypothesis reaches the end state. This is the only hypothesis that the accumulated weight is minimum, so that the output symbol string ddcb is output as the conversion result, and the symbol string conversion process is terminated in S114.

を計算し、このＷ（Ｘ）を与えるＹとＺの組のうちＺを記号列変換結果とする。 (Conversion of symbol string by two WFST)
Next, let us consider a problem in which there are two WFSTs and the symbol strings are converted in order. That is, when there are two WFSTs, A and B, and an input symbol string X is given, the symbol string X is first converted to the symbol string Y using A, and the symbol string Y is converted to the input symbol of B. This means that the output symbol string Z is further converted as a string.
In WFST theory, the problem is that when transforming X by A, all possible Ys that can be its output symbol string are considered as B input symbol strings, and possible B outputs for those input symbol strings From the set of symbol strings Z, there is a problem of obtaining a conversion result that minimizes the sum of the cumulative weight of the state transition process in A and the cumulative weight of the state transition process in B. Therefore,

And Z of the set of Y and Z giving this W (X) is taken as the symbol string conversion result.

(Example of WFST converting kana symbol string to kanji symbol string and giving weight to kanji concatenation)
As an example, consider the two WFSTs shown in FIGS. FIG. 5 is an example of a WFST that converts a kana symbol string into a kanji symbol string. However, when the symbol “ε” appearing in FIG. 5 is on the left side of “:”, the state transition is performed without an input symbol, and when it is on the right side, nothing is output in the state transition.
This WFST, for example, converts the symbol string “A, Me” into “rain” by the state transition processes 0, 1, 5 (cumulative weight 1), and “A, Me, Da, Ma” is the state transition process 0, It is converted into “rain, ball” by 1, 5, 0, 4, 5 (cumulative weight 3). However, in Japanese, the kanji corresponding to “Amedama” is generally “Kamatama” rather than “Ametama”, but the WFST in FIG. 5 outputs the symbol string “列, ball”. The cumulative weight of state transition processes 0, 2, 5, 0, 4, and 5 is 4, whereas the cumulative weight to be converted to “raindrop” is 3, so that “rain, The result is a “ball”.

  On the other hand, FIG. 6 is a WFST that gives weight to kanji connection. In this WFST, the input symbol and the output symbol are the same in all state transitions, that is, the symbol string is output as it is without being converted. The weight is small when it seems to be large, and the weight is large when it is not likely. For example, the cumulative weights of the state transition processes 0, 1, and 3 that accept the input symbol string “rain, descend” are 0, and the cumulative weights of the state transition processes 0, 2, and 3 that accept the input symbol string “飴, descend” 3 indicates that the input symbol string “rain, rain” is the most likely connected symbol string in Japanese.

When the symbol string is converted using the WFST of FIGS. 5 and 6, for example, if the kana symbol string “A, Me, Da, Ma” is input, the output symbol string obtained from the WFST of FIG. , “Rain, ball” may have a cumulative weight of 3 and “carp, ball” may have a cumulative weight of 4. When these output symbol strings are respectively input to the WFST in FIG. 6, the cumulative weight 5 for “rain, ball” is 1, the cumulative weight for “carp, ball” is 1, and the cumulative weight by the two WFSTs in FIG. 5 and FIG. The sum of is calculated as follows.
“A, Me, Da, Ma” → Cumulative weight in “Rain, Jade” 3 + 5 = 8
Cumulative weight in “A, Me, Da, Ma” → “Aoi, Jade” 4 + 1 = 5
As a result of comparing these accumulated weights, an output symbol string having a smaller accumulated weight is “飴, ball”. In order to obtain such a conversion result, the information of WFST in FIG. 6 having weights related to kanji connection is indispensable.

(Conventional symbol string conversion using two WFSTs)
FIG. 7 shows an example of conventional symbol string conversion using two WFSTs.
First, the WFST used in the preceding stage when performing two-stage symbol string conversion using two WFSTs is called the preceding WFST, and the WFST used in the succeeding stage is called the succeeding WFST. A “hypothesis” in the case of using two WFSTs means that a certain symbol string is sequentially input, and a state transition is performed from the initial state in the previous stage WFST by the input symbol string in the previous stage WFST. Represents a set of one state transition process that may be repeated, and one state transition process when the output symbol string corresponding to the state transition process of the preceding WFST is used as the input symbol string of the succeeding WFST And

  The symbol string input unit 103 in FIG. 7 sequentially reads the symbols and sends them to the hypothesis developing unit 104. In the hypothesis developing unit 104, a set of hypotheses for the symbol string read up to now is obtained according to the symbol received from the symbol string input unit 103, the preceding WFST read from the preceding WFST storage unit 201, and the subsequent WFST read from the subsequent WFST storage unit 202. A new hypothesis is generated by updating the state transition process of each hypothesis using the newly received symbol, and is sent to the hypothesis narrowing unit 105. The hypothesis narrowing unit 105 narrows down the hypotheses by deleting hypotheses other than the hypothesis having the smallest cumulative weight among the hypotheses reaching the same state from the hypothesis set received from the hypothesis developing unit 104. If the input symbol string has been read to the end, the output symbol string corresponding to the hypothesis with the smallest cumulative weight is sent to the symbol string output unit 106. If the input symbol string has not been read to the end, the hypothesis is sent to the hypothesis developing unit 104. The symbol string output unit 106 outputs the output symbol string received from the hypothesis narrowing unit 105.

Even when two WFSTs are used, the calculation can be performed by the same method as the minimum cost search in the above-mentioned one WFST. As shown in FIG. 7, the hypothesis developing unit 104 reads the preceding WFST (referred to as WFST A) from the preceding WFST storage 201 and the subsequent WFST (referred to as WFST B) from the subsequent WFST storage 202. The hypothesis is updated according to these two WFSTs, and a new hypothesis is generated. A procedure for performing a minimum cost search using two WFSTs A and B is shown. However, unlike the case of one WFST, the hypothesis h is an arrival state s _A [h] in WFST A and an arrival state s B in WFST _B. Assume that there is a set [h]. When a hypothesis is inserted into the hypothesis list, if there is a hypothesis in the hypothesis list where the arrival state in WFST A and the arrival state in WFST B are both equal, only the smaller cumulative weight is left in the hypothesis list to narrow down the hypotheses .

(Conventional symbol string conversion procedure using two WFSTs)
A conventional symbol string conversion procedure using these two WFSTs is shown in FIG.
Hereinafter, the procedure of FIG. 8 will be described in comparison with an embodiment (FIG. 7) of a conventional symbol string conversion using two WFSTs.
Starting from step S201, as a default setting, empty hypothesis lists H and H ′ are generated in step S202. In S203, an initial hypothesis h is generated, and s _A [h] = 0 (initial state of WFST A), s _B [h] = 0 (initial state of WFST B), W [h] = 0, O [ h] = φ (empty symbol string) and insert into hypothesis list H.
In step S204, the symbol string input unit 103 reads one symbol and substitutes the symbol for x.

The following steps S205 to S208 are executed in the hypothesis developing unit 104.
In step S205, one hypothesis is extracted from the hypothesis list H and assigned to h, and the state transition e _A whose input symbol is equal to x from the state s _A [h] and the input symbol o [e _A from the state s _B [h]. ] _A list P of a set (e _A , e _B ) of state transitions e _B equal to] is prepared, provided that when o [e _A ] = ε, e _B = φ (empty state transition) ( e _A , φ) is inserted into list P.
If P = φ (empty list) in step S206, the process proceeds to S210. Otherwise, the process proceeds to S207, and one set of state transitions is extracted from P and assigned to (e _A , e _B ).
In step S208, a new hypothesis f is generated, and s _A [f] = n [e _A ] is set. Furthermore, if e _B = φ, then s _B [f] = s _B [h], W [f] = W [h] + w [e _A ], O [f] = O [h], and so on Otherwise, s _A [f] = n [e _A ], s _B [f] = n [e _B ], W [f] = W [h] + w [e _A ] + w [e _B ], O [f] = O [h] · o [e _B ] (when e _B = φ, since the output symbol of the state transition e _A is ε, the state transition does not occur in WFST B, so the arrival state s _B [ f] does not change and s _B [f] = s _B [h], and since there is no symbol output from WFST B, the output symbol string does not change and O [f] = O [h].
In step S209, the hypothesis narrowing unit 105 narrows down the hypotheses by inserting the hypothesis f into the hypothesis list H ′.

Returning to step S206, a hypothesis is developed for the next set of state transitions.
In step S210, if H = φ (all hypotheses have been expanded), the process proceeds to S211. Otherwise, return to S205 to develop the next hypothesis.
In step S211, all the elements of the newly generated hypothesis list H ′ are moved to the already empty H, and the process proceeds to S212.
In step S212, if there is a next input symbol in the symbol string input unit 103, the process returns to S204. Otherwise, it is determined that all the input symbol strings have been read, and the process proceeds to S213.
In step S213, a hypothesis h (s _A [h] εF _A and s _B [h] εF _B , F _A , F _A is a set of end states of WFST A, _B represents the set of end states of WFST B), and the end weight (ρ (s _A [h]) + ρ (s _B [h])) is added to the accumulated weight W [h], and the accumulated weight (W [ h]) selects the minimum hypothesis h ′ and outputs the output symbol string O [h ′] as a conversion result in the symbol string output unit 106.
In step S214, the conventional symbol string conversion procedure using two WFSTs is terminated.

(The process of obtaining the output symbol string when the input symbol string “A, Me, Da, Ma” is given)
In accordance with this symbol string conversion procedure, the process of obtaining the output symbol string when the input symbol string “A, Me, Dama,” is given as WFST A in FIG. 5 and WFST B in FIG. 6 will be described in order. However, a certain hypothesis (current state number s _A of WFST A, current state number s _B of WFST B, the output symbol string O, cumulative weight W) If there is, the hypothesis _{(s A, s B, O} , W ).
Starting from S201, empty hypothesis lists H and H ′ are created in S202.
A hypothesis (0, 0, φ, 0) is inserted into the hypothesis list H by S203.

"A" reading
The symbol “a” is read in S204, and a hypothesis (0, 0, φ, 0) is extracted from the hypothesis list H in S205. A pair of state transitions in which the input symbol is “A” from the current state 0 of WFST A of this hypothesis and a state transition in which the input symbol from the current state 0 of WFST B is equal to the output symbol of the state transition of WFST A (<0 → 1 , Oh: Rain / 0>, <0 → 1, Rain: Rain / 0>),
(<0 → 1, A: Rain / 0>, <0 → 3, Rain: Rain / 0>),
(<0 → 2, A: 飴 / 0>, <0 → 2, 飴: 飴 / 0>),
(<0 → 2, A: 飴 / 0>, <0 → 4, 飴: 飴 / 0>)
A list P of state transition sets including is created.
Since P = φ is not satisfied in S206, the process proceeds to S207, and a set of state transitions (<0 → 1, A: rain / 0>, <0 → 1, rain: rain / 0>) is extracted from P, and a new one is generated in S207. A hypothesis (1, 1, rain, 0) is generated and inserted into H ′ in S209.
Returning to S206, since P = φ is not established, the process proceeds to S207, and a set of state transitions (<0 → 1, A: rain / 0>, <0 → 3, rain: rain / 0>) is extracted from P, and is newly added in S208. A hypothesis (1, 3, rain, 0) is generated and inserted into H ′ in S209.
Returning to S206, since P = φ is not satisfied, the process proceeds to S207, and a set of state transitions (<0 → 2, A: 飴 / 0>, <0 → 2, 飴: 飴 / 0>) is extracted from P, and new in S208 A hypothesis (2, 2, 飴, 0) is generated and inserted into H ′ in S209.
Returning to S206, since P = φ is not established, the process proceeds to S207, and a set of state transitions (<0 → 2, A: 飴 / 0>, <0 → 4, 飴: 飴 / 0>) is extracted from P, and new in S208 A hypothesis (2, 4, 飴, 0) is generated and inserted into H ′ in S209.
Returning to S206, since P = φ, the process proceeds to S210. Since H = φ, the process proceeds to S211 and the hypotheses (1,1, rain, 0), (1,3, rain, 0), (2,2, 飴, 0), (2,4, 飴) in H ′ , 0)
Is moved to H, and since there is a next input symbol in S212, the process returns to S204.

Next, the symbol “me” is read in S204, and a hypothesis (1, 1, rain, 0) is extracted from the hypothesis list H in S205. A pair of state transitions in which the input symbol is “m” from the current state 1 of WFST A and a state transition in which the input symbol from the current state 1 of WFST B is equal to the output symbol of the state transition of WFST A (<1 → 5 Me: ε / 1>, φ)
A list P of state transition sets including is created. Here, since the output symbol of the state transition <1 → 5, fifth: ε / 1> of WFST A is ε, the state transition of WFST B is φ.
Since P = φ is not satisfied in S206, the process proceeds to S207, a set of state transitions (<1 → 5, ε / 1>, φ) is extracted from P, and a new hypothesis (5, 1, rain, 1) is obtained in S208. And is inserted into H ′ in S209.
Returning to S206, since P = φ, the process proceeds to S210. Since H ≠ φ, the process returns to S205, and the hypothesis (1, 3, rain, 0) is extracted from the hypothesis list H. A pair of state transitions in which the input symbol is “M” from the current state 1 of this hypothesis and state transitions whose input symbol is equal to the output symbol of the state transition of WFST A from the current state 3 of WFST B (<1 → 5: ε / 1>, φ)
A list P of state transition sets including is created.
Since P = φ is not satisfied in S206, the process proceeds to S207, a set of state transitions (<1 → 5: ε / 1>, φ) is extracted from P, and a new hypothesis (5, 3, rain, 1) is obtained in S208. And is inserted into H ′ in S209.
Returning to S206, since P = φ, the process proceeds to S210. Since H ≠ φ, the process returns to S205, and the hypothesis (2, 2, 飴, 0) is extracted from the hypothesis list H. A set of state transitions in which the input symbol is “m” from the current state 2 of this hypothesis and a state transition in which the input symbol from the current state 2 of WFST B is equal to the output symbol of the state transition of WFST A
(<2 → 5: ε / 2>, φ)
A list P of state transition sets including is created.
Since P = φ is not satisfied in S206, the process proceeds to S207, a set of state transitions (<2 → 5: ε / 2>, φ) is extracted from P, and a new hypothesis (5, 2, 飴, 2) is obtained in S208. And is inserted into H ′ in S209.
Returning to S206, since P = φ, the process proceeds to S210. Since H ≠ φ in S210, the process returns to S205, and a hypothesis (2, 4, 飴, 0) is extracted from the hypothesis list H. A set of state transitions in which the input symbol is “m” from the current state 2 of this hypothesis and a state transition in which the input symbol from the current state 2 of WFST B is equal to the output symbol of the state transition of WFST A
(<2 → 5: ε / 2>, φ)
A list P of state transition sets including is created.
Since P = φ is not satisfied in S206, the process proceeds to S207, a set of state transitions (<2 → 5: ε / 2>, φ) is extracted from P, and a new hypothesis (5, 4, 飴, 2) is obtained in S208. And is inserted into H ′ in S209.
Returning to S206, since P = φ, the process proceeds to S210. Since H = φ, the process proceeds to S211 and the hypotheses (5, 1, rain, 1), (5, 3, rain, 1), (5, 2, hail, 2), (5, 4, 4) in H ′.飴, 2) is moved to H, and since the next input symbol exists in S212, the process returns to S204.

Reading “DA” Subsequently, the symbol “DA” is read in S204, and a hypothesis (5, 1, rain, 1) is extracted from the hypothesis list H in S205. A pair of state transitions in which the input symbol is “da” from the current state 5 of WFST A of this hypothesis and a state transition in which the input symbol from the current state 1 of WFST B is equal to the output symbol of the state transition of WFST A (<0 → 4 , Da: Jade / 1>, <1 → 4, Jade: Jade / 5>)
A list P of state transition sets including is created. Here, since it is possible to transition from the current state 5 of WFST A to state 0 without an input symbol, the state transition of WFST A included in P is <0 → 4: ball / 1>.
Since P = φ is not satisfied in S206, the process proceeds to S207, and a set of state transitions (<0 → 4, ball: 1/1, <1 → 4, ball: ball / 5>) is extracted from P, and a new one is created in S208. A hypothesis (4, 4, rainball, 7) is generated and inserted into H ′ in S209.
Returning to S206, since P = φ, the process proceeds to S210. Since H ≠ φ, the process returns to S205, and the hypothesis (5, 3, rain, 1) is extracted from the hypothesis list H. In this hypothesis, there is no set of state transitions in which the input symbol is “da” from the current state 5 of WFST A and state transitions in which the input symbol from the current state 3 of WFST B is equal to the output symbol of the state transition of WFST A, so P = φ And
Since P = φ in S206, the process proceeds to S210, and because H ≠ φ, the process returns to S205.
In S205, a hypothesis (5, 2, 飴, 2) is extracted from the hypothesis list H. A pair of state transitions in which the input symbol from the current state 5 of the hypothesis WFST A is “da” and a state transition in which the input symbol from the current state 2 of WFST B is equal to the output symbol of the state transition of WFST A (<0 → 4, Da: ball / 1>, <2 → 4, ball: ball / 1>)
A list P of state transition sets including is created.

Since S is not P = φ in S206, the process proceeds to S207, and a set of state transitions (<0 → 4, ball: 1/1, <2 → 4, ball: ball / 1>) is extracted from P, and a new one is created in S208. A hypothesis (4, 4, jasper, 4) is generated and inserted into H ′ in S209. However, since there is a hypothesis (4, 4, rainball, 7) in H ′ and the set of the arrival state in WFST A and the arrival state in WFST B is the same, the hypothesis (4, 4, 4, Leave jasper 4) and delete (4, 4, rainball 7).
Returning to S206, since P = φ, the process proceeds to S210. Since H ≠ φ, the process returns to S205, and the hypothesis (5, 4, 飴, 2) is extracted from the hypothesis list H. Since there is no set of state transitions in which the input symbol is “da” from the current state 5 of this hypothesis, and there is no set of state transitions from the current state 4 of WFST B which is the output symbol of the state transition of WFST A, P = φ To do.
Since P = φ in S206, the process proceeds to S210, and because H = φ, the process proceeds to S211.
The hypothesis (4, 4, jasper, 4) in H ′ is moved to H, and since the next input symbol exists in S212, the process returns to S204.

Next, the symbol “ma” is read in S204, and the hypothesis (4, 4, jasper, 4) is extracted from the hypothesis list H in S205. A set of state transitions in which the input symbol is “ma” from the current state 4 of WFST A of this hypothesis, and a state transition in which the input symbol from the current state 4 of WFST B is equal to the output symbol of the state transition of WFST A (<4 → 5 , MA: ε / 1>, φ)
A list P of state transition sets including is created. Here, since the output symbol of state transition <4 → 5, or ε / 1> of WFST A is ε, the state transition of WFST B is φ.
Since P = φ is not satisfied in S206, the process proceeds to S207, and a set of state transitions (<4 → 5, or ε / 1>, φ) is extracted from P, and a new hypothesis (5, 4, Kodama, 5) is extracted in S208. ) And inserted into H ′ in S209.
Returning to S206, since P = φ, the process proceeds to S210. Since H = φ, the process proceeds to S211 and the hypothesis (5, 4, jasper, 5) in H ′ is moved to H. Since there is no next input symbol in S212, the process proceeds to S213.
In S213, the arrival state 5 in WFST A and the arrival state 4 in WFST B of the hypothesis (5, 4, jasper, 5) in H are both end states, and the respective end weights are added (5, 4). , Kodama, 5), and since this hypothesis is the only hypothesis that has reached the end state, the output symbol string “Kadama” of the hypothesis is output as a conversion result, and the symbol string conversion process is terminated in S214. .
As another means for converting a symbol string using two WFSTs, there is a method of combining two WFSTs in advance to form one WFST and applying a symbol string conversion procedure for one WFST. A method for synthesizing WFST is disclosed in Non-Patent Document 1, for example.
However, when the WFST is combined, the number of states and state transitions of the WFST may become very large because the state and state transition can be performed with respect to the combination of two state transitions. For this reason, when the WFST is handled by a computer, it may be difficult to execute symbol string conversion due to restrictions on the memory size and the like.
E. Roche and Y. Schabes, “Finite-State Language Processing,” mit Press, 1997, Chapter 15 “Speech Recognition by Composition of Weighted Finite Automata” Nagao Makoto "Iwanami Lecture Software 14: Knowledge and Reasoning", Iwanami Shoten, 109-113

  In the conventional method of performing a minimum cost search using two WFSTs, when each hypothesis is updated and a new hypothesis is generated, a state transition that is possible in the preceding WFST and a state transition that is possible in the subsequent WFST Since new hypotheses are generated for all the combinations, there is a problem that the number of hypotheses increases and the amount of calculation increases.

The present invention
A symbol string input section for sequentially reading the symbol strings;
A front-stage weighted finite state converter used in the preceding stage and a back-stage weighted finite state converter used in the subsequent stage for converting the symbol string in two stages using two weighted finite state converters that convert the symbol string by state transition A symbol string conversion unit for converting a symbol string using and
A symbol string output unit that outputs a conversion result obtained by the latter-stage weighted finite state converter;
When the symbols are sequentially read from the symbol string input unit and the input symbol string has been read, cumulative weight values for state transitions respectively applied to the weighted finite state converters at the preceding and succeeding stages of the symbol string converter ( In the symbol string conversion method for outputting the output symbol string corresponding to the state transition process of the latter-stage weighted finite state converter with the smallest (cumulative weight) from the symbol string output unit,
While reading the symbols in order, the cumulative weight for the hypothesis representing one state transition process of the preceding stage weighted finite state transformer, the output symbol string in the hypothetical state transition process, and the input symbol string of the latter stage weighted finite state transformer The state transition process that minimizes the cumulative weight among the possible state transition processes in the latter-stage weighted finite state converter is corrected, and the cumulative weight is corrected by adding it to the hypothetical cumulative weight,
When all input symbol strings are read, the latter weighted finite state when the hypothesis with the smallest cumulative weight and the output symbol string corresponding to the state transition process of that hypothesis are used as the input symbol string of the latter weighted finite state converter An output symbol string corresponding to a state transition process having a minimum cumulative weight among possible state transition processes in the converter is used as a symbol string conversion result.
The symbol string is efficiently converted by this symbol string conversion method.

従来の記号変換方法を用いた音声認識装置と比べて、処理時間は約３分の２、記号変換手順の違いに関係のない音響モデルによる音響スコア計算の計算部分を除けば約３分の１に短縮された。また、どちらの方法においても音声認識結果は同一であったことから、本発明を用いた場合の音声認識における精度に劣化はない。 According to the present invention, the procedure for converting a symbol string using two WFSTs is almost the same as the procedure for searching using only the preceding WFST (FIG. 4) except for the hypothesis correction unit, and the conventional procedure (FIG. 8). ) Can reduce the amount of calculation. Although there is a calculation load in the hypothesis correction unit, hypothesis correction calculation is performed only when the output symbol is not ε in the state transition of the preceding WFST. Therefore, the more the ratio that the output symbol of the preceding WFST is ε, the more processing is performed. The effect of volume reduction is increased.
Table 1 shows the processing time required for the speech recognition processing when the subject inputs speech that reads 100 sentences in a newspaper article using the speech recognition method according to the present invention. However, the processing time is a value (actual time ratio) obtained by dividing the actual utterance time by the processing time required for the speech recognition processing. The vocabulary size of the word pronunciation dictionary is 20,000. In the experiment, an IBM compatible machine, a CPU with a Pentium (registered trademark) III, and a computer with a clock frequency of 800 MHz were used.

Compared to a speech recognition apparatus using a conventional symbol conversion method, the processing time is about two thirds, and about one third except for the calculation part of the acoustic score calculation by an acoustic model that is not related to the difference in the symbol conversion procedure. Shortened to Moreover, since the speech recognition result is the same in both methods, there is no deterioration in the accuracy in speech recognition when the present invention is used.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
(Symbol converter)
FIG. 9 is a diagram showing an embodiment of the present invention.
The symbol string input unit 103 sequentially reads the symbol strings and sends them to the hypothesis developing unit 104. The hypothesis expansion unit 104 performs one possible state transition process for the input symbol string read up to the present time according to the symbol received from the symbol input unit 103 and the previous WFST read from the previous WFST storage unit 201. When expressed as a hypothesis, a new hypothesis is generated by updating the state transition process of each hypothesis using a newly received symbol in a set of existing hypotheses, and the hypothesis correction unit 301 transmits the new hypothesis. The hypothesis correction unit 301 sets the output symbol string in the hypothesis state transition process as the input symbol string of the subsequent WFST for each hypothesis received from the hypothesis expansion unit 104 in accordance with the subsequent WFST read from the subsequent WFST storage unit 202. The state transition process having the smallest accumulated weight among the possible state transition processes in the subsequent WFST is obtained, corrected by adding the accumulated weight to the accumulated weight of the hypothesis, and sent to the hypothesis narrowing unit 105. The hypothesis narrowing unit 105 narrows down the hypotheses by leaving only the hypothesis with the smallest cumulative weight among the hypotheses having the same state transition process from the hypothesis received from the hypothesis correction unit 301 and deleting other hypotheses. . If the input symbol string has been read to the end, the state transition possible in the subsequent WFST when the hypothesis having the minimum cumulative weight and the output symbol string corresponding to the state transition process of the hypothesis are used as the input symbol string of the subsequent WFST An output symbol string for the state transition process having the smallest accumulated weight in the process is sent to the symbol string output unit 105. If the input symbol string has not been read to the end, the remaining hypothesis is sent to the hypothesis developing unit 104. The symbol string output unit 105 outputs the output symbol string received from the hypothesis narrowing unit 105.

(Symbol string conversion procedure)
FIG. 10 shows a specific procedure for converting a symbol string in one embodiment of the present invention.
First, in the present invention, when a “hypothesis” in the preceding WFST and a symbol string output from the state transition process of the hypothesis are used as an input symbol string of the succeeding WFST, one possible state transition of the succeeding WFST The process is distinguished from “hypothesis” and called “auxiliary hypothesis”. Since at least one auxiliary hypothesis always exists for one hypothesis, in this embodiment, a set of auxiliary hypotheses for the hypothesis h is represented by an auxiliary hypothesis list L [h].
Further, when there is a hypothesis h, s _A [h] is the state reached last in the state transition process of hypothesis h, and W [h] is the (corrected) cumulative weight in the state transition process of hypothesis h. On the other hand, when there is an auxiliary hypothesis g (gεL [h]) with respect to the hypothesis h, s _B [g] is the last state reached in the state transition process of the auxiliary hypothesis g, and W [g] is the hypothesis h. The sum of the cumulative weight in the state transition process and the cumulative weight in the state transition process of the auxiliary hypothesis g, O [g], is a symbol string output in the state transition process of the auxiliary hypothesis g.

Hereinafter, the procedure of FIG. 10 will be described in comparison with the embodiment of FIG.
Starting from step S301, as initial settings, empty hypothesis lists H and H ′ are generated in step S302. In step S303, an initial hypothesis h is generated, and s _A [h] = 0, W [h] = 0 and L [h] = φ (empty list) are inserted into the hypothesis list H.
Also, an auxiliary hypothesis g is generated, and s _B [g] = 0, W [g] = 0, and L [g] = φ (empty symbol string) are inserted into the auxiliary hypothesis list L [h] of h. .
In step S304, the symbol string input unit 103 reads one symbol and substitutes the symbol for x.
The following steps S305 to S308 are executed in the hypothesis developing unit 104.
In step S305, one hypothesis is extracted from the hypothesis list H and substituted into h, and a state transition list E _A that can be transitioned with the input symbol x is generated from the state s _A [h].
If E _A = φ (empty list) in step S306, the process proceeds to S311. Otherwise, the process proceeds to S307.
In step S307, fetches one state transition from E _A, is substituted into e _A,
In step S308, a new hypothesis f is generated, and s _A [f] = n [e _A ], W [f] = W [h] + w [e _A ], and L [f] = L [h].
In step S309, in the hypothesis correction unit 301, if o [e _A ] ≠ ε, means for correcting the hypothesis f (described separately after the description of this procedure) is executed.
In step S310, the hypothesis narrowing unit 105 narrows down the hypotheses by inserting the hypothesis f into the hypothesis list H ′.

In step S311, if H = φ (all hypotheses have been expanded), the process proceeds to S312. Otherwise, return to S305 and develop the next hypothesis.
In step S312, all the elements of the newly generated hypothesis list H ′ are moved to the already empty H, and the process proceeds to S313.
In step S313, if there is a next input symbol in the symbol string input unit 103, the process returns to S304. Otherwise, it is determined that all the input symbol strings have been read, and the process proceeds to S314.
In step S314, a hypothesis h that has reached the end state (s _A [h] εF _A ) and has the smallest cumulative weight (W [h]) is selected from the hypothesis list H, and the hypothesis is further selected. From the auxiliary hypothesis list L [h] corresponding to, a hypothesis g ′ that has reached the end state (s _B [g] ∈F _B ) and has the smallest cumulative weight (W [g]) The output symbol string O [g ′] is selected and output in the symbol string output unit 106 as a conversion result.
In step S315, the symbol string conversion procedure in the embodiment of the present invention is terminated.

となり、先に述べた式（２）における二つのＷＦＳＴの累積重みの和の最小値と一致する。
従って、補正された累積重みを用いて、それを最小とする前段ＷＦＳＴの出力記号列Ｙを求めて、そのＹを後段ＷＦＳＴの入力記号列としたときの可能な状態遷移過程の中で累積重みが最小となる状態遷移過程に対応する出力記号列を記号列変換結果となる。 (Hypothesis correction procedure)
Next, a hypothesis correction procedure in the hypothesis correction unit 301 will be described.
The hypothesis correction in the present invention is a state transition process that is possible in the subsequent WFST when the cumulative weight for the hypothetical state transition process in the preceding WFST is the output symbol string in the hypothetical state transition process as the input symbol string of the subsequent WFST. It means that a state transition process in which the cumulative weight is the smallest is obtained and the cumulative weight is added to the hypothetical cumulative weight. When the input symbol string X is read, the corrected cumulative weight of the hypothesis that accepts the symbol string X and outputs the symbol string Y is:

Thus, it agrees with the minimum value of the sum of the cumulative weights of the two WFSTs in the above-described equation (2).
Therefore, using the corrected accumulated weight, the output symbol string Y of the preceding WFST that minimizes the obtained weight is obtained, and the accumulated weight in the possible state transition process when Y is used as the input symbol string of the succeeding WFST. The output symbol string corresponding to the state transition process that minimizes is the symbol string conversion result.

（Ａ３）後段ＷＦＳＴの状態ｓ_B[ｇ]からｓ_B[ｊ]への状態遷移ｅ_Bにおける重み：ｗ[ｅ_B]
従って、補助仮説ｊの累積重みはＷ[ｊ]＝Ｗ[ｇ]＋ＡＷ[ｆ]＋ｗ[ｅ_B]のように計算する。
そして、仮説ｆの補正は、Ｗ[ｆ]を新たに生成された補助仮説ｊの累積重みＷ[ｊ]の中で最小値

In the present invention, correction of hypotheses is efficiently performed using an auxiliary hypothesis list associated with each hypothesis.
The hypothesis to be corrected is f, the auxiliary hypothesis list is L [f], and the output symbol in the last state transition of the hypothesis f is y.
An auxiliary hypothesis j from the state s _B [g] (g represents one auxiliary hypothesis included in L [f]) to the next state n [e _B ] through the state transition e _B by the input symbol y And its arrival state is s [j] = n [e _B ]. In addition, in the state transition process of hypothesis f, let t be the arrival state when the previous correction calculation was performed before the last state transition (e _A ) of f.
At this time, the cumulative weight W [j] of the auxiliary hypothesis j can be calculated by adding the following three elements.
(A1) Cumulative weight of auxiliary hypothesis g generated in state t in the state transition process of hypothesis f:
This value is equal to W [g]. That is, in the state transition process whose output symbol is ε, the auxiliary hypothesis list is directly inherited by the hypothesis of the subsequent state transition process, so that the auxiliary hypothesis list generated in the state t is equal to L [f], The cumulative weight W [g] of each auxiliary hypothesis in the hypothesis list L [f] is used as it is.
(A2) Weights accumulated from state t to state s _A [f]:
However, in the state transition process in which the output symbol is ε, the auxiliary hypothesis list is carried over to the newly generated hypothesis as it is, so that the corrected cumulative weight of the hypothesis reaching the state t is the current auxiliary hypothesis list L The smallest cumulative weight in [f], ie

(A3) the weight in the state transition e _B from subsequent WFST state s _B [g] s to _{B [j]: w [e} B]
Therefore, the cumulative weight of the auxiliary hypothesis j is calculated as W [j] = W [g] + AW [f] + w [e _B ].
Then, the correction of the hypothesis f is performed by reducing the minimum value among the cumulative weights W [j] of the newly generated auxiliary hypothesis j.

(Hypothesis correction procedure)
The hypothesis correction procedure will be described below with reference to FIG.
This hypothesis correction procedure is executed in step S308 shown in FIG. The hypothesis corrected in this procedure is f, and the output symbol from the previous stage WFST is y (= o [e _A ]).
In step S401, a procedure for correcting hypothesis f is started.
In step S402, an empty auxiliary hypothesis list L ′ is generated.
In step S403, the weight correction term AW [f] is obtained as AW [f] = W [f] −W [g ′]. However, g ′ indicates an auxiliary hypothesis having the smallest cumulative weight in the auxiliary hypothesis list L [f] of the hypothesis f.
In step S404, one auxiliary hypothesis is extracted from L [f] and substituted into g. A list E _{B of} state transitions that can be transitioned with the input symbol y is generated from the state s _B [g], and the process proceeds to step S405.
In step 405, if E _B = φ (empty list), the process proceeds to S406. Otherwise, go to S409.
In step S406, it is taken out one state transition from the list E _B, is substituted into e _B.
In step S407, an auxiliary hypothesis j is generated, and s _B [j] = n [e _B ], W [j] = W [g] + AW [f] + w [e _B ], O [j] = O [g ] · O [e _B ].
In step S408, the auxiliary hypothesis j is inserted into the auxiliary hypothesis list L ′.
In step S405, if E _B is empty (the auxiliary hypothesis has been expanded for all state transitions from s _B [g]), the process proceeds to S409. Otherwise, the process returns to S406 and the next state transition is examined.
In step S409, if L [f] is empty (all auxiliary hypotheses have been expanded), the process proceeds to S410. Otherwise, return to S404 and develop the next hypothesis.
In step S410, all the elements of the newly generated auxiliary hypothesis list L ′ are moved to the already empty auxiliary hypothesis list L ′ [f], and the process proceeds to S411.
In step S411, the cumulative weight is set to W [f] = W [g "] for the first auxiliary hypothesis g" in the auxiliary hypothesis list L [f].
In step S412, the hypothesis correction procedure in one embodiment of the present invention is terminated.

のように補正する。この補正は、先に示した仮説の補正手順における（Ａ１）と（Ａ２）の項を加えたものに等しい。そして、
Ｗ[ｈ]＜Ｗ[ｆ] ならば Ｌ[ｈ]＝Ｌ[ｈ]＋Ｌ[ｆ]、ｆを削除
Ｗ[ｈ]＞Ｗ[ｆ] ならば Ｌ[ｆ]＝Ｌ[ｈ]＋Ｌ[ｆ]、ｈを削除
のように補助仮説リストを連結する。 In the present invention, when a hypothesis is inserted into the hypothesis list, as in the conventional method, if there is a hypothesis that reaches the same state of the preceding WFST in the hypothesis list, the hypothesis with the smaller cumulative weight is selected. Leave the list and narrow down the hypotheses. However, since the auxiliary hypotheses may be lost by this hypothesis narrowing down, the auxiliary hypothesis list of the narrowed down hypotheses is connected to the auxiliary hypothesis list of the remaining hypotheses. At that time, it is necessary to correct the cumulative weight of the auxiliary hypotheses in the auxiliary hypothesis list. This is because when hypothesis h and hypothesis f reach the same state in the previous stage WFST, the cumulative weight of the auxiliary hypothesis is

Correct as follows. This correction is equivalent to the addition of the terms (A1) and (A2) in the hypothesis correction procedure described above. And
If W [h] <W [f] L [h] = L [h] + L [f], delete f If W [h]> W [f] L [f] = L [h] + L [ f], concatenate the auxiliary hypothesis lists like h is deleted.

  The symbol conversion procedure (FIG. 10) according to the present invention is almost the same calculation procedure as the procedure (FIG. 4) for searching using only the preceding WFST except for the hypothesis correction procedure (step S309). Compared with FIG. 8), the amount of calculation can be reduced. Although there is a calculation load in the hypothesis correction procedure, the hypothesis correction procedure is executed only when the output symbol is not ε in the state transition of the preceding WFST. Therefore, the higher the ratio that the output symbol of the previous WFST is ε, The effect of reducing the amount of processing increases.

(The process of obtaining the output symbol string when the input symbol string “A, Me, Da, Ma” is given)
Next, according to the symbol string conversion procedure of the present invention, a process of obtaining an output symbol string when an input symbol string “A, M, D, D” is given with WFST A as FIG. 5 and WFST B as FIG. I will explain in order. However, the hypothesis (WFST A current state number s _A, the cumulative weight W, auxiliary hypothesis list L) is (s _A, W, L) expressed as, current state number s _B auxiliary hypothesis (WFST B, output symbols Column O, cumulative weight W) is expressed as (s _B , O, W). The auxiliary hypothesis list L is expressed as {(s _B , O, W), (s ′ _B , O ′, W ′),.
Starting from S301, empty hypothesis lists H and H ′ are created in S302.
A hypothesis (0, 0, {0, φ, 0}) is inserted into the hypothesis list H through S303.

"A" reading
The symbol “a” is read in S304, and a hypothesis (0, 0, {0, φ, 0}) is extracted from the hypothesis list H in S305.
State transition <0 → 1, a: rain / 0> from the current state 0 of this hypothesis WFST A where the input symbol is “a”
<0 → 2, A: 飴 / 0>
Make a list E _A, including.
Since E _A = φ is not satisfied in S306, the process proceeds to S307, where state transition <0 → 1, a: rain / 0> is extracted from E _A , and a new hypothesis (1, 0, {0, φ, 0}) is obtained in S308. Is generated. In S309, since the output symbol of the state transition <0 → 1, a: rain / 0> is not ε, the process proceeds to step S401 for correcting the hypothesis (1, 0, {0, φ, 0}).
In S402, an empty auxiliary hypothesis list L ′ is generated.
In S403, the weight AW = 0.
In S404, the auxiliary hypothesis (0, φ, 0) is extracted from the auxiliary hypothesis list L = {(0, φ, 0)}, and the state transition that can be transitioned from the state 0 of WFST B with the input symbol “rain” <0 → 1 , Rain: Rain / 0>,
<0 → 3, rain: rain / 0>
Make a list E _B, including.
Since E _A is not φ in S405, the process proceeds to S406. State transition from E _A <0 → 1, rain: Rain / 0> insert was removed, auxiliary hypothesis S407 (1, rain, 0) generates the auxiliary hypotheses S408 (1, rain, 0) to L ' To do.
Returning to S405, since E _B = φ is not satisfied, the process proceeds to S406. State transition from E _B <0 → 3, rain: Rain / 0> was removed, S407 auxiliary hypothesis (3, rain, 0) generates, insertion auxiliary hypothesis (3, rain, 0) into L 'in S408 To do.
Returning to S405, since E _B = φ, the process proceeds to S409. Since the auxiliary hypothesis list L = φ, the process proceeds to S410, and the elements (1, rain, 0) and (3, rain, 0) of L ′ are moved to L.
In S411, the cumulative weight of the hypothesis becomes 0, and the correction of the hypothesis is terminated in S412. As a result, the hypothesis is {1, 0, (1, rain, 0), (3, rain, 0)}.

Returning to S310, the hypothesis (1, 0, {(1, rain, 0), (3, rain, 0)}) is inserted into H ′.
Returning to S306, since E _A is not φ, the process proceeds to S307, where state transition <0 → 2, a: 飴 / 0> is extracted from E _A , and a new hypothesis (2, 0, {0, φ, 0} is obtained in S308. ) And the output symbol of the state transition <0 → 2, A: 飴 / 0> is not ε in S309, and the process proceeds to S401 to correct the hypothesis (2, 0, {0, φ, 0}). .
In S402, an empty auxiliary hypothesis list L ′ is generated.
In S403, the weight AW = 0.
In S404, the auxiliary hypothesis (0, φ, 0) is extracted from the auxiliary hypothesis list L = {(0, φ, 0)}, and the state transition that can be transitioned from the state 0 of WFST B with the input symbol “rain” <0 → 2 , 飴: 飴 / 0>,
<0 → 4, 飴: 飴 / 0>
Make a list E _B, including.
Since E _B is not equal to φ in S405, the process proceeds to S406. State transition from E _B <0 → 2, candy: candy / 0> was removed, auxiliary hypothesis S407 (2, candy, 0) generates, insertion auxiliary hypothesis (2, candy, 0) into L 'in S408 To do.
Returning to S405, since E _B is not φ, the process proceeds to S406. State transition from E _B <0 → 4, candy: candy / 0> was removed, auxiliary hypothesis S407 (4, candy, 0) generates, insertion auxiliary hypothesis (4, candy, 0) into L 'in S408 To do.
Returning to S405, since E _B = φ, the process proceeds to S409. Since the auxiliary hypothesis list L = φ, the process proceeds to S410, and the elements (2, 飴, 0) and (4, 飴, 0) of L ′ are moved to L.
In S411, the cumulative weight of the hypothesis becomes 0, and the correction of the hypothesis is terminated in S412. As a result, the hypothesis is (2, 0, {(2, 飴, 0), (4, 飴, 0)}).
Returning to S310, the hypothesis (2, 0, {(2, 飴, 0), (4, 飴, 0)}) is inserted into H ′.
Returning to S306, since E _A = φ, the process proceeds to S311. Since H = φ, the process proceeds to S312 and the hypotheses (1, 0, {(1, rain, 0), (3, rain, 0)}) and (2, 0, {(2, 飴, 0) in H ′ ), (4, 飴, 0)}) are moved to H, and the next input symbol is present in S313, so that the process returns to S304.

Next, the symbol “me” is read in S304, and the hypothesis (1, 0, {(1, rain, 0), (1, 1, rain, 0)}) is extracted from the hypothesis list H in S305. . State transition from the current state 1 of the hypothetical WFST A to the input symbol “M” <1 → 5, M: ε / 1>
Make a list E _A, including.
Since E _A is not φ in S306, the process proceeds to S307, where state transition <1 → 5: ε / 1> is extracted from E _A , and a new hypothesis (5, 1, {(1, rain, 0) is acquired in S308. , (1, rain, 0)}) and proceeds to S309.
Since the output symbol of state transition <1 → 5, ε / 1> is ε, the process proceeds to the next S310 and the hypothesis (5, 1, {(1, rain, 0), (1, rain, 0)} ) Is inserted into H ′.
Returning to S306, since E _A = φ, the process proceeds to S311. Since H ≠ φ, the process returns to S305, and a hypothesis (2, 0, {(2, 飴, 0), (4, 飴, 0)}) is extracted from the hypothesis list H. State transition from the current state 2 of this hypothesis to the input symbol “M” (<2 → 5, M: ε / 2>, φ)
Make a list E _A, including.

Since E _A is not φ in S306, the process proceeds to S307, where state transition <2 → 5: ε / 2> is extracted from E _A , and a new hypothesis (5, 2, {(2, 飴, 0), (4, 飴, 0)}) is generated, and the process proceeds to S309.
Since the output symbol of the state transition <2 → 5th: ε / 2> is ε, the process proceeds to the next S310 and the hypothesis (5, 2, {(2, 飴, 0), (4, 飴, 0)} ) Is inserted into H ′.
However, since H ′ already has a hypothesis (5, 1, {(1, rain, 0), (1, rain, 0)}) and the arrival state in WFST A is 5, a hypothesis with a small cumulative weight ( 5,1, {(1, rain, 0), (1, rain, 0)}) and the hypothesis (5,2, {(2, 飴, 0), (4, 飴, 0)}) Delete the auxiliary hypothesis list, but connect the hypotheses to (5, 1, {(1, rain, 1), (1, rain, 1), (2, 飴, 2), (4, 飴, 2 )}).
Returning to S306, since E _A = φ, the process proceeds to S311. Since H = φ, the process proceeds to S312 and the hypothesis (5, 1, {(1, rain, 1), (1, rain, 1), (2, 飴, 2), (4, 飴, 2) in H ′ )}) Is moved to H, and since the next input symbol exists in S313, the process returns to S304.

“DA” reading Subsequently, the symbol “DA” is read in S304, and hypotheses (5, 1, {(1, rain, 1), (1, rain, 1), (2, 飴, 2), (4, 飴, 2)}). State transition from the current state 5 of the hypothetical WFST A where the input symbol is “da” <0 → 4: ball / 1>
Make a list E _A, including. Here, because from the current state 5 of WFST A to state 0 can transition with no input symbol, the state transition of WFST A that is included in the E _A is: has become a <0 → 4, It is the ball / 1>.
Since E _A is not φ in S306, the process proceeds to S307, where state transition <0 → 4: ball / 1> is extracted from E _A and a new hypothesis (4, 2, {(1, rain, 1) is extracted in S308. , (1, rain, 1), (2, 飴, 2), (4, 飴, 2)}), and the output symbol of state transition <0 → 4: ball / 1> is ε in S309 Therefore, S401 is corrected to correct the hypothesis (4, 2, {(1, rain, 1), (3, rain, 1), (2, 飴, 2), (4, 飴, 2)}). move on.
In S402, an empty auxiliary hypothesis list L ′ is generated.
In S403, the weight AW [f] = 2−min (1, 1, 2, 2) = 1.
In S404, auxiliary hypothesis list L = {(1, rain, 1), (3, rain, 1), (2, 飴, 2), (4, 飴, 2)} from auxiliary hypothesis (1, rain, 1) State transition that can be transitioned from the state 1 of WFST B with the input symbol “ball” <1 → 4, ball: ball / 5>
Make a list E _B, including.

Since E _B is not equal to φ in S405, the process proceeds to S406. State transition from E _B <1 → 4, Ball: Ball / 5> was removed, auxiliary hypothesis S407 (4, candy, 7) for generating a. Here, the cumulative weight of the auxiliary hypothesis is W [g] + AW [f] + w [E _B ] = 1 + 1 + 5 = 7
It is calculated as follows. In S408, the auxiliary hypothesis (4, rainball, 7) is inserted into L ′.
Returning to S405, since E _B = φ, the process proceeds to S409.
Since it is not L = φ, the process proceeds to S404, and the auxiliary hypothesis (3, rain, 1) is extracted. Since there is no state transition that can be transitioned from the state 3 of WFST B with the input symbol “ball”, E _B = φ.
Returning to S405, since E _B = φ, the process proceeds to S409. Since the auxiliary hypothesis list L is not φ, the process proceeds to S404, the auxiliary hypothesis (2, 飴, 2) is extracted, and the state transition that can be transitioned from the state 2 of WFST B with the input symbol “ball” <2 → 4, ball: ball / 1>
Make a list E _B, including.
Since E _B is not equal to φ in S405, the process proceeds to S406. State transition from E _B <2 → 4, Ball: Ball / 1> was removed, auxiliary hypothesis S407 (4, hard candy, 4) for generating a. Here, the cumulative weight of the auxiliary hypothesis is W [g] + AW [f] + w [E _B ] = 2 + 1 + 1 = 4
It is calculated as follows.
In S408, the auxiliary hypothesis (4, jasper, 4) is inserted into L ′. At this time, since the auxiliary hypothesis (4, rainball, 7) exists in L ′, the auxiliary hypothesis (4, jasper, 4) having a small cumulative weight is left and (4, rainball, 7) is deleted.

Returning to S405, since E _B = φ, the process proceeds to S409.
Since it is not L = φ, the process proceeds to S404, and the auxiliary hypothesis (4, 飴, 2) is extracted. Since there is no state transition that can be transitioned from the state 4 of WFST B with the input symbol “ball”, E _B = φ.
Returning to S405, since E _B = φ, the process proceeds to S409. Since the auxiliary hypothesis list L = φ, the process proceeds to S410, and the element of L ′ (4, jasper, 3) is moved to L.
In S411, the cumulative weight of the hypothesis is 4, and the correction of the hypothesis is finished in S412. As a result, the hypothesis is (4, 4, {(4, jasper, 4)}).
Returning to S310, the hypothesis (4, 4, {(4, jasper, 4)}) is inserted into H ′.
Returning to S306, since E _A = φ, the process proceeds to S311. Since H = φ, the process proceeds to S312 and the hypothesis (4, 4, {(4, jasper, 4)}) in H ′ is moved to H. Since the next input symbol exists in S313, the process returns to S304.

Next, the symbol “ma” is read in S304, and the hypothesis (4, 4, {(4, jasper, 4)}) is extracted from the hypothesis list H in S305. This hypothesis is the state transition from the current state 1 of WFST A to the input symbol “MA” <4 → 5, MA: ε / 1>
Make a list E _A, including.
S306 state transition from E _B proceeds to S307 because it is not E _A = phi in <4 → 5, or: epsilon / 1> was removed, a new hypothesis S308 (5,5, {(4, hard candy, 4) }) And proceeds to S309.
Since the output symbol of state transition <4 → 5, or ε / 1> is ε, the process proceeds to the next S310, and hypothesis 5, 5, {(4, jasper, 4)}) is inserted into H ′.
Returning to S306, since E _A = φ, the process proceeds to S311. Since H = φ, the process proceeds to S312 and the hypothesis (5, 5, {(4, jasper, 4)}) in H ′ is moved to H. Since there is no next input in S313, the process proceeds to S314.
In S314, the hypothesis (5, 5, {(4, Kodama, 4)}) in H is the only hypothesis that has reached the end state, and the output symbol string “Kadama” of the auxiliary hypothesis of that hypothesis is converted The result is output, and the symbol string conversion process ends in S315.

(voice recognition)
On the other hand, the present invention can be applied to speech recognition to efficiently perform speech recognition.
FIG. 12 shows an embodiment of the present invention.
The voice signal sent from the voice signal input unit 401 for inputting voice is converted into an acoustic feature symbol string by the voice feature symbol string extraction unit 405 that extracts the time series of the short-time acoustic pattern of the voice as a symbol string, and the sound The feature symbol string is input and sent to the symbol string converter 102 which performs symbol string conversion according to the present invention. Subsequently, in the symbol string conversion unit 102, an acoustic pattern obtained by analyzing a standard feature of a speech fixed unit (for example, phonemes) from the acoustic model storage unit 401 for each short time (for example, 10 milliseconds) is analyzed. From the word pronunciation dictionary storage unit 402, an acoustic model to be given by sequence matching, a word pronunciation dictionary to give the pronunciation of various words by a sequence of the fixed speech unit, and from the speech recognition language storage unit 403, A language model giving ease of concatenation is read, the acoustic feature symbol string sent from the acoustic feature symbol string extraction unit 401 is read, an output symbol string with the smallest cumulative weight is obtained, and sent to the symbol string output unit 106. The symbol string output unit 106 outputs the received output symbol string as a speech recognition result.

The method of describing a word pronunciation dictionary and language model for speech recognition by WFST is, for example, “Weighted finite-state transducers in speech recognition”, Proceeding of M. Mohri, F. Pereira, M. Riley at the international conference ASR2000. ASR2000, pp.97-106,2000.
As an acoustic model that represents a set of standard acoustic pattern sequences of various speech fixed units (for example, phonemes), for example, a hidden Markov model method that models a set of these acoustic pattern sequences based on probability / statistical theory ( Hidden Markov Model (hereinafter referred to as HMM) is the mainstream. Details of the HMM method are disclosed in, for example, “Recognition of Speech by Stochastic Model” by Seichi Nakagawa, Institute of Electronics, Information and Communication Engineers.
In the case of speech recognition, the score of the acoustic feature symbol (acoustic pattern) calculated by the acoustic model is used as the weight of the preceding WFST. However, the larger the score, the closer the input acoustic pattern is to the sound fixed unit represented by the acoustic model, so a negative acoustic score is used as the weight. In the calculation of the acoustic score by the hidden Markov model, for example, a probability value based on a Gaussian distribution is used.
The acoustic patterns used for speech recognition include mel-frequency cepstral coefficients (referred to as MFCC), delta MFCC, LPC cepstrum, logarithmic power obtained by analyzing a speech signal every short time (for example, 10 milliseconds). and so on.

The figure showing an example of a weighted finite state converter. The figure which defined the weighted finite state converter of FIG. 1 with the table | surface. The figure showing embodiment of the symbol conversion method by a weighted finite state converter. The figure explaining the procedure of the symbol conversion method by a weighted finite state converter. The figure showing an example of the weighted finite state converter which converts the symbol string showing the series of kana into the kanji series corresponding to the kana. The figure showing an example of the weighted finite state converter which gives the possibility of the connection of the kanji series in Japanese by weight. The figure showing embodiment of the conventional symbol sequence conversion method by two weighted finite state converters. The figure explaining the procedure of the symbol sequence conversion method by two weighted finite state converters. The figure showing embodiment of the symbol sequence conversion method by the two weighted finite state converters in this invention. The figure explaining the procedure of the symbol sequence conversion method by the two weighted finite state converters in this invention. The figure explaining the correction method of the hypothesis in the symbol sequence conversion method by the two weighted finite state converters in this invention. The figure showing one Embodiment at the time of using the symbol sequence conversion method by the two weighted finite state converters in this invention as a speech recognition method.

Explanation of symbols

10 State 11 End State 12 State Transition 101 WFST Storage Unit 102 Symbol String Conversion Unit 103 Symbol String Input Unit 104 Hypothesis Development Unit 105 Hypothesis Narrowing Unit 106 Symbol String Output Unit 201 Preceding WFST Storage Unit 202 Subsequent WFST Storage Unit 301 Hypothesis Correction Unit 401 Acoustic model storage unit 402 Word dictionary WFST storage unit 403 Language model WFST storage unit 404 Speech signal input unit 405 Speech feature symbol string extraction unit

Claims (4)

A symbol string input section for sequentially reading the symbol strings;
A front-stage weighted finite state converter used in the preceding stage and a back-stage weighted finite state converter used in the subsequent stage for converting the symbol string in two stages using two weighted finite state converters that convert the symbol string by state transition A symbol string conversion unit for converting a symbol string using and
A symbol output unit that outputs a conversion result obtained by the latter-stage weighted finite state converter;
When the symbols are sequentially read from the symbol string input unit and the input symbol string has been read, cumulative weight values for state transitions respectively applied to the weighted finite state converters at the preceding and succeeding stages of the symbol string converter ( In the symbol string conversion method for outputting the output symbol string corresponding to the state transition process of the latter-stage weighted finite state converter with the smallest (cumulative weight) from the symbol string output unit,
While reading the symbols in order, the cumulative weight for the hypothesis representing one state transition process of the preceding stage weighted finite state transformer, the output symbol string in the hypothetical state transition process, and the input symbol string of the latter stage weighted finite state transformer The state transition process that minimizes the cumulative weight among the possible state transition processes in the latter-stage weighted finite state converter is corrected, and the cumulative weight is corrected by adding it to the hypothetical cumulative weight,
When all input symbol strings are read, the latter weighted finite state when the hypothesis with the smallest cumulative weight and the output symbol string corresponding to the state transition process of that hypothesis are used as the input symbol string of the latter weighted finite state converter A symbol string conversion method characterized in that an output symbol string for a state transition process having a minimum cumulative weight among possible state transition processes in a converter is used as a symbol string conversion result.   An input speech signal is converted into a symbol string representing its acoustic characteristics, and the symbol string is converted into an output symbol string corresponding to the input speech signal using the symbol string conversion method according to claim 1. Voice recognition method. A symbol string input section for sequentially reading the symbol strings;
A front-stage weighted finite state converter used in the preceding stage and a back-stage weighted finite state converter used in the subsequent stage for converting the symbol string in two stages using two weighted finite state converters that convert the symbol string by state transition A symbol string conversion unit for converting a symbol string using and
A symbol output unit that outputs a conversion result obtained by the latter-stage weighted finite state converter;
When the symbols are sequentially read from the symbol string input unit and the input symbol string has been read, cumulative weight values for state transitions respectively applied to the weighted finite state converters at the preceding and succeeding stages of the symbol string converter ( In the symbol string converter that outputs the output symbol string corresponding to the state transition process of the weighted finite state converter of the subsequent stage with the smallest (cumulative weight) from the symbol string output unit,
The symbol string converter
While reading the symbols in order, the cumulative weight for the hypothesis representing one state transition process of the preceding stage weighted finite state transformer, the output symbol string in the hypothetical state transition process, and the input symbol string of the latter stage weighted finite state transformer A state transition process in which the cumulative weight is minimized among the possible state transition processes in the latter-stage weighted finite state converter, and correcting by adding the cumulative weight to the hypothetical cumulative weight;
When all input symbol strings are read, the latter weighted finite state when the hypothesis with the smallest cumulative weight and the output symbol string corresponding to the state transition process of that hypothesis are used as the input symbol string of the latter weighted finite state converter And a means for converting the output symbol string for the state transition process having the smallest accumulated weight among the possible state transition processes in the converter into a symbol string conversion result.   An input speech signal is converted into a symbol string representing its acoustic characteristics, and the symbol string is converted into an output symbol string corresponding to the input speech signal using the symbol string converter according to claim 2. Voice recognition device.

Images