JP4631076B2 - Method and system for optimizing phoneme unit sets - Google Patents

Description

  The present invention relates to automatic speech recognition (ASR), and more particularly to optimization of phoneme unit sets such as phoneme sets used in ASR.

  ASR is an essential tool in man-machine interaction. ASR allows the computer to understand the operator's commands in natural language, eliminating the need for the operator to learn a complex command system for the computer.

  FIG. 6 shows the basic ASR mechanism. Referring to FIG. 6, ASR system 162 decodes input speech X160, and recognized (decoded) word ^ W164 (the symbol “^” in the sentence is originally added above letter W). .) Is output using Equation 166 below.

ここでＰ（Ｘ｜Ｗ）は音響モデル確率を示し、Ｐ（Ｗ）は言語モデル確率を示す。これらのモデルは対象となる言語の単語を、それぞれの音素のシーケンスと共に記載するレキシコンを用いて構築される。音素は予め定められた基本音素セットのうちから選択される。

Here, P (X | W) represents the acoustic model probability, and P (W) represents the language model probability. These models are built using lexicons that describe the words of the language of interest along with their phoneme sequences. The phonemes are selected from a predetermined basic phoneme set.

  In the Large Vocabulary Continuous Speech Recognition (LVCSR) system, a widely accepted phoneme set is used.

  There is a question of whether the same phoneme set should be used for a simple LVCSR task and a more complex LVCSR task. In small vocabulary tasks such as number recognition tasks, words such as numbers are used as basic units. Similarly, for simple LVCSR tasks, it may be advantageous to use a simple phoneme set.

  In many studies on ASR, phone sets determined by several heuristics are tried and a set is selected based on ASR recognition performance.

  If more units are included in a phoneme set, it will provide phonemeologically more discriminating information. However, this also means that a more detailed acoustic difference is used. In the case of speech recognition, if more detailed or smaller acoustic differences need to be modeled, the robustness of AM (Acoustic Model) tends to decrease.

  If the number of units included in a phoneme set is small, each phoneme AM usually has more training data than a larger phoneme set. Furthermore, when the number of phonemes is small, the difference between phonemes tends to be larger than the difference between many phonemes. As a result, AM can be more robust if the phoneme set is smaller. However, reducing the phoneme set size creates another problem. That is, the discriminating power in the language space is lost. For example, Japanese long vowel “A” and short vowel “a” are merged into one vowel, so confusion between words will increase.

  In this regard, the latest ASR optimization is performed according to the following concept. The above equation can be written in the following form:

ここでＦは基本単位シーケンスを示し、Ｐ（Ｘ｜Ｆ）は頑健な音響モデル化の優勢なトピックを示し、Ｐ（Ｆ｜Ｗ）は発音モデル化の注目のトピックを示し、Ｐ（Ｗ）は顕著な言語モデル化を示す。多くの場合、Ｆは音素セットである。

Where F represents the basic unit sequence, P (X | F) represents the dominant topic of robust acoustic modeling, P (F | W) represents the topic of interest for pronunciation modeling, and P (W) Indicates remarkable language modeling. In many cases, F is a phoneme set.

  However, in the prior art, there have been some heuristic attempts to make comparisons with various sets of basic units, but optimization of the basic unit set has been particularly considered in consideration of the entire ASR framework using probability. It can be said that almost nothing has been done.

  Accordingly, one object of the present invention is to provide a method and apparatus for optimizing the basic unit set for a particular ASR task.

  Another object of the present invention is to provide a method and apparatus for optimizing phoneme sets for specific ASR tasks.

  According to one aspect of the present invention, a method for optimizing a phoneme unit set in a predetermined language includes: preparing a basic unit set in a computer readable format on a computer; and Generating a plurality of basic unit subsets by applying an out method; calculating a predetermined measure of linguistic discriminatory power for each of the basic unit subsets; The steps of replacing with the one with the highest linguistic discriminatory power, and the steps of generating, calculating, and repeating the replacement are repeated until a predetermined criterion is satisfied.

Preferably, the step of calculating comprises calculating the basic unit set, a mutual information between each of the basic unit subsets.

More preferably, replaced step includes replacing at those having a basic unit set, the highest value of the mutual information amount calculated by the step of calculating of the basic units subset.

  More preferably, the basic unit set is a basic phoneme set for a predetermined language.

  According to another aspect of the present invention, a system for optimizing a unit set of a predetermined language includes storage means for storing a basic unit set in a computer-readable format, and leave one in the basic unit set. A generating means for generating a plurality of basic unit subsets by applying the out method, a calculating means for calculating a predetermined measure of linguistic discriminating power for each of the basic unit subsets, and storing in the storage means The replacement means for replacing the set of basic units with the basic unit subset having the highest linguistic discriminatory power, and the storage means, the generation means, the calculation means, and the replacement means are repeatedly operated until a predetermined criterion is satisfied. Control means for controlling the operation.

  In the case of ASR, there are two types of identification means for identifying two words. One is pronunciation, and the other is a word context, that is, a language model (LM). When it is difficult to identify a pair of words by an acoustic score, for example, in the case of a homophone or a homolog, it will be easy to identify if there is contextual word information. For example, “bridge” and “chopsticks” are clearly different contextual words.

  Based on the above discussion, this example proposes an optimal design of a phoneme set for a particular ASR task, ie a task-based phoneme design. The basic idea is that deleting a phoneme from a large phoneme set may delete the phoneme to reduce the phoneme set size if the linguistic discriminatory power is not significantly reduced. is there.

In this embodiment, the maximum mutual information (Mutual Information: MI) employing a phoneme set design based on the reference. That is, MI is used as a measure of the linguistic discriminatory power of the basic unit subset. This example relates to designing an optimized phoneme set for Chinese.

  The basic unit set Φ is important in two specific aspects. That is, it defines the main classification of the entire acoustic space, and further provides important clues for language space classification.

Figure 1 shows the different basic unit _{set, Φ 1 = {f 1,} f 2, ... f N} and _{Φ 2 = {p 1, p} 2, ... p M} intuitive influence due. Referring to FIG. 1, assume that the number of phonemes M of Φ ₂ is much larger than the number of phonemes N of Φ ₁ (ie, N << M). [Phi ₁ is the acoustic space 20 N sub space _{f 1,} f 2, ... is divided into _{f N,} [Phi ₂ is smaller than the same acoustic space 22 subspaces _{p 1, p 2,} it is divided into ... _{p M.} Therefore, [Phi ₁ is able to provide a robust acoustic model, on the other hand, discrimination is weak compared to that of [Phi _2.

  FIG. 2 shows the overall configuration for unit set training in this embodiment. Referring to FIG. 2, the training system includes a state-of-the-art ASR system 40 for training, a storage unit 42 for a language model, and a lexicon-based decoding system 44.

  The training ASR system 40 is a speech generation and ASR module 50 for converting the input text W into a phoneme sequence F, and the most probable of the decoded word text in the word lattice formed by the phoneme sequence F. A word lattice scoring module 52 for selecting the high word text ^ W by scoring each path of the lattice with reference to the language model 42.

Lexicon based decoding system 44, each entry word, a dictionary 62 and 64 described with respective phoneme set [Phi ₁ and [Phi _2, respectively using the dictionary 62 and 64, the phoneme sequence F input text W ₁ and a lexicon-based conversion module 60 for converting the F _2, the phoneme sequence F ₁ and higher word text W ₁ and W ₂ the most probable among the word text in the word lattice formed by the F _2, And a word lattice scoring module 66 for selecting by scoring each path of the lattice with reference to the language model 42. In FIG. 2, only two dictionaries are shown for the sake of brevity. This example relates to a Chinese ASR system, phoneme set Φ ₁ contains tone information, while phoneme set Φ ₂ does not.

  The training ASR system 40 receives the corpus of the training text W and outputs a decoded word ^ W according to the following maximization formula.

確率Ｐ（Ｗ｜Ｆ）を最大にする音素セットが最適な音素セット＾Φとして選択される。すなわち、

The phoneme set that maximizes the probability P (W | F) is selected as the optimal phoneme set ^ Φ. That is,

トレーニング用ＡＳＲシステム４０とレキシコンベースのデコードシステム４４との動作により、上述の式に従って、Ｐ（Ｗ｜Ｆ）の要素を計算し、最適な音素セット＾Φを選択することができる。

By the operation of the training ASR system 40 and the lexicon-based decoding system 44, an element of P (W | F) can be calculated according to the above-described equation, and an optimal phoneme set ^ Φ can be selected.

  FIG. 3 is a diagram showing the overall structure of the phoneme set optimization system 80 of this embodiment. Referring to FIG. 3, the phoneme set optimization system 80 optimizes a phoneme set by using a storage device of a basic unit set 90, a storage device of a training text 92, a basic unit set 90, and a training text 92. A phoneme set optimizing module 96 for outputting the phoneme set 94.

The phoneme set optimization module 96 can be realized by software executed on a computer. The flow of software control is shown in the flowchart of FIG. Referring to FIG. 4, the phoneme set optimization module 96 performs the following steps. The working phoneme set Φ is replaced with the initial phoneme set Φ ₀ (ie, the basic unit set 90) (step 100). A phoneme subset Φ _i (from i = 1 to the number of elements of Φ; Φ _i = Φ− {e _i }; e _i is the i-th phoneme in Φ) is generated (step 102). Calculating a mutual information MI _i between each set [Phi and subset [Phi _i are working (step 104). An index M satisfying the following equation is specified (step 106).

その後Ｍ番目の音素サブセットΦ_Ｍを選択し、選択されたサブセットΦ_Ｍ中の音素を用いてレキシコン及びテキストコーパスを作り変える（ステップ１０８）。作り変える過程において、レキシコンとテキストコーパスとは、レキシコンとテキストコーパス中で用いられている削除された音素を、それぞれ最も近い音素とマージするように更新される。

Then select the M-th phoneme subset [Phi _M, reshape lexicon and the text corpus with phonemes in the subset [Phi _M selected (step 108). In the remake process, the lexicon and text corpus are updated to merge the deleted phonemes used in the lexicon and text corpus with the nearest phonemes, respectively.

The phoneme set optimization module 96 further executes a step of determining whether or not a predetermined stop condition is satisfied (step 110). If the condition is met, the phoneme set optimization module 96 stops operating. Otherwise, control proceeds to step 112 where the working subset Φ is replaced with the selected subset Φ _M , after which control returns to step 102.

After repeating a predetermined number of times, the operation stops. Alternatively, the operation can be stopped when the decrease in the mutual information amount exceeds a predetermined threshold value.

The phoneme set optimization module 96 operates as follows. First, in step 100, the basic unit set 90 is selected as the working set Φ. In step 102 the phoneme subset [Phi ₁ until [Phi _N is generated. The subset Φ _i is generated by removing the phoneme e _i from the working set Φ. In other words, Φ _i is generated by applying a leave-one-out method to the working set Φ.

In step 104, the amount of mutual information MI _i between each [Phi _N from the set [Phi and subset [Phi ₁ in operation is calculated. In step 106, an index M that maximizes the corresponding mutual information amount MI _M among the mutual information amounts MI _i is selected.

At step 108, the Mth phoneme subset (subset Φ _M ) is selected and the lexicon and text corpus are recreated using the selected phoneme subset Φ _M.

In step 110, it is determined whether the stop condition is satisfied. Unless if condition is satisfied, control proceeds to step 112, where [Phi is replaced with [Phi _M. Thereafter, the control returns to step 102, and steps 102 to 108 are repeated. When the stop condition is met, the operation stops.

  Thus, starting with a large initial unit set 90 based on detailed phoneme classification, the phoneme set optimization module 96 can reduce phoneme sets while iterating according to some criteria.

  FIG. 5 shows the result of the verification experiment of this example. In this experiment, the original set of 203 units containing tone information is reduced. A set of 59 units without tone information was used for comparison. These two sets are widely used in the latest Chinese ASR system. The verification text corpus includes 1,614 short sentences, and the total number of words is 9,484.

Referring to FIG. 5, as compared to the unit set without tone of 59 C (indicated by box 132), tone with the original set of 203, has a higher mutual information in bit representation. In the reduction process indicated by the line 130, the unit set generated with the same number of 59 units maintained a higher mutual information amount than the unit set without tone, as indicated by the point A in FIG. In other words, the generated set A has better linguistic discriminatory power than the traditional 59 toneless unit set, despite the same number. Unit Set point B in FIG. 5, but maintains the mutual information about the same amount that no tone unit set C, the number of units is much less. The number is 47, so it is more efficient than C set.

As described above, the system and method of this embodiment can successfully reduce the number of phonemes in a phoneme set without reducing the amount of mutual information. If the text specifying the task is used for training, the phoneme set can be optimized for the task, and if the phoneme set is used, a robust acoustic model having sufficient discrimination power for the task can be obtained. In addition, it is possible to provide a language model having sufficiently detailed discrimination power.

  In the above embodiment, the phoneme set is optimized, but the present invention is not limited to the optimization of the phoneme set. The present invention is applicable to optimization of any basic phoneme unit set that can be replaced with a phoneme set in ASR. For example, if the vocabulary is relatively small, the unit set can be a set of words (word pronunciations) in the vocabulary.

  The embodiment disclosed herein is merely an example, and the present invention is not limited to the above-described embodiment. The scope of the present invention is indicated by each of the claims after taking into account the description of the detailed description of the invention, and all modifications within the meaning and scope equivalent to the wording described therein are intended. Including.

It is a figure which shows the intuitive influence from a different basic unit set. It is a figure which shows the whole structure of the training of the unit set of this Example. It is a figure which illustrates the whole structure of the phoneme set optimization system 80 of this Example. It is a figure which shows the control flow of the software which implement | achieves the phoneme set optimization module 96 of this Example. It is a figure which shows the verification experiment result of this Example in the form of a graph. FIG. 2 shows a basic ASR scheme according to the prior art.

Explanation of symbols

40 training ASR system 42 language model 44 lexicon based decoding system 50 ASR module 52 word lattice scoring module 60 lexicon based transformation module 62, 64 dictionary 66 word lattice scoring module 80 phoneme set optimization system 90 basic unit set 92 Training text 94 Optimized phoneme set 96 Phoneme set optimization module

Claims (5)

A method for optimizing a phoneme unit set of a predetermined language, comprising:
Preparing a basic unit set in a computer readable format;
Generating a plurality of basic unit subsets by applying a leave-one-out method to the basic unit set;
Calculating a predetermined measure of linguistic discriminatory power for each of said basic unit subsets;
Replacing the basic unit set with the highest linguistic discriminatory of the basic unit subsets;
A method for optimizing a phoneme unit set of a predetermined language, wherein the generating step, the calculating step, and the replacing step are repeated until a predetermined criterion is satisfied. It said calculating steps includes a step of calculating the basic unit set, a mutual information between each of the basic units subset The method of claim 1. The replacing step, the basic unit set comprises replacing at one with the highest value of the mutual information amount calculated in said step of calculating one of the basic units subset The method of claim 2 . The method according to claim 1, wherein the basic unit set is a basic phoneme set for the predetermined language. A system for optimizing a unit set of a predetermined language,
Storage means for storing the basic unit set in a computer readable format;
Generating means for generating a plurality of basic unit subsets by applying a leave-one-out method to the basic unit set;
Calculating means for calculating a predetermined measure of linguistic discriminatory power for each of said basic unit subsets;
Replacement means for replacing the basic unit set stored in the storage means with a basic unit subset having the highest linguistic discriminatory power;
And a control means for controlling the storage means, the generating means, the calculating means, and the replacing means so as to repeatedly operate until a predetermined criterion is satisfied.

Images