JP2011164124A - Acoustic model parameter learning method based on linear classification model and device, method and device for creating finite state converter with phoneme weighting, and program therefor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To express an acoustic model and a language model as the same model. <P>SOLUTION: The acoustic model parameter learning method includes a model parameter initializing process and a model parameter updating process. In the model parameter initializing process, a model parameter for calculating a recognition score is initialized. In the model parameter updating process, a feature quantity vector is input, and an objective function based on accumulation of an inner product value of the feature quantity vector and the model parameter is given from outside. The model parameter which maximizes the objective function is calculated by updating the initialized model parameter to output a partial model parameter composed of the predetermined number of frames corresponding to each phoneme. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention has a simple structure compared to HMM (Hidden Markov Model), a method and apparatus for generating an acoustic model by a linear classification model capable of expressing an N-gram language model, and an acoustic model created by the method The present invention relates to a phoneme-weighted finite state transducer generation method and apparatus for associating a speech signal with a phoneme, and a program thereof.

  Of the elements constituting a standard speech recognition system, the main elements are an acoustic model and a language model. An acoustic model is a model that associates a speech signal with a phoneme. In general, HMMs and models based on them are widely used as acoustic models. It is a model that considers time expansion and contraction, and has been widely used to model physical phenomena with expansion and contraction such as audio signals.

  As the language model, the N-gram language model is widely used. This model has a different form of expression from the acoustic model. FIG. 9 shows an example in which a search space for word strings is configured by an acoustic model (HMM) and a language model (reference document 1). As shown in this figure, a search space in speech recognition is represented by a network in which a phoneme determined by a language model and an acoustic model are embedded.

  In general, the acoustic model and the language model are learned independently, but in recent years, several attempts have been made for global optimization. However, in addition to the complexity of the HMM itself, the model parameters of the language model expressed in different formats must be adjusted at the same time, which complicates the speech recognition system.

Takaaki Hori, Hajime Tsukada “State-of-the-Art 3 of Speech Information Processing“ Speech Recognition by Weighted Finite State Transducer ”, Information Processing Society of Japan Information Processing, Vol. 45, No. 10, pp.1020-1026 (2004.10). Akio Ando “Real-time Speech Recognition” The Institute of Electronics, Information and Communication Engineers Annette J Dobson, Adrian G Barnett, “An Introduction to Generalized Linear Models”, CRC Press. Koby Crammer, Ofer Dekel, Joseph Keshet, Shai Shalev-Shwartz, Yoram Singer, “Online passive aggressive algorithms”, Journal of Machine Learning Research, Vol.7, pp.531-583, 2006. Taku Kudo, Satoshi Yamamoto, Yuji Matsumoto “Japanese Morphological Analysis Using Conditional Random Fields” SIGNL-161, 2004. Hal Daume Iii, Daniel Marcu, “Learning as search optimization: Approximate large margin methods for structured prediction”, International Conference on Machine Learning, pp.169-176, 2005.

  Therefore, if the acoustic model and the language model are expressed by the same model, the whole model can be easily optimized, and the process can be expected to be simplified. However, the conventional linear classification model is not formed so as to output a score according to the feature amount input in time series in time series. For this reason, there is a problem that cannot be used for continuous speech recognition when the acoustic model is expressed by a linear classification model.

  The present invention has been made in view of such problems, and an acoustic model learning method and apparatus capable of expressing an acoustic model and a language model in a unified framework, and an acoustic model created by the method. An object of the present invention is to provide a phoneme-weighted finite state transducer generation method and apparatus for associating a speech signal with a phoneme, and a program thereof.

  The acoustic model parameter learning method of the present invention includes a model parameter initialization process and a model parameter update process. In the model parameter initialization process, a partial model parameter consisting of a predetermined number of frames corresponding to each phoneme for obtaining a recognition score is initialized. In the model parameter update process, an objective function based on the accumulation of the inner product value of the feature vector and the partial model parameter is given from the outside with the feature vector as input, and the model parameter that maximizes the objective function is initialized as described above. The model parameter is updated to obtain a partial model parameter corresponding to each phoneme.

The phoneme-weighted finite state transducer generation method of the present invention includes an initial state setting process, an intermediate state setting array process, and a final state setting process. In the initial state setting process, a partial model parameter corresponding to a phoneme is input, and an initial state in which a score corresponding to the phoneme and the first frame is output is set. In the intermediate state setting array process, a function defined as an inner product of model parameters W _{p, i} constituting partial model parameters and an input feature vector is used as a score, and an intermediate having a state transition that outputs a score 0 without input. Set the state and arrange. In the final state setting process, a self-transition state in which score 0 is output is set, and a final state in which score 0 is output without input and transition to an end state is set.

  According to the acoustic model parameter learning method of the present invention, since the acoustic model can be expressed in the same expression format as the language model, it is easy to optimize the entire speech recognition system. Also, according to the phoneme weighted finite state transducer generating method of the present invention, the acoustic model created by the acoustic model parameter learning method is described in the form of a weighted finite state transducer. The acoustic model enables high-speed and highly accurate speech recognition.

The figure which shows the function structural example of the acoustic model parameter learning apparatus 100 of this invention. The figure which shows the operation | movement flow of the acoustic model parameter learning apparatus. The figure which shows the function structural example of the model parameter update part 14. FIG. The figure which shows the operation | movement flow of the model parameter update part. The figure which shows the function structural example of the finite state converter production | generation apparatus 200 with a phoneme weight of this invention. The figure which shows the operation | movement flow of the finite state converter production | generation apparatus 200 with a phoneme weight. The figure which shows an example of the state transition model produced with the phoneme weighted finite state converter production | generation apparatus 200. FIG. The figure which shows the simple functional structure of the speech recognition apparatus 300 using the state transition model produced with the phoneme weighted finite state converter production | generation apparatus 200. FIG. The figure which shows notionally the example of the search space of the speech recognition using the conventional acoustic model.

  Embodiments of the present invention will be described below with reference to the drawings. The same reference numerals are given to the same components in a plurality of drawings, and the description will not be repeated. Prior to the description of the embodiments, the overall optimization of the acoustic model and the language model of the present invention will be described.

[Overall optimization of acoustic model and language model]
Let C and L be an acoustic model and a language model, respectively. The functions governed by these model parameters are M _C and N _L. M _C is a function that returns the acoustic score and N _L is a function that returns a language score.

When performing the overall optimization of the acoustic model and the language model, _C and _L that maximize the objective function O (M _C , N _L ) prepared in advance are obtained. For example, if the optimum model parameter is searched while considering the gradient of the objective function by the gradient method or the like, it is necessary to calculate the equations (1) and (2).

If M _C and N _L are completely different functions, it is necessary to learn two types of models. This is the reason why conventional acoustic models cannot be handled uniformly.

  On the other hand, if the acoustic model and the language model are linear classification models, the functions for calculating the acoustic score and the language score in the process of performing speech recognition can be expressed by Expressions (3) and (4).

Here, T is a transposed symbol, A ′ is a feature vector used for acoustic model learning, A ″ is a feature vector used for language model learning. A feature vector obtained by arranging feature vectors A ′ and A ″ is A. When written, the voice recognition score is W ^T A. W matches C and L side by side.

Therefore, since the objective function that unifies the acoustic model and the language model can be written as O (W ^T A), the model parameter learning device can be integrated into one.

  According to the present invention, a feature quantity vector A originally calculated over a plurality of time frames is divided for each frame, and the inner product value of the divided feature quantity vector and model parameter is accumulated for each frame. Thus, a score corresponding to a feature vector input in time series can be generated in time series.

  FIG. 1 shows an example of the functional configuration of an acoustic model parameter learning apparatus 100 according to the present invention. FIG. 2 shows the operation flow. The acoustic model parameter learning device 100 includes a model parameter initialization unit 12 and a model parameter update unit 14. The functions of the respective units are realized by a predetermined program being read into a computer constituted by, for example, a ROM, a RAM, and a CPU, and the CPU executing the program.

The model parameter initialization unit 12 initializes model parameters for obtaining a recognition score (step S12). Model parameter updating unit 14, a model parameter that maximizes the objective function O _W as input feature vector X ^{_{t +} n t +} _1, obtained by updating the model parameters initialized with the model parameter initializing unit 12 A partial model parameter consisting of a predetermined number of frames corresponding to each phoneme p is output (step S14). The notation of X ^{t + n} _{t + 1} is correct as shown in FIG. X ^{t + n} _{t + 1} is composed of feature quantity vectors obtained in each time frame from t + 1 to t + n. In contrast, a feature vector obtained in a frame at a certain time _t is denoted as Xt.

Feature vector X ^{_{t +} n t +} ₁ is input from the correct database 10 which stores the feature vector X ^{_{t +} n t +} ₁ of the correct phoneme p. The objective function O _W is a function based on the accumulation of inner product of the feature vector X ^{_{t +} n t +} ₁ and the model parameters given from the outside. With this configuration, the acoustic model parameter learning device 100 generates an acoustic model having the same expression format as the language model.

(Linear classification model)
The linear classification model is a classification model based on element function G _W that satisfies the constraints shown in equation (5).

Element function _{G W} is, for example Formula (6) or expression (7) can be considered like.

Objective function O _W applied to the model parameter updating unit 14 is in the form obtained by accumulating the elements function G _W. What is meant by a linear classification model, recognition is carried out in a manner represented by the formula shown on the right-hand side of equation (5), the learning of the model is be carried out using the objective function O _W. The objective function O _W, since it is possible to use a non-linear function for the model parameters W, the generation of highly accurate model can be expected from performing learning using a linear inner product value W ^{T p} _A.

  In the present invention, the accumulation of the inner product values of the feature quantity vector and the model parameter (equation (8)) is calculated, and this is used as the score of the phoneme p.

Here, n means a time frame to be considered. When the number of time frames of the input phoneme signal exceeds t + n, the period after t + n is ignored. Therefore, the predetermined number n is set sufficiently long. For example, the maximum value of the learning data is set. In Expression (8), when the number of time frames of the input audio signal is less than t + n, the feature amount vector X ^{t + n} _{t + m + 1} after the input length is considered as a zero vector. t + m (<(t + n)) corresponds to the last frame of the input signal.

  The entire model parameter to be estimated by learning can be expressed by equation (9).

  The predetermined number n is set to a value such as 20, for example.

FIG. 3 shows a more specific functional configuration example of the model parameter update unit 14, and the operation will be described in more detail. FIG. 4 shows the operation flow. Inclination calculation means 140, inclination evaluation means 142, and parameter update means 144 are provided. Inclination calculation means 140, the partial differential function given externally, the feature vector X ^{t + n t +} _1, calculates a tilt of the objective function O _W giving the model parameters are updated by the parameter updating unit 144 (step S140 ).

Inclination evaluation unit 142, until the gradient of the objective function O _W becomes monotonous increase to extreme, instructs the updating of the parameters in the parameter updating unit 144 obtains the gradient of the objective function O _W calculated in the current parameter until it is determined that the objective function _{O W} has converged, and repeats the operation (repetition loop No in step S142~S145). The parameter update unit 144 updates the parameter based on the control signal from the inclination evaluation unit 142 (step S144). When the slope becomes an extreme value (meaning that it has been determined that it has converged), the parameter at that time is output as a model parameter (step S146).

The partial differential function, for example function obtained by partially differentiating the objective function O _W shown in Equation (10) in each element of W is used.

[Phoneme-weighted finite state transducer generator]
FIG. 5 shows a functional configuration example of the phoneme-weighted finite state transducer generation apparatus 200 of the present invention. The operation flow is shown in FIG. The phoneme weighted finite state transducer generating apparatus 200 includes an initial state setting unit 20, an intermediate state setting array unit 22, a final state setting unit 24, and a control unit 26.

A weighted finite state converter is an extension of a finite automaton (Finite Automaton) widely known as a model of a state transition machine for conversion of input / output sequences. Specific examples will be described later. The phoneme-weighted finite state transducer generating apparatus 200 includes partial model parameters {W _{p, i} | 1 ≦ i ≦ n} corresponding to a certain phoneme p among model parameters output from the acoustic model parameter learning apparatus 100 (hereinafter, (Abbreviated as {W _{p, i} }), and generates a phoneme weighted finite normal converter of the phoneme p.

The control unit 26 stores the input partial model parameters {W _{p, i} } in a memory or the like (not shown). Then, the repetition variable i and the like are initialized (i = 1) (step S260).

When the feature value vector is input, the initialization setting unit 20 uses, as an output weight, a function defined as the inner product of the first model parameter W _{p, 1} of the partial model parameter {W _{p, i} } and the input feature value vector. The state transition with the phoneme p as an output symbol is set (step S20). The control unit 26 updates i (step S261).

The intermediate state setting array unit 22 uses, as an output weight, a function defined as the inner product of the model parameter W _{p, i} constituting the partial model parameter {W _{p, i} } and the input feature vector corresponding to _i , and The state transition is set to have no output signal (the symbol ε representing that nothing is output is the output symbol). The former transitions to the next state and the latter transitions to the end state. This process is repeated until i reaches a predetermined number n + 1 (No repeat loop in steps S22 to S262). That is, after the initial state, states corresponding to the n model parameters W _{p, i} including the initial state are arranged.

  The last state setting unit 24 sets a self-state transition having no output signal to 0 as a weighted output when a feature vector is input to the predetermined number n + 1 first state (step S24).

  FIG. 7 shows an example of a weighted finite state transducer generated through the above-described process. In the figure, ◯ represents a state, 1 is an initial state, and a state represented by a double circle means completion. κ is a symbol that means that a state transition is made when a feature vector is given as an input of the converter, μ always makes a transition, and ε means nothing is output.

  In this weighted state transition converter, a score for each frame is calculated through an inner product operation between each element of the feature vector and the corresponding model parameter.

  The speech recognition apparatus can be configured by using the weighted finite state transducer generated by the phoneme weighted finite state transducer generating apparatus 200. FIG. 8 shows a simple functional configuration example of the speech recognition apparatus 300 using the weighted finite state.

The voice recognition device 300 includes a WFST database 32 and a voice recognition unit 30. The WFST database 32 stores a plurality of phoneme weighted finite state transducers. Speech recognition unit 30, as input feature vector X _t ~X _{t + n,} executes the voice recognition processing it by calculating a weighted finite state seeking scores.

[Confirmation experiment]
For the purpose of confirming the usefulness of the acoustic model based on the linear classification model according to the present invention, the performance was compared with the conventional HMM acoustic model. The experiment was performed with isolated phoneme recognition. Isolated phoneme recognition is a problem in which only phoneme labels are determined under a phoneme boundary.

  As the learning data, a lecture 150 of the Japanese Spoken Language Corpus (CSJ) was used. A PA algorithm (Passive Aggressive), which is a high-speed online margin maximization learning technique, was used to estimate the model parameters of the acoustic model of the present invention. The evaluation data is for 10 lectures at the conference. As the feature vector, general MFCC 12 dimensions + logarithmic power + Δ + ΔΔ all 39 dimensions were used.

  As a result, the accuracy rate of the conventional HMM acoustic model is 60.1%, and the accuracy rate when the acoustic model according to the present invention is used is 59.9%. It was. From this result, it was confirmed that the acoustic model according to the present invention has the ability to replace the conventional HMM acoustic model.

  When the processing means in the above apparatus is realized by a computer, the processing contents of functions that each apparatus should have are described by a program. Then, by executing this program on the computer, the processing means in each apparatus is realized on the computer.

  The program describing the processing contents can be recorded on a computer-readable recording medium. As the computer-readable recording medium, for example, any recording medium such as a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory may be used. Specifically, for example, as a magnetic recording device, a hard disk device, a flexible disk, a magnetic tape or the like, and as an optical disk, a DVD (Digital Versatile Disc), a DVD-RAM (Random Access Memory), a CD-ROM (Compact Disc Read Only). Memory), CD-R (Recordable) / RW (ReWritable), etc., magneto-optical recording medium, MO (Magneto Optical disc), etc., semiconductor memory, EEP-ROM (Electronically Erasable and Programmable-Read Only Memory), etc. Can be used.

  The program is distributed by selling, transferring, or lending a portable recording medium such as a DVD or CD-ROM in which the program is recorded. Further, the program may be distributed by storing the program in a recording device of a server computer and transferring the program from the server computer to another computer via a network.

  In addition, the functional configuration unit of each device may be configured by causing a predetermined program to be executed on a computer, or at least a part of these processing contents may be realized in hardware.

Claims (7)

A model parameter initialization unit that initializes a partial model parameter composed of a predetermined number of frames corresponding to each phoneme for obtaining a recognition score; and
An objective function based on the accumulation of the inner product value of the feature vector and model parameter is input from the phoneme feature vector as input, and the model parameter that maximizes the objective function is updated to the initialized model parameter. A model parameter update process for outputting partial model parameters corresponding to each phoneme,
Acoustic model parameter learning method including The acoustic model parameter learning method according to claim 1,
The model parameter update process
A slope calculation step for calculating a slope of the objective function based on a feature vector and a model parameter to be updated;
A slope evaluation step for evaluating the slope of the objective function until the slope of the objective function monotonically increases to an extreme value and outputting a partial model parameter consisting of a predetermined number of frames corresponding to each phoneme;
A parameter update step for updating the model parameter based on the evaluation result of the inclination evaluation step;
An acoustic model parameter learning method comprising: An initial state setting process in which an initial state setting unit sets a partial model parameter including a predetermined number of frames corresponding to a phoneme and sets an initial state for outputting a score corresponding to the phoneme and the first frame;
The intermediate state setting array unit has a state transition in which a function defined as the inner product of the model parameters W _{p, i} constituting the partial model parameters and the input feature quantity vector is used as a score and score 0 is output without input. An intermediate state setting sequence process for setting and arranging intermediate states;
A final state setting process in which the final state setting unit sets a self-transition state that outputs a score of 0, outputs a score of 0 without input, and transitions to an end state;
A phoneme-weighted finite state transducer generation method including: A model parameter initialization unit that initializes partial model parameters composed of a predetermined number of frames corresponding to each phoneme for obtaining a recognition score;
A phoneme feature vector is input, an objective function based on the accumulation of the inner product value of the feature vector and the model parameter is given from the outside, and the model parameter that maximizes the objective function A model parameter update unit that calculates and outputs partial model parameters corresponding to each phoneme;
An acoustic model parameter learning device comprising: The acoustic model parameter learning device according to claim 4,
The model parameter update unit
An inclination calculating means for calculating an inclination of the objective function based on a feature vector and a model parameter to be updated;
A slope evaluation means for evaluating the slope of the objective function until the slope of the objective function monotonically increases to an extreme value and outputting a partial model parameter consisting of a predetermined number of frames corresponding to each phoneme;
Parameter updating means for updating the model parameter based on the evaluation result of the inclination evaluation step;
An acoustic model parameter learning device comprising: Input a partial model parameter consisting of a predetermined number of frames corresponding to phonemes,
An initial state setting unit for setting an initial state for outputting a score corresponding to the phoneme and the first frame;
A function defined as the inner product of the model parameters _{Wp, i} and the input feature vector constituting the partial model parameters is set as a score, and an intermediate state having a state transition that outputs a score of 0 without input is set and arranged An intermediate state setting array unit,
A final state setting unit that outputs a score 0 and outputs a score 0 without any input and transitions to an end state;
A phoneme-weighted finite state transducer generator comprising:   A device program for causing a computer to execute the function of each unit of the acoustic model parameter learning device according to any one of claims 4 to 7 or the function of each unit of the phoneme-weighted finite state transducer generation device.

Images