JP5113797B2 - Dissimilarity utilization type discriminative learning apparatus and method, and program thereof - Google Patents

Description

  The present invention is based on a feature amount sequence that is dynamically changed on a time axis and / or a space axis of voice, still image, moving image, etc. The present invention relates to a discriminative learning method, an apparatus thereof, and a program for pattern recognition for identifying a specified signal type as a symbol series expressed by discrete values.

  Many pattern recognition errors occur due to confusion of patterns located around the boundary with other symbols adjacent in the feature amount space. In order to suppress this, it is effective to obtain information from learning data of both correct symbols and adjacent symbols at the learning stage and estimate model parameters so as to reduce confusion. Such a framework for improving the inter-symbol discriminating ability is generically called discriminative training.

  An example in which minimum identification error (MCE: Minimum Classification Error, hereinafter referred to as MCE) learning, which is one of the typical implementations of the discriminative learning method, is applied to pattern recognition for identifying symbol sequences will be described. FIG. 7 shows an example of a functional configuration of a discriminative learning device 700 in which the MCE learning method is applied to continuous word speech recognition which is a kind of pattern recognition.

  The discriminative learning device 700 includes an acoustic model recording unit 70, a positive example language model recording unit 71, a positive example identification function value generation unit 73, a positive example speech recognition unit 74, a negative example identification function value generation unit 75, and a voice recognition unit. 76, a positive / negative example comparison unit 77, and a model parameter optimization unit 78. The acoustic model recording unit 70 includes an acoustic model 701 and a language model 702. The acoustic model 701 is realized by, for example, a hidden Markov model (HMM) widely used for continuous word speech recognition. The language model 702 is a word N-gram probability model, and includes a word pronunciation dictionary that holds part-of-speech attribute information, pronunciation information, and the like.

The correct example language model recording unit 71 records a correct example language model that is a correct language symbol sequence corresponding to the feature amount sequence X of the input speech signal. A feature quantity sequence group Z = {X ₁ , X ₂ , X ₃ ,... The correct example identification function value generation unit 73 receives the feature quantity sequence X and the correct answer R (X) as an input, refers to the correct example language model recording unit 71, and receives the correct symbol series R (to which the feature quantity series X belongs). X) (hereinafter referred to as positive example symbol series R (X)) is output as an identification function value G (expression (1)) for evaluating whether or not it corresponds.

  Here, Λ is a set of model parameters possessed by the symbols recorded in the acoustic model recording unit 70.

  The correct example speech recognition unit 74 rearranges the correct example symbol series R (X) and the discriminant function value G, and outputs a speech recognition result estimated to be correct.

  The speech recognition unit 76 receives the feature amount series X and its correct answer R (X) as input, and other than the positive example symbol series R (X), that is, a symbol series S other than the correct answer (hereinafter, negative example symbol series S (S ≠ R (X)) is generated, and the feature amount series X and the negative example symbol series S (S ≠ R (X)) are output to the negative example identification function value generation unit 75.

  The negative example identification function value generation unit 75 refers to the acoustic model recording unit 70 and evaluates whether or not the feature amount series X of the input audio signal corresponds to the negative example symbol series S (S ≠ R (X)). A discrimination function value G￣ (formula (2)) is output. The notation of G￣ is correct in the formula and the figure.

Here, W is a set of all assumed symbol sequences. P _ΛA (X | S) is a probability that the feature amount sequence of speech uttered with the intention of the negative example symbol sequence S (S ≠ R (X)) is X, and is calculated using an acoustic model. The P _ΛL (S) is a prior probability regarding the appearance of the negative example symbol sequence S (S ≠ R (X)), and is calculated using a language model. η is a coefficient that controls the effect of prior probability P _ΛL (S) that can be artificially determined. The larger η is, the larger the contribution of prior probability P _ΛL (S) in equation (2) is.

  The positive example / negative example comparison unit 77 receives an identification function value G that evaluates a positive example symbol series and an identification function value G 評 価 that evaluates a negative example symbol series S, and uses all these discrimination function values G￣. A misidentification scale d (X; Λ) shown in Expression (3), which is a misclassification scale for the feature amount series X of the input speech signal, is output.

  This misidentification scale d (X; Λ) means that the maximum value among the discriminant function values G￣ given by a plurality of negative example symbol sequences S (S ≠ R (X)) and the positive example symbol sequences are This is a difference from the given discriminant function value G. If this takes a positive value, the discriminant function value G￣ of at least one negative example symbol sequence S (S ≠ R (X)) is equal to that of the positive example symbol sequence R (X). It exceeds the discrimination function value G, and the feature amount X of the input audio signal is erroneously discriminated.

  The misidentification scale d (X; Λ) can be expressed more generally by the equation (4).

In this way, in the second term on the right side of Equation (4), it is possible to consider a misidentification scale that takes into account the influence of more negative symbol sequences. Here, h (•) is an arbitrary monotonically increasing reversible function, and h ⁻¹ (•) is its inverse function. In continuous word speech recognition, the misidentification scale d (X; Λ) is defined by, for example, Expression (5).

Here, φ is a positive constant, and as φ is larger, the second term on the right side satisfies S ≠ R (X).
The exp (g (X, S; Λ) having the maximum value is dominant.

  The model parameter optimization unit 78 receives the misidentification measure d (X; Λ) as an input and defines a loss function loss (d) representing the magnitude of loss caused by the misidentification measure d (X; Λ). Find the model parameter Λ where the loss is minimized. As the loss function, for example, a continuous nonlinear one as shown in the equation (6) can be considered.

  The loss function loss (d) in equation (6) takes a value between 0 and 1 according to the value of the misclassification measure d (X; Λ) in a narrow region around the symbol sequence boundary where d = 0. Asymptotically approaches 0 when <0, and asymptotically approaches 1 when d> 0. Further, as the simplest linear loss function loss (d), equation (7) is conceivable.

式（７）の損失関数の場合は、損失値は誤識別尺度ｄ（Ｘ；Λ）と一致した値となる。

In the case of the loss function of Equation (7), the loss value is a value that matches the misclassification measure d (X; Λ).

Now, a group of feature quantity series groups Z = {X ₁ , X ₂ , X ₃ ,...}, And positive example symbol series {R (X ₁ ), R (X ₂ ), for each of the feature quantity series groups Z When R (X ₃ ),...} Is given as learning data, the total loss L (Z; Λ) of the entire feature quantity sequence group Z is obtained by Expression (8).

  Finding the model parameter Λ that minimizes the total loss L (Z; Λ) by the optimization method corresponds to enhancing the discrimination ability of the discriminative learning method. As an optimization method, a probabilistic effect (PD: Probabilistic Descent) method, a Quickprop method, or the like can be used.

In addition, in a maximum mutual information (MMI) learning method, which is another typical example of discriminative learning, a model that maximizes the MMI objective function F _MMI (Z; Λ) defined by Equation (9). The parameter Λ is found by an optimization method.

Here, W ′ is a symbol sequence W ′ (W′⊂W) obtained as a result of continuous word speech recognition for the entire assumed symbol sequence W. Further, when equation (5) is modified, it can be expressed by equation (10).

Thus, f _MMI (X, Λ) and the misclassification measure d (X; Λ) in the MCE learning method can be calculated by substantially the same procedure. In particular, the minimization of the total loss when the linear loss function (formula (6)) is applied in the MCE learning method is almost the same procedure as that of the MMI objective function.

In the calculations of Expressions (9) and (10), a plurality of opposing positive example word series and negative example word series obtained as a result of continuous word speech recognition of the feature amount series X are used. In discriminative learning, it is important to consider a wider variety of recognition errors by using a sufficiently large number of positive and negative recognition word sequences. For this reason, the calculation of Expressions (9) and (10) is performed using a word lattice or the like that can efficiently express a word sequence group composed of a large number of words with a word network structure. Then, the model parameter Λ is optimized so that the total loss is minimized by using the positive example word sequence as teacher information.

[Juang & Katagiri 92] Biing-Hwang JUANG and Shigeru KATAGIRI; Discriminative Learning for Minimum Error Classification, IEEE, Trans. On SP., Vol. 40, No. 12, pp. 3043-3054 (1992). [Katagiri et al., 98] Shigeru KATAGIRI, Biing-Hwang JUANG and Chin-Hui LEE; Pattern Recognition Using a Family of Design Algorithms Based Upon the Generalized Probabilistic Descent Method, Proc.IEEE, Vol. 86, No. 11, pp 2345-2373 (1998). "McDermott & Katagiri, 97" Erik MCDERMOTT and Shigeru KATAGIRI; String-Level MCE for Continuous Phoneme Recognition, Proc.Eurospeech97, Vol. 1, pp. 123-126 (1997). [McDermott et al., 07] E. McDermott, T. Hazen, J. Le Roux, A. Nakamura, and S. Katagiri: Discriminative training for large vocabulary speech recognition using Minimum Classification Error, IEEE Transactions on Audio, Speech and Language Processing, vol. 15, no. 1, pp. 203-223 (2007). [Macherey et al., 05] W. Macherey, L. Haferkamp, R. Schlueter, and H. Ney: Investigations on error minimizing training criteria for discriminative training in automatic speech recognition, in Proc. Interspeech '05-Eurospeech, pp. 2133-2136 (2005).

  In the conventional discriminative learning method, a positive example symbol series and a negative example symbol series are calculated separately, and each misclassification measure is minimized or a difference between respective discriminant function values is maximized. I was seeking a symbol series. This causes the following problems. First, there is a problem that the required calculation resource amount is large due to the necessity of obtaining the discrimination function values of the positive example symbol series and the negative example symbol series.

Secondly, there is a problem that selection of a symbol series according to a positive example is arbitrarily and manually performed. Compared with the positive example symbol series R (X) of the feature quantity series X, a plurality of negative example symbol series {R ₁ ′ (X), R ₂ ′ (X) having a slight difference in the meaning of the sentence , R ₃ ′ (X),...} Are treated as equivalent to the positive example, and the knowledge amount of the positive example is increased. By increasing the amount of knowledge, it is possible to realize an identification learning device that generates a more robust model parameter for a feature amount sequence not included in the learning data. However, the selection of the plurality of negative example symbol sequences {R ₁ ′ (X), R ₂ ′ (X), R ₃ ′ (X),...} Is arbitrarily performed.

  The present invention has been made in view of the above points, and reduces the amount of calculation by generalizing the distinction between positive examples and negative examples for each symbol series as the degree of difference from the positive examples. A difference-based discriminative learning device that automatically reflects symbol sequences according to examples in an objective function based on objective criteria and does not need to manually select symbol sequences according to positive examples, and a method and program thereof The purpose is to provide.

  The discriminating-type discriminating learning device of the present invention includes a model parameter recording unit, a pattern recognition unit, an identification function value generation unit, a difference calculation unit, a positive example recognition comparison unit, a model parameter optimization unit, It comprises. The model parameter recording unit records model parameters. The pattern recognition unit generates a recognition symbol series obtained by pattern recognition of the learning data. The discriminant function value generation unit refers to the model parameter recording unit and outputs an discriminant function value for evaluating whether or not the recognized symbol sequence corresponds to the feature amount of the learning data. The difference calculation unit calculates a difference between the recognized symbol series and the positive example. The positive example recognition comparison unit receives N (N ≧ 2) attenuation coefficients, discriminant function values, and dissimilarities as inputs, and uses the N attenuation coefficients to determine the positive example side integrated value and the positive example side integrated value. An integrated value for correction is obtained, and an objective function obtained by correcting the positive-side integrated value is output. The model parameter optimization unit optimizes the model parameter corresponding to the recognized symbol sequence using the objective function.

According to the discriminant learning device using the degree of difference of the present invention, the difference degree estimation unit estimates the degree of difference between the recognized symbol sequence and the positive example, and the distinction between the positive example and the negative example is generally used as the degree of difference with respect to the positive example. Turn into.
Then, the model parameter is optimized by an objective function using the degree of difference. Therefore, unlike the conventional discriminative learning device, pattern recognition processing for generating a negative example symbol series is not required. Further, since it is not necessary to calculate the identification function value of the negative example symbol series, the required calculation resource amount can be reduced.

  In addition, since the recognized symbol series according to the positive example and the recognized symbol series according to the positive example are automatically weighted by the attenuation coefficient and reflected in the objective function, it is not necessary to select the symbol series according to the positive example arbitrarily and manually.

The figure which shows the function structural example of the difference degree utilization type | formula discriminative learning apparatus 100 of this invention. The figure which shows the operation | movement flow of the dissimilarity utilization type discriminative learning apparatus 100. FIG. The figure which shows the function structural example of the positive example recognition comparison part 14 by the maximum mutual information learning method. The figure which shows the operation | movement flow of the positive example recognition comparison part. The figure which shows the function structural example of the positive example recognition comparison part 50 by the minimum identification error learning method. The figure which shows the operation | movement flow of the positive example recognition comparison part. The figure which shows an example of a function structure of the conventional discriminative learning apparatus.

  The discriminant-based discriminative learning device of the present invention has functions corresponding to a positive example side speech recognition unit 74 and a positive example discrimination function value generation unit 73 that perform pattern recognition of a positive example symbol series of the conventional discriminative learning device 700. New in that no configuration is required. Before describing the embodiments of the present invention, the basic idea of the present invention will be described.

[Basic idea]
The discriminant-based discriminative learning method of the present invention abstracts the distinction between positive examples and negative examples for each recognition symbol sequence by using the degree of difference with respect to the positive example, and weights the discriminant function value based on the degree of difference. It is a method of learning using the sum.

  First, a function Δ (V, S) representing the degree of difference between two arbitrary symbol sequences V and S is introduced. As a method for realizing the function Δ (V, S), for example, an edit distance between V and S can be used. When V and S are associated with a common feature quantity sequence X, a difference scale based on the correspondence between each symbol constituting a symbol series and each feature quantity constituting a feature quantity series (reference) Literature: J. Zheng and A. Stolcke: Improved Discriminative Training Using Phone Lattices, in Proc. Interspeech, pp. 2125-2128, (2005)).

The difference Δ (R (X), S) between the positive example symbol series R (X) and the arbitrary recognition symbol series S (S∈W ′) is an error of the recognition symbol series S (S∈W ′). It can be regarded as a scale. Using this difference Δ (R (X), S), a new objective function F ⁺ (Z; Λ) can be defined as shown in equation (11). F ⁺ means that it is proposed in the present invention.

Each term of f ⁺ (X; Λ) passes the discriminant function value g (X, S; Λ) corresponding to each recognition symbol sequence S (S∈W ′) through an arbitrary monotonically increasing function h (•), This value is multiplied by an exponential function value obtained by multiplying the difference Δ (R (X), S) from the positive example by the attenuation coefficient σ. In other words, it consists of a value that exponentially decays and a sum of loads. By appropriately setting the attenuation coefficients σ ₁ and σ ₂ , discriminative learning can be performed by maximizing the objective function F ⁺ (Z; Λ).

For example, by setting the attenuation coefficient σ ₁ to a large value, the first term on the right side of the equation (11) becomes a larger value as the dissimilarity Δ (R (X), S) is smaller. Therefore, a positive example of the difference degree Δ (R (X), S) = 0 and a recognition symbol series S (S∈W ′) having a very small difference degree Δ (R (X), S) according to the positive example. The effect of the function value becomes dominant. When the attenuation coefficient σ ₂ = 0, the weight values in the second term on the right side of Equation (11) are all 1, and all discriminant function values in the recognition symbol sequence group (S _* ∈W ′) are treated fairly. That is, the value of the first term on the right side of Equation (11) is a weighted sum of the discriminant function values of the symbol series that are close to the positive example or the positive example, and the second term on the right side is mostly the discriminant function of the symbol series of the negative example. Cumulative value.

As described above, according to the present invention, it is possible to generate an objective function by a single pattern recognition. Therefore, the amount of calculation can be reduced as compared with the conventional method. Further, the recognized symbol sequence S (S∈W ′) according to the positive example and the positive example is automatically set according to the magnitude of the difference by the exponential attenuation coefficient σ ₁ of the first term on the right side of the second expression of the equation (11). Can be weighted and reflected in the objective function. As a result, it is not necessary to manually select a symbol series according to the positive example as in the conventional method.

  Embodiments of the present invention will be described below with reference to the drawings. The same reference numerals are given to the same components in a plurality of drawings, and the description will not be repeated.

  FIG. 1 shows an example of the functional configuration of the discriminant utilization type discriminative learning device 100 of the present invention. The operation flow is shown in FIG. The dissimilarity utilization type discriminative learning device 100 includes a model parameter recording unit 12, a discriminant function value generation unit 11, a pattern recognition unit 10, a dissimilarity calculation unit 13, a positive example recognition comparison unit 14, and a model parameter optimum. And the conversion unit 15. The dissimilarity utilization type discriminative learning device 100 is realized by reading a predetermined program into a computer composed of, for example, a ROM, a RAM, a CPU, and the like, and executing the program by the CPU.

The dissimilarity utilization type discriminative learning device 100 outputs an optimized model parameter Λ _m by using a feature value sequence X of learning data and a correct example symbol sequence R (X) as a correct answer thereof as input signals. 1, 3, and 5 are represented by a large number of feature quantity sequences {X ₁ , X ₂ , X ₃ ,...} And a number of positive example symbol sequences {R (X ₁ ), R (X ₂ ), R (X ₃ ),...} Are denoted as X _* and R (X _* ). Note that this notation is not used in the text.

  The model parameter recording unit 12 records model parameters corresponding to a recognition target symbol sequence including an acoustic model and a language model. The pattern recognition unit 10 generates a recognition symbol series S obtained by pattern recognition of the feature quantity series X of the learning data input from the outside (step S10). The discriminant function value generation unit 11 receives the recognition symbol sequence S as an input and refers to the model parameter recording unit 12 to evaluate whether or not the recognition symbol sequence S corresponds to the feature amount sequence X of the learning data. g (X, S; Λ) is output (step S11).

  The discriminant function value g (X, S; Λ) is input to the positive example recognition comparison unit 14 via the pattern recognition unit 10. The dissimilarity calculation unit 13 receives the positive example symbol series R (X) corresponding to the feature amount series X of the learning data and the recognition symbol series S as input, and calculates the dissimilarity Δ (R (X, S)) between them. (Step S13).

The positive example recognition / comparison unit 14 has predetermined N (N ≧ 2) attenuation coefficients σ ₁ and σ ₂ , the discrimination function value g (X, S; Λ), and the difference Δ (R (X, S )) As inputs, and the N-type attenuation coefficient is used to obtain a positive-side integrated value and an integrated value for correcting the positive-side integrated value, and the objective function f ⁺ ( X; Λ) is output (step S14).

Model parameter optimization unit 15, ⁺ objective function f model parameters corresponding to the recognized symbol sequence in the set lambda parameter as a larger;; ^(Λ X) objective function f ⁺ using ^(X lambda) Optimize (step S15). The model parameter optimization unit 15 optimizes the model parameter until the increment of the objective function f ⁺ (X; Λ) becomes a value smaller than a predetermined convergence condition threshold value.

As described above, the discriminant learning device 100 using the degree-of-difference according to the first embodiment has a pattern recognition unit (positive speech recognition unit 74 for positive examples) and a positive example identification required by the conventional discriminative learning device 700. There is no functional configuration corresponding to the function value generation unit 73. The objective function f ⁺ (X; Λ) is generated by one pattern recognition operation of the pattern recognition unit 10. Therefore, the calculation amount can be reduced as compared with the conventional discriminative learning device 700.

  The pattern recognition unit 10, the discrimination function value generation unit 11, the model parameter recording unit 12, and the model parameter optimization unit 15 according to the first embodiment are respectively a speech recognition unit 76 and a negative example discrimination function of the conventional discriminative learning device 700. The operations correspond to the value generation unit 75 and the model parameter optimization unit 78, and the operations are the same.

  In the dissimilarity utilization type discriminative learning device 100, the functional configurations of the dissimilarity calculation unit 13 and the positive example recognition comparison unit 14 are new. In the following description, only this new configuration will be described. Note that, in the dissimilarity utilization type discriminative learning device 100, an example in which the correct symbol sequence R (X) corresponding to the feature amount sequence X of the learning data is input has been described. However, the correct symbol sequence R (X) is input. Even if there is no difference, the dissimilarity utilization type discriminative learning device 100 of the present invention can be realized.

[Modification]
FIG. 1 shows a functional configuration example of the dissimilarity utilization type discriminative learning device 100 ′ of the modification of the first embodiment that does not require the input of the correct symbol series R (X) by a broken line. In the modification, only the input signal of the dissimilarity calculation unit 13 ′ and its operation are different. The dissimilarity estimation unit 13 ′ receives as input a feature quantity sequence X of learning data, a recognition symbol series S (S∈W ′) obtained by pattern recognition of the feature quantity series X, and an identification function value g (X, S; Λ). Then, an estimated value Δ ^ (S) of the difference Δ (R (X, S)) between the recognized symbol sequence S (S∈W ′) and the positive example is estimated. The estimated difference Δ ^ (S) can be calculated, for example, by estimating the certainty factor θ (S) of the recognition result by the method of [Wessel et ai., 01]. (Reference: F. Wessel, R. Schiuter, K. Macherey, and H. Ney: Confidence Measures for Large Vocabulary Continuous Speech Recognition, IEEE Transactions on Speech and Audio Processing, vol. 9, no. 3, pp. 288- 298 (2001))

The certainty factor θ (S) is a scale that represents the degree of confidence that the recognition result is a correct answer. For example, consider the certainty of S ₁ among a plurality of recognition symbol series S ₁ , S ₂ , S ₃ ,... Obtained by continuous pattern recognition of the feature quantity series X. If the discriminant function value _{g (X 1, S 1,} Λ) the value of _{g (X 2, S 2,} Λ), (X 3, S 3, Λ), larger projects in comparison ... and, _{S 1} It can be considered that the certainty that is correct is large.

Conversely, when many of g (X ₂ , S ₂ , Λ), (X ₃ , S ₃ , Λ),... Have the same value as g (X ₁ , S ₁ , Λ), confidence that S ₁ is the correct answer is reduced. In addition, considering the validity of the length of the partial feature quantity sequence corresponding to each symbol constituting S ₁ and the validity of the symbols constituting S ₁ being present in the same symbol series. , S ₁ represents a degree of reliability as a correct answer by a numerical value θ (S ₁ ) of 0 to 1 or less.

The estimated difference Δ ^ (S ₁ ) between S ₁ and the positive example symbol series is closer to 0 as S ₁ is more reliable as a correct answer, that is, as the numerical value θ (S ₁ ) is closer to ₁ , and S ₁ is taken as a correct answer. Determine to be unreliable. This difference degree estimated value Δ ^ (S ₁ ) can be calculated by, for example, Expression (12).

  By using this difference degree estimation value Δ ^ (S), it is possible to execute discriminative learning even if the positive symbol series R (X) is not explicitly given. In the conventional discriminative enemy learning, since the discriminative learning cannot be performed unless the positive example symbol sequence is given, it is necessary to assign the positive example symbol sequence to all of a large amount of data. However, by providing the dissimilarity calculation unit 13 ′ for calculating the dissimilarity estimated value Δ ^ (S), it is not necessary to prepare a normal example symbol series manually.

FIG. 3 shows a more specific functional configuration example of the positive example recognition / comparison unit 14, and the first embodiment will be described in more detail. The operation | movement flow of FIG. 4 is shown. FIG. 3 shows an implementation example of the calculation of the objective function f ⁺ (X; Λ) in the MMI learning method.

  The positive example recognition comparison unit 14 includes a discriminant function smoothing / reverse logarithmizing unit 140, a positive example side load unit 141, a recognition side load unit 142, a positive example side integration / logarithmization unit 143, and a recognition side integration / A logarithmizing means 144 and an integrated value comparing means 145 are provided. In addition, although the example which provides one each of discriminant function smoothing / antilogarithmizing means 140, positive example side load means 141, and recognition side load means 142 is shown, recognition symbol series S (S Each means 140, 141, 142 corresponding to εW ′) may be provided to simultaneously process the discrimination function value g (X, S; Λ) and the difference Δ (R (X), S). . The example shown in FIG. 3 is an example of a functional configuration of a method for processing a recognition symbol sequence S (SεW ′), which is input in large numbers, at different times.

  The discriminant function smoothing / antilogarithmizing unit 140 calculates the discriminant function smoothing value A by the equation (13) (step S140).

  The discriminant function smoothing value A is an exponential function value obtained by multiplying the discriminant function value g (X, S; Λ) by a predetermined positive constant φ.

  The positive example side load means 141 calculates the positive example side load value B by the equation (14) (step S141).

The positive side load value B is a value obtained by multiplying an exponential function value obtained by multiplying the degree of difference Δ (R (X), S) by the first attenuation coefficient −σ ₁ by the discrimination function smoothing value A. .

  The recognition-side load means 142 calculates the recognition-side integrated value C using equation (15) (step S142).

The recognition-side load value C is a value obtained by multiplying an exponential function value obtained by multiplying the degree of difference Δ (R (X), S) by the second attenuation coefficient −σ ₂ by the discrimination function smoothing value A.

The positive example side integration / logarithmization means 143 calculates the positive example side integration value D by the equation (16) (step S143).

  The positive example side integrated value D is a logarithmic function value of the sum of the positive example side load values B for all the recognized symbol sequences S (SεW ′).

  The recognizing side integration / logarithmizing means 144 calculates the recognizing side integrated value E by the equation (17) (step S144).

The recognition-side integrated value E is a recognition-side load value C corresponding to all recognition symbol sequences S (S∈W ′).
Is a logarithmic function value of the accumulated value, and an integrated value for correcting the positive example side integrated value.

The integrated value comparison means 145 outputs the objective function f ⁺ (X; Λ) shown in the equation (18) with the positive example integrated value D and the recognition integrated value C as inputs (step S145).

In Expression (18), for example, by taking a sufficiently large attenuation coefficient σ ₁ and maximizing it by various optimization methods with σ ₂ = 0, the conventional MMI learning method can be compared with the conventional MMI learning method with a smaller required calculation resource amount. Equivalent recognition accuracy can be improved. The method for reducing the required calculation resource amount is not limited to the configuration of the first embodiment, and another method will be described as a second embodiment.

FIG. 5 shows a functional configuration example of the positive example recognition / comparison unit 50 of the second embodiment. The operation flow is shown in FIG. The positive example recognition / comparison unit 50 includes a discriminant function smoothing / antilogarithmizing unit 140 and a positive example side loading unit 1.
41, positive example side integration / logarithmization means 143, negative example side load means 242, negative example side integration / logarithmization means 244, and integrated value comparison means 245. Discriminant function smoothing / antilogarithmization means 14
The positive example side load means 141 and the positive example side integration / logarithmization means 143 are the same as the positive example recognition comparison unit 14 of the first embodiment, and the positive example side integration / logarithmization means 143 is the positive example side. The integrated value D (Formula (16)) is output. Therefore, description of this part is omitted.

  The negative example side load means 242 calculates the negative example side integrated value K by the equation (19) (step S242).

The negative example-side integrated value K is an identification function smoothed by an exponential function value obtained by multiplying the degree of difference Δ (R (X), S) by a _second attenuation coefficient σ ₂ smaller than the first attenuation coefficient σ ₁ . The first negative example side load value multiplied by the value A and the third attenuation coefficient σ ₃ larger than the second attenuation coefficient are identified as exponential function values obtained by multiplying the difference Δ (R (X), S) by the difference Δ The second negative example side load value multiplied by the function smoothing value A is calculated, and the positive example side integrated value D is corrected by subtracting the second negative example side load value from the first negative example side load value. It is calculated as the negative example side integrated value that is the integrated value of.

  The negative example side integration / logarithmizing means 244 calculates the negative side integration value L by the equation (20) (step S244).

The negative example side integration value L is a logarithmic function value of the sum of the negative example side integration values L for all recognition symbol sequences S (SεW ′).

The integrated value comparison means 245 calculates the identification measure d (X; Λ) obtained by subtracting the negative example-side integrated value L from the positive example-side integrated value D by the equation (21), and the identification measure d (X; Λ) is lost. The objective function loss (d ⁺ (X; Λ)) passed through the function is output. The loss function is, for example, as shown in the above equation (6).

If the attenuation coefficient σ ₁ in the positive example side integrated value D is increased and the attenuation coefficient σ ₂ = 0 in the negative example side integrated value L, for example, σ ₃ ≈σ ₁ , a negative example as an opposing symbol sequence in learning is used. Only symbol sequences can be used. That is, by setting the attenuation coefficient σ3 to a value close to the attenuation coefficient σ1, the negative example symbol series that is very close to the positive example and the positive example can be deleted from the negative example side symbol series.

As described above, according to the discriminant learning device using the degree of difference provided with the positive example recognition comparison unit 50, the MCE
The recognition accuracy equivalent to that of the MCE learning method can be improved with a smaller amount of required calculation resources than the learning method.

[Evaluation experiment]
An experiment for acquiring model parameters was performed using the discriminating-type discriminative learning device 100 of Example 1 of the present invention. In the experiment, about 230 hours of speech for a Japanese conference was used as learning data. First, when the initial model was learned by the maximum likelihood learning method which is an existing technique and the continuous word speech recognition apparatus was operated using the initial model as it was, the word error rate was 21.2%.

The word error rate in the case of using the model parameters obtained by the existing MMI learning method using the same learning data for 21.2% was 18.6%. Using this word error rate as a comparison target, model parameters obtained by setting the second attenuation coefficient σ ₂ = −4 and the first attenuation coefficient σ ₁ as ₁ , 2, 3, 4 of the dissimilarity utilization type discriminative learning device 100 Table 1 shows the word error rates of the continuous word speech recognition results obtained from the above.

The word error rate when the continuous word speech recognition apparatus is operated with the model parameters by the discriminative learning apparatus 100 using the degree-of-difference of the present invention is 18.5% to 18.7%, and the word error by the conventional MMI learning method A result equivalent to the rate was obtained. In addition, although the comparison experiment of Example 2 of this invention and the conventional MCE learning method is not implemented, it is estimated that it becomes the same level word error rate similarly to the comparison result of the MMI learning method.

The dissimilarity utilization type discriminative learning device described above can be used for a speech recognition device, for example. Also, as other uses, it may be applied to a recognition apparatus that uses a feature quantity series that changes on the time axis, the space axis, or both of a still image, a moving image, etc., and expresses some conceptual information as a pattern authentication target. Is possible. As a specific example, it can be used for pattern recognition of image information of handwritten characters.

Although the positive constant φ and the attenuation coefficients σ _{1 to} σ ₃ have been described as examples set in advance in the positive example recognition comparison units 14 and 50, these values may be given from the outside. Further, the processes described in the above method and apparatus are not only executed in time series according to the order of description, but also may be executed in parallel or individually as required by the processing capability of the apparatus that executes the processes. Good.

  Further, when the processing means in the above apparatus is realized by a computer, the processing contents of functions that each apparatus should have are described by a program. Then, by executing this program on the computer, the processing means in each apparatus is realized on the computer.

  The program describing the processing contents can be recorded on a computer-readable recording medium. As the computer-readable recording medium, for example, any recording medium such as a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory may be used. Specifically, for example, as a magnetic recording device, a hard disk device, a flexible disk, a magnetic tape or the like, and as an optical disk, a DVD (Digital Versatile Disc), a DVD-RAM (Random Access Memory), a CD-ROM (Compact Disc Read Only). Memory), CD-R (Recordable) / RW (ReWritable), etc., magneto-optical recording medium, MO (Magneto Optical disc), etc., semiconductor memory, EEP-ROM (Electronically Erasable and Programmable-Read Only Memory), etc. Can be used.

  The program is distributed by selling, transferring, or lending a portable recording medium such as a DVD or CD-ROM in which the program is recorded. Further, the program may be distributed by storing the program in a recording device of a server computer and transferring the program from the server computer to another computer via a network.

  Each means may be configured by executing a predetermined program on a computer, or at least a part of these processing contents may be realized by hardware.

Claims (12)

A model parameter recording unit for recording model parameters;
A pattern recognition unit that generates a recognition symbol sequence obtained by pattern recognition of a feature amount sequence of learning data;
An identification function value generation unit that outputs an identification function value for evaluating whether or not the recognition symbol sequence corresponds to a feature amount sequence of the learning data with reference to the model parameter recording unit;
A difference calculation unit for calculating a difference between the recognition symbol series and the positive example;
N predetermined attenuation coefficients (N ≧ 2), the discriminant function value, and the dissimilarity are input, and the positive example side integrated value and the positive example side integrated value are obtained using the N attenuation coefficients. A positive example recognition / comparison unit that calculates an integrated value for correction and outputs an objective function that corrects the positive example side integrated value;
A model parameter optimization unit that optimizes the model parameter corresponding to the recognition symbol sequence using the objective function;
Dissimilarity utilization type discriminative learning device comprising: The dissimilarity utilization type discriminative learning device according to claim 1,
The positive example recognition comparison unit includes two attenuation coefficients,
The positive example recognition comparison unit
Obtain an identification function smoothing value that is an exponential function value obtained by multiplying the above identification function value by a predetermined positive constant;
The exponential function value obtained by multiplying the degree of difference by the first attenuation coefficient is multiplied by the discriminant function smoothing value to obtain a positive example side load value, and the sum of the positive example side load values for all recognition symbol sequences can be arbitrarily set. Through the monotonically increasing reversible function of
An exponential function value obtained by multiplying the degree of difference by a second attenuation coefficient smaller than the first attenuation coefficient is multiplied by the discrimination function smoothing value to obtain a recognition-side load value, and the recognition for all recognition symbol sequences is performed. The recognition side integrated value, which is an integrated value for correcting the positive example side integrated value by passing the sum of the side load values through an arbitrary monotonically increasing reversible function,
A discriminant learning apparatus using the degree-of-difference, wherein a value obtained by subtracting the recognition-side integrated value from the positive example integrated value is output as an objective function. In the dissimilarity utilization type discriminative learning device according to claim 1 or 2,
The positive example recognition comparison unit
Discriminant function smoothing / antilogarithmizing means for multiplying the discriminant function value by a predetermined positive constant and calculating an exponential function value of the value as a discriminant function smoothing value;
Positive example side load means for calculating a positive example side load value obtained by multiplying the exponential function value obtained by multiplying the difference by the first attenuation coefficient by the discriminant function smoothing value;
Positive example side integration / logarithmizing means for calculating a logarithmic function value of the sum of the positive example side load values for all recognition symbol sequences as a positive example side integrated value;
Recognition-side load means for calculating a recognition-side load value obtained by multiplying the discriminant function smoothing value by an exponential function value obtained by multiplying the difference by the second attenuation coefficient smaller than the first attenuation coefficient;
Recognizing side integration / logarithm that outputs a logarithmic function value obtained by accumulating the above recognition side load values corresponding to all recognition symbol series as the above recognition side integration value, which is an integrated value for correcting the positive example side integration value And
Integrated value comparison means for outputting, as an objective function, a value obtained by subtracting the recognized integrated value from the positive example integrated value;
A discriminant learning device using the degree of difference, characterized by comprising: The dissimilarity utilization type discriminative learning device according to claim 1,
The positive example recognition comparison unit includes three attenuation coefficients,
The above objective function is a misclassification measure passed through a loss function,
The positive example recognition comparison unit
Obtain an identification function smoothing value that is an exponential function value obtained by multiplying the identification function value by a positive constant set in advance,
The exponential function value obtained by multiplying the degree of difference by the first attenuation coefficient is multiplied by the discriminant function smoothing value to obtain a positive example side load value, and the sum of the positive example side load values for all recognition symbol sequences can be arbitrarily set. Through the monotonically increasing reversible function of
A first negative example side load value obtained by multiplying the discriminant function smoothing value by an exponential function value obtained by multiplying the degree of difference by a second damping coefficient smaller than the first damping coefficient, and a second damping coefficient A second negative example side load value obtained by multiplying the exponential function value obtained by multiplying the degree of difference by the larger third attenuation coefficient by the discriminant function smoothing value, and calculating from the first negative example side load value. The negative example side load value is reduced to the negative example side load value, and the sum of the negative example side load values for all recognition symbol sequences is passed through an arbitrary monotonically increasing reversible function to correct the positive example side integrated value. It is calculated as a negative example side integrated value that is an integrated value for
The discriminant learning apparatus using the degree of difference is characterized in that the positive example side integrated value is subtracted from the negative example side integrated value to obtain the discrimination scale. In the dissimilarity utilization type discriminative learning device according to claim 1 or 4,
The positive example recognition comparison unit
Discriminant function smoothing / antilogarithmizing means for multiplying the discriminant function value by a predetermined positive constant and calculating an exponential function value of the value as a discriminant function smoothing value;
Positive example side load means for calculating a positive example side load value obtained by multiplying the exponential function value obtained by multiplying the difference by the first attenuation coefficient by the discriminant function smoothing value;
Positive example side integration / logarithmizing means for calculating a logarithmic function value of the sum of the positive example side load values for all recognition symbol sequences as a positive example side integrated value;
A first negative example side load value obtained by multiplying the discriminant function smooth value by an exponential function value obtained by multiplying the degree of difference by a second attenuation coefficient smaller than the first attenuation coefficient, and a second attenuation coefficient. A second negative example side load value obtained by multiplying the exponential function value obtained by multiplying the degree of difference by the large third attenuation coefficient and the discriminant function smoothing value is calculated, and the first negative example side load value is used to calculate the above value. A negative example side load means for calculating a value obtained by subtracting the second negative example side load value;
Negative side integration for accumulating the negative side load values corresponding to all recognized symbol series and outputting the logarithmic function value of the cumulative value as a negative example side integrated value for correcting the positive example side integrated value・ Logarithmization means,
An integrated value comparison means for subtracting the positive example side integrated value from the negative example integrated value as the discrimination scale, and outputting the discrimination scale passed through a loss function as an objective function;
A discriminant learning device using the degree of difference, characterized by comprising: The dissimilarity utilization type discriminative learning device according to any one of claims 1 to 5,
The dissimilarity calculation unit inputs the feature amount of the learning data, the recognition symbol sequence, and the discriminant function value, and estimates the dissimilarity as a certainty of a scale representing the degree to which the recognition symbol sequence is reliable as a correct answer. A difference utilization type discriminative learning device characterized by being output as a difference degree estimated value. A pattern recognition process in which a pattern recognition unit generates a recognition symbol sequence obtained by pattern recognition of a feature amount sequence of learning data;
An identification function value generation process in which an identification function value generation unit outputs an identification function value for evaluating whether or not the recognition symbol sequence corresponds to a feature quantity of the learning data with reference to a model parameter in a model parameter recording unit; ,
A difference calculation unit calculates a difference between the recognized symbol sequence and the positive example,
The positive example recognition / comparison unit receives N (N ≧ 2) attenuation coefficients, the discriminant function value, and the dissimilarity as inputs, and uses the N attenuation coefficients to integrate the positive example side value and its positive example side. A positive example recognition comparison process for obtaining an integrated value for correcting the integrated value and outputting an objective function in which the positive example side integrated value is corrected,
A model parameter optimization unit for optimizing the model parameter corresponding to the recognition symbol sequence using the objective function;
Including
The positive example recognition comparison process comprises two attenuation coefficients,
The positive recognition comparison process is as follows:
Obtain an identification function smoothing value that is an exponential function value obtained by multiplying the above identification function value by a predetermined positive constant;
The exponential function value obtained by multiplying the degree of difference by the first attenuation coefficient is multiplied by the discriminant function smoothing value to obtain a positive example side load value, and the sum of the positive example side load values for all recognition symbol sequences can be arbitrarily set. Through the monotonically increasing reversible function of
An exponential function value obtained by multiplying the degree of difference by a second attenuation coefficient smaller than the first attenuation coefficient is multiplied by the discrimination function smoothing value to obtain a recognition-side load value, and the recognition for all recognition symbol sequences is performed. The sum of the side load values is passed through an arbitrary monotonically increasing reversible function to obtain the integrated value for correcting the positive example side integrated value as the recognized side integrated value,
What is obtained by subtracting the recognition side integrated value from the positive example integrated value is output as an objective function. The difference degree utilization type discriminative learning method according to claim 7,
The positive recognition comparison process is as follows:
Discriminant function smoothing / antilogarithmizing means is a discriminant function smoothing / antilogarithmizing step for multiplying the discriminant function value by a positive constant and calculating an exponential function value of the value as a discriminant function smoothing value;
A positive example side load means for calculating a positive example side load value obtained by multiplying the exponential function value of the value obtained by multiplying the degree of difference by the first attenuation coefficient by the discrimination function smoothing value;
Positive example side integration / logarithmization means for calculating a logarithmic function value of the sum of the positive example side load values for all recognition symbol sequences as a positive example side integration value;
A recognition side load means for calculating a recognition side load value obtained by multiplying an exponential function value obtained by multiplying the degree of difference by a second attenuation coefficient smaller than the first attenuation coefficient by the discrimination function smoothing value; A side load step;
The recognition side integration / logarithmization means is an integration value for accumulating the recognition side load values corresponding to all recognition symbol sequences and correcting the logarithmic function value of the value to the positive example side integration value. Recognition side integration / logarithmization step to output as integrated value,
An integrated value comparing step, wherein the integrated value comparing means outputs the objective function obtained by subtracting the recognized integrated value from the positive integrated value;
A discriminative learning method using the degree of difference. The difference degree utilization type discriminative learning method according to claim 7,
The positive example recognition comparison process comprises three attenuation coefficients,
The objective function is a misclassification measure passed through a loss function,
The positive recognition comparison process is as follows:
Obtain an identification function smoothing value that is an exponential function value obtained by multiplying the identification function value by a positive constant set in advance,
The exponential function value obtained by multiplying the degree of difference by the first attenuation coefficient is multiplied by the discriminant function smoothing value to obtain a positive example side load value, and the sum of the positive example side load values for all recognition symbol sequences can be arbitrarily set. Through the monotonically increasing reversible function of
A first negative example side load value obtained by multiplying the discriminant function smoothing value by an exponential function value obtained by multiplying the degree of difference by a second damping coefficient smaller than the first damping coefficient, and a second damping coefficient A second negative example side load value obtained by multiplying the exponential function value obtained by multiplying the degree of difference by the larger third attenuation coefficient by the discriminant function smoothing value, and calculating from the first negative example side load value. The negative example side load value is reduced to the negative example side load value, and the sum of the negative example side load values for all recognition symbol sequences is passed through an arbitrary monotonically increasing reversible function to correct the positive example side integrated value. It is calculated as a negative example side integrated value that is an integrated value for
A method of discriminative learning using a degree of difference, which is a process of subtracting the positive example side integrated value from the negative example side integrated value to obtain the discrimination scale. The difference degree utilization type discriminative learning method according to claim 7,
The positive recognition comparison process is as follows:
A discriminant function smoothing / antilogarithmizing step, wherein the discriminant function smoothing / antilogarithmizing means multiplies the discriminant function value by a predetermined positive constant and calculates an exponential function value of the value as a discriminant function smoothing value; ,
A positive side load step for calculating a positive side load value obtained by multiplying the exponential function value obtained by multiplying the degree of difference by the first attenuation coefficient by the discrimination function smoothing value;
Positive example side integration / logarithmization means for calculating a logarithmic function value of the sum of the positive example side load values for all recognition symbol sequences as a positive example side integration value;
The negative example side load means obtains a first negative example side load value obtained by multiplying the degree of difference by a second attenuation coefficient smaller than the first attenuation coefficient, and a third attenuation coefficient larger than the second attenuation coefficient. A second negative example load value obtained by multiplying the exponential function value obtained by multiplying the degree of difference by the discriminant function smoothing value is calculated, and the second negative example side load value is calculated from the first negative example side load value. A negative example side load step that calculates a reduced value as a negative example side integrated value that is an integrated value for correcting the positive example side integrated value;
A negative integration / logarithmization step in which the negative integration / logarithmization means accumulates the negative load values corresponding to all recognition symbol sequences and outputs a logarithmic function value of the value as a negative example integration value;
An integrated value comparison step, wherein the integrated value comparison means subtracts the positive example integrated value from the negative example integrated value to make the identification scale, and outputs the identification scale passed through a loss function as an objective function;
A discriminative learning method using the degree of difference. The dissimilarity utilization type discriminative learning method according to any one of claims 7 to 10,
The degree-of-difference calculation process uses the feature amount of the learning data, the recognition symbol series, and the identification function value as inputs, and estimates the degree of difference as a certainty of a scale representing the degree to which the recognition symbol series is reliable as a correct answer. A discriminant utilization type discriminative learning method characterized in that the discriminating factor is output as a dissimilarity estimation value.   An apparatus program for causing a computer to function as the dissimilarity utilization type discriminative learning apparatus according to any one of claims 1 to 6.

Images