JP4801108B2 - Voice recognition apparatus, method, program, and recording medium thereof - Google Patents

Description

  The present invention relates to speech recognition technology. In particular, the present invention relates to a technique for improving the speed of voice recognition processing.

With reference to FIG. 7, a conventional speech recognition apparatus 100 ′ will be described.
The input voice is input to the acoustic analysis unit 10. The acoustic analysis unit 10 calculates a feature vector for each frame having a fixed time length from the input speech, and generates a time series of the feature vector. The time series of the generated feature vector is sent to the search unit 30 ′.
The search unit 30 ′ uses the acoustic model read from the acoustic model storage unit 40 to collate the word or word string expressed in the grammar read from the grammar storage unit 50 with the time series of the feature vector, that is, A search process is performed, and the word or word string having the highest likelihood is output as a recognition result.

  Cepstrum analysis is often used as a voice analysis method in the acoustic analysis unit 10. For example, there are MFCC (Mel Frequency Cessential Coefficient), ΔMFCC, ΔΔMFCC, logarithmic power, Δlogarithmic power, etc. as feature quantities, and these feature quantities constitute a feature quantity vector of about 10 to 100 dimensions. For example, the voice analysis is performed with a frame width of about 30 ms and a frame shift width of about 10 ms.

  The acoustic model stored in the acoustic model storage unit 40 is obtained by holding a feature amount of speech such as MFCC as a standard pattern in an appropriate category. Is used as a likelihood to estimate which category it belongs to.

  Currently, hidden Markov models (hereinafter abbreviated as HMMs) modeled on the basis of probability / statistical theory are widely used as acoustic models. Usually, the HMM is created for each phoneme category. Each HMM created for each phoneme category is called a phoneme HMM. A set of phoneme HMMs composed of a plurality of phoneme HMMs constructs one acoustic model.

As phoneme HMMs, monophone-HMM, biphone-HMM, and triphone-HMM are often used.
The monophone-HMM is a phoneme environment-independent phoneme HMM that does not consider both phonemes preceding and following the central phoneme as phoneme environments. For example, the monophone-HMM of phoneme a can be represented as * -a- *, where * is an arbitrary phoneme.

  The biphone-HMM includes a preceding phoneme environment-dependent phoneme HMM that considers only the phoneme preceding the central phoneme as the phoneme environment, and a subsequent phoneme environment-dependent phoneme HMM that considers only the phoneme following the center phoneme as the phoneme environment. is there. For example, the preceding phoneme environment-dependent biphone-HMM of the phoneme a whose preceding phoneme is p can be expressed as p-a- *. The subsequent phoneme environment-dependent biphone-HMM of the phoneme a whose subsequent phoneme is t can be expressed as * -at.

A triphone-HMM is a phoneme HMM that considers both phonemes preceding and following the central phoneme as phoneme environments. For example, a triphone-HMM of a phoneme a whose leading phoneme is p and whose subsequent phoneme is t can be expressed as p-at.
The biphone-HMM is a model that represents the phoneme environment in more detail than the monophone-HMM, and the triphone-HMM is more detailed than the biphone-HMM.
The number of types of phoneme categories expressed by the phoneme HMM depends on the learning data of the acoustic model, but it excludes those that are not possible as a Japanese phoneme chain such as ttt. Thousands to tens of thousands.

The structure of the phoneme HMM included in the acoustic model will be described with reference to FIGS. The phoneme HMM is composed of a plurality of states S as will be described later.
The state S is expressed as a mixed probability distribution as illustrated in FIG. Each element distribution of the mixed probability distribution includes a discrete probability distribution and a continuous probability distribution. Currently, the most commonly used one is a multidimensional normal distribution (also called a multidimensional Gaussian distribution). .) Among them, a multidimensional uncorrelated normal distribution in which there is no correlation between dimensions, that is, the diagonal component of the covariance matrix is 0 is most often used. Each dimension of the multidimensional normal distribution corresponds to each dimension of the feature vector.

In FIG. 8, the state S is expressed as a multidimensional mixed normal distribution M having four multidimensional normal distributions as element distributions. Although FIG. 8 shows a dimension i having a multidimensional normal distribution, the other dimensions of the multidimensional normal distribution are also expressed in the same manner.
A phoneme HMM is configured by a probability chain of several to about a dozen states S as exemplified in FIG. There are various variations in what probability chain of phoneme HMMs is composed of in what state. Also, different phoneme HMMs may have different structures.

The structure most commonly used at present is what is called a three-state left-to-right HMM as illustrated in FIG. This is an arrangement of three states consisting of a first state S ₁ , a second state S ₂ and a third state S ₃ from left to right. Probability chain of states, that is, state transitions are transitions to themselves (self-transitions) S ₁ → S ₁ , S ₂ → S ₂ , S ₃ → S ₃ and transitions S ₁ → S ₂ , S ₃ to the next state. consisting of ₂ → _{S 3} Metropolitan. All phoneme HMMs in an acoustic model often take the structure of this three-state left-to-right HMM.

The acoustic likelihood calculation of the phoneme HMM will be described. Specifically, acoustic likelihood calculation when a time series of a certain feature vector is input to the phoneme HMM in FIG. 9 will be described. For example, the time series X = X ₁ , X ₂ , X ₃ , X ₄ , X ₅ , X ₆ of feature quantity vectors for 6 frames is one state transition sequence S _e = S ₁ → S ₁ with a phoneme HMM. The acoustic likelihood P (X | S _e , HMM), which is the probability output from S ₂ → S ₂ → S ₃ → S ₃ , is calculated as follows.

Here, a _jk is a transition probability from the state S _j to the state S _k . Further, the state likelihood b _j (X _t ) is a probability that the feature quantity vector X _{t at} the time t, that is, the frame t is output from the multidimensional mixed normal distribution M _j representing the state S _j . The state likelihood b _j (X _t ) is calculated as follows using the output probability P _jm (X _t ) of the m-th multidimensional normal distribution constituting the multidimensional mixed normal distribution M _j .

Here, the mixture number m _j is the number of normal distributions constituting the multidimensional mixed normal distribution M _j , and W _jm is the distribution weight of the mth normal distribution constituting the multidimensional mixed normal distribution M _j. . For W _jm , the following equation is satisfied:

When the normal distribution constituting the multidimensional mixed normal distribution M _j is a multidimensional uncorrelated normal distribution, P _jm (X _t ) is calculated as follows.

Here, μ _jmi and σ _jmi ² are the average value and variance in the dimension i of the m-th multidimensional uncorrelated normal distribution constituting the multidimensional mixed normal distribution M _j . X _ti is the value of dimension i of X _t of the feature vector. I is the number of dimensions of the feature vector and the multidimensional uncorrelated normal distribution.

Acoustic likelihood calculations described above are those for a one state transition sequence S _e. Such a state transition sequence can also be given elsewhere. The probability of outputting a time series of feature vector for all such state transition series is calculated, and the time series X of the feature vector is input to the phoneme HMM by adding these calculated probabilities. The method of obtaining the acoustic likelihood is sometimes called a trellis algorithm.

  On the other hand, the state transition sequence that gives the highest acoustic likelihood among all the state transition sequences is sequentially obtained in units of frames by the time series of feature vectors, and the likelihood when the final frame is reached is characterized by the phoneme HMM. The method of obtaining the acoustic likelihood when the time series X of the quantity vector is input is called a Viterbi algorithm. In general, a Viterbi algorithm that can significantly reduce the amount of calculation compared to the trellis algorithm is often used.

  Further, the above-described acoustic likelihood calculation is for one phoneme HMM, but actually, the phoneme HMM is concatenated and stored in the grammar storage unit 50 before performing the search process in the search unit 30 ′. A search network for HMMs of words or word strings expressed in a grammar is created, and a time series of feature vectors of input speech is matched with words or word strings expressed in the search network, that is, search processing is performed. Then, the word or word string having the highest acoustic likelihood is output as the recognition result.

  In the case of continuous speech recognition, in addition to the acoustic likelihood described above, the language likelihood based on a language model that statistically expresses the ease of connection of words is taken into consideration, and the word having the highest integrated likelihood or Output as a word string. In the above-described acoustic likelihood calculation, the probability value is handled as it is. However, in order to prevent underflow, the logarithm of the probability value is actually used for calculation (for the above contents, for example, Non-Patent Document 1). , 2).

By the way, since the ratio of the time for calculating the state likelihood b _j (X _t ) in the speech recognition processing time is increased from 45% to 65%, the state likelihood b _j ( The processing for obtaining X _t ) may be accelerated. A number of techniques for speeding up the process of obtaining the state likelihood b _j (X _t ) have been proposed (see, for example, Non-Patent Documents 3 and 4).
Hereinafter, a method for speeding up the process for obtaining the state likelihood b _j (X _t ) described in Non-Patent Document 4 will be described. The method of Non-Patent Document 4 realizes speeding up of processing for obtaining the state likelihood b _j (X _t ) based on the following two experimental facts.

1. As a result of investigating the movement of the CPU in the calculation of the state likelihood b _j (X _t ), the time consumption is the longest in the calculation of the state likelihood b _j (X _t ) defined by the above equation (2). Rather, it is a process of fetching the state parameter of the state j to be calculated necessary for calculating the state likelihood b _j (X _t ) from the main memory to the CPU cache.

2. When the state likelihood b _j (X _t ) is calculated for a frame t in a certain state j, the state likelihood b _j (X _{t + 1} ) is calculated for the next frame t + 1 of the state _j. Probability is high. Non-Patent Document 4 describes that the state likelihood b _j (X _{t + 1} ) for the next frame t + 1 is calculated with a probability of 75% or more.

The method of Non-Patent Document 4 will be described with reference to the state likelihood table illustrated in FIG. The state likelihood table is a time series of frames for calculating the state likelihood b _j (X _t ) for each state.

For example, assume that it is necessary to calculate the state likelihood b _j (X _t ) for the frame t in the state j. At this time, not only the state likelihood b _j (X _t ) but also the state likelihoods b _j (X _{t + 1} ),..., B _j (X _{t + K} ) up to K frames ahead are calculated together, and these calculations are performed. Store the results in a table. The process of calculating the state likelihood up to K frames ahead is called “batch state likelihood calculation process”. K is an integer of about 7.

Thereafter, when it becomes necessary to calculate the state likelihoods b _j (X _{t + 1} ),..., B _j (X _{t + K} ), they are obtained by referring to the table without actually calculating them. Thus, it is possible to speed up the process of obtaining the status likelihood b _{j (X t).}

According to the method of Non-Patent Document 4, it is possible to reduce the number of times of fetching the state parameter having a long consumption time into the CPU cache as described in “1.”, so that the calculation of acoustic likelihood is accelerated. The speech recognition process can be speeded up.
Kiyohiro Shikano and 4 others, “IT Text Speech Recognition System”, Ohmsha, May 2001, p. 1-51 Akio Ando, “Real-time Speech Recognition”, The Institute of Electronics, Information and Communication Engineers, September 2003, p. 1-58, p. 125-170 Shigeki Hiyama, 4 others, “New acceleration in speech recognition”, Proceedings of the Acoustical Society of Japan, 1-5-12, March 1996, p. 25-28 M. Saraclar, three others, “Towards automatic closed captioning: low latency real time broadcast news transcription”, Proc. ICSLP'02, September 2002, p. 1741-1744

Incidentally, it is unclear whether the state likelihood b _j (X _{t + 1} ),..., B _j (X _{t + K} ) for the K frames calculated next is actually used. This is a wasteful calculation.
In the method of Non-Patent Document 4, since the value of K is fixed without considering various circumstances, there is a possibility that useless calculation of state likelihood has been performed. For this reason, there is a possibility that the voice recognition processing has not been sufficiently accelerated.

  In view of the above problems, an object of the present invention is to provide a speech recognition apparatus, method, program, and recording medium thereof that further improve the speed of speech recognition processing.

According to one aspect of the present invention, the acoustic model storage unit is a storage unit that stores an acoustic model including state parameters and self-transition probabilities, and the state parameter storage unit is a storage unit that is faster than the acoustic model storage unit. To do. The acoustic analysis unit obtains a feature vector for each frame having a predetermined time length from the input speech, and stores a time series of the feature vector in the feature vector storage unit. The state likelihood b _j (X _t ) is a state likelihood b _j (X _t ), where the fetch unit uses j and t as arbitrary integers, and a certain state j outputs the feature quantity vector X _t of the frame t. Is calculated from the acoustic model storage unit to the state parameter storage unit. In the speech recognition processing performed by the likelihood calculation rate calculation unit on the speech having an acoustic property close to the target speech to be processed, the ratio of frames in which the state likelihood is calculated in all frames ( Hereinafter, the likelihood calculation rate q _j is obtained for each state. The likelihood calculation rate frame number determination unit determines the larger integer K _B (j) as the frame number K as the obtained likelihood calculation rate q _j is higher. The state likelihood calculation unit calculates the state likelihood b _j (X _t ) using the state parameter of the state j read from the state parameter storage unit and the feature quantity vector X _t read from the feature quantity vector storage unit. In addition, state likelihood b _j (X _{t + 1} ),..., Using the state parameter of state j read from the state parameter storage unit and feature quantity vectors X _{t + 1} ,..., X _{t + K} read from the feature quantity storage unit. , B _j (X _{t + K} ) are further calculated, and the further calculated state likelihoods b _j (X _{t + 1} ),..., B _j (X _{t + K} ) are stored in the state likelihood storage unit. When the state likelihood reference unit needs any of the state likelihoods b _j (X _{t + 1} ),..., B _j (X _{t + K} ), the state likelihood storage unit refers to the state likelihoods. Ask for.

  By appropriately changing the value of the number K of frames according to the state, it is possible to reduce the amount of wasteful calculation processing of the state likelihood. As a result, the voice recognition processing can be speeded up as compared with the conventional art.

As exemplified in the state likelihood table described in FIG. 6, the present invention is characterized in that the number of frames K for which the state likelihood is calculated is appropriately changed for each state.
Hereinafter, embodiments of the present invention will be described with reference to the drawings. The same parts as those in the background art are denoted by the same reference numerals, and redundant description is omitted.

[First embodiment]
The first embodiment is an embodiment in the case where a voice (hereinafter referred to as “adaptation destination data” or “development data”) having a property that is acoustically close to the target voice that is the target of the voice recognition process is obtained.
For the development data, speech recognition processing was performed by normal state likelihood calculation without performing batch state likelihood calculation, for example, state likelihood calculation was performed for all frames by obtaining a state likelihood table. The ratio of frames (hereinafter referred to as likelihood calculation rate q _j ) is obtained for each state j. The state likelihood b _j (X _{t + 1} ) for the next frame t + 1 when the state likelihood b _j (X _t ) for a certain frame t is calculated as the state _j has a higher likelihood calculation rate q _j. It can be considered that the possibility of being calculated is high.

Using this property, a large frame number K is given to a state j with a high likelihood calculation rate q _j , and conversely a small frame number K is given to a state j with a low likelihood calculation rate q _j. . That is, the higher the likelihood calculation rate q _j , the larger the number of frames K is given.
In this way, by appropriately changing the number of frames K for which the state likelihood is calculated for each state, the amount of wasteful state likelihood calculation processing can be reduced. Therefore, it is possible to speed up the calculation of the acoustic likelihood as compared with the prior art, and to speed up the speech recognition process.

An example of the first embodiment of the present invention will be described with reference to FIGS. FIG. 1 is a functional block diagram of an example of a speech recognition apparatus. FIG. 2 is a flowchart illustrating the processing flow of the speech recognition method.
The speech recognition apparatus 100 according to the first embodiment includes an acoustic analysis unit 10, a feature vector storage unit 20, a search unit 30, an acoustic model storage unit 40, a grammar storage unit 50, a fetch unit 60, and a state indicated by a solid line in FIG. For example, a parameter storage unit 70, a state likelihood storage unit 80, and a frame number determination unit 90 are included. The search unit 30 includes a state likelihood calculation unit 31 and a state likelihood reference unit 32, for example. The frame number determination unit 90 includes, for example, a likelihood calculation rate calculation unit 92 and a likelihood calculation rate frame number determination unit 93.

<Step S1>
The input voice is input to the acoustic analysis unit 10. Acoustic analysis section 10, from the input speech, the feature vector X _t is calculated for each frame of a fixed time length, to generate a time series of feature vectors X _t. Time series of the generated feature vector X _t is sent to the feature quantity vector storage unit 20.
Feature quantity vector storage unit 20 is, for example, a buffer for temporarily storing the feature vectors X _t.

<Step S2>
Before the state likelihood calculation unit 31 calculates the state likelihood b _j (X _t ) for the frame t of the state j, the fetch unit 60 stores the state parameter of the state j as an acoustic model storage in which the acoustic model is stored. The data is read from the unit 40 and stored in the state parameter storage unit 70.
The state parameter is a numerical value necessary for calculating the state likelihood B _j (X _t ). For example, the distribution weight W _jm (m = 1, m) appearing in Equation (2) in the background art column. , _{M j} ), mean μ _jmi (m = 1,..., _{M j} , i = 1,..., I), variance σ _jmi ² (m = 1,..., _{M j} , i = 1, ..., I).

  The state parameter storage unit 70 is a storage medium that is faster in reading and writing than the acoustic model storage unit 40, and is, for example, the cache 1a of the CPU 1 (see FIG. 5).

<Step S31>
The likelihood calculation rate calculation unit 92 performs speech recognition processing on the development data by normal state likelihood calculation without performing batch state likelihood calculation, and obtains a likelihood calculation rate q _j for each state j. The likelihood calculation rate q _j is sent to the likelihood calculation rate frame number determination unit 93.

<Step S32>
The likelihood calculation rate frame number determination unit 93 determines an integer K _B (j) that is larger for a state j having a higher likelihood calculation rate q _j . K _B (j) is sent to the state likelihood calculation unit 31.
For example, q _l is a number between 0 and 1, q _h is a number between q _{l and} 1; K _min is an integer greater than or equal to 0; K _max is an integer greater than or equal to K _min +1; K _B (j) can be obtained by the following equation as a function that outputs an integer by rounding down. q _l , q _h , K _min, and K _max are numbers that are appropriately determined according to the target speech, hardware performance, target speech recognition processing speed, and the like. For example, q _l is set to 0.2 to 0.3, q _h is set to 0.7 to 0.8, K _min is set to 3 to 4, and K _max is set to 10 to 12.

すなわち、図３に例示するように、尤度計算率ｑ_ｊがｑ_ｌより下であればＫ_Ｂ（ｊ）＝Ｋ_ｍｉｎとし、尤度計算率ｑ_ｊがｑ_ｌ以上ｑ_ｈより下であればＫ_Ｂ（ｊ）＝ｆ（（Ｋ_ｍａｘ−Ｋ_ｍｉｎ）ｑ_ｊ／（ｑ_ｈ−ｑ_ｌ））＋（（Ｋ_ｍｉｎｑ_ｈ−Ｋ_ｍａｘｑ_ｌ）／（ｑ_ｈ−ｑ_ｌ）））とし、尤度計算率ｑ_ｊがｑ_ｈ以上であればＫ_Ｂ＝Ｋ_ｍａｘとする。

That is, as illustrated in FIG. 3, if the likelihood calculation rate q _j is lower than q _l, K _B (j) = K _min is set, and the likelihood calculation rate q _j is higher than q _{l and} lower than q _h. K _B (j) = f ((K _max −K _min ) q _j / (q _h −q _l )) + ((K _min q _h −K _max q _l ) / (q _h −q _l ))) and then, likelihood calculation factor _{q j} is a _K B _{= K max} equal to or greater than _{q h.}

In this way, a function K _B (j) that outputs a larger integer as the likelihood calculation rate q _j is higher is determined, and an individual frame number K is determined for each state according to this function.

<Step S4>
The state likelihood calculating unit 31 uses the parameter of the state j read from the state parameter storage unit 70 and the feature quantity vector X _{t of the} frame t read from the feature quantity vector storage unit 20 for the frame t of the state j. The state likelihood b _j (X _t ) is calculated. At the same time, using the parameters of the state j read from the state parameter storage unit 70 and the feature quantity vectors X _{t + 1} ,..., X _{t + K of the} frames t + 1,. The state likelihoods b _j (X _{t + 1} ),..., B _j (X _{t + K} ) for the frame t + 1,.

The calculated state likelihood b _j (X _t ) is used for calculation of acoustic likelihood by the search unit 30. On the other hand, the calculated state likelihoods b _j (X _{t + 1} ),..., B _j (X _{t + K} ) are stored in the state likelihood storage unit 80.

<Step S5>
When the search unit 30 needs any of the state likelihoods b _j (X _{t + 1} ),..., B _j (X _{t + K} ) in order to calculate the acoustic likelihood, the state likelihood reference unit 32 determines the state likelihood. The state likelihood is obtained by referring to the degree storage unit 80.
The search unit 30 calculates the acoustic likelihood using the state likelihood obtained by the state likelihood reference unit 32, and outputs a speech recognition result.

[Second Embodiment]
The first embodiment uses the self-transition probabilities a _jj of each state of the phoneme HMM together with the likelihood calculation rate q _j . The duration of a vowel such as / a / is usually longer than the duration of a consonant such as / p /. For this reason, the self-transition probability of each state of the phoneme HMM whose central phoneme is a vowel is larger than the self-transition probability of each state of the phoneme HMM whose central phoneme is a consonant. Self-transition probability higher state j, when the calculation of the state likelihood b _j for a certain frame t _{(X t)} is performed, the calculation of the state likelihood b _j for the next frame t + 1 _{(X t)} It can be considered that there is a high possibility of being performed.

Using this property, a large frame number K is given to a state with a high self-transition probability, and conversely a small frame number K is given to a state with a low self-transition probability. That is, the higher the number of frames K, the higher the self-transition probability.
That is, in the second embodiment, the number K of frames is determined in consideration of both the self-transition probability a _jj and the likelihood calculation rate q _j .

As described above, by considering both the self-transition probability a _jj and the likelihood calculation rate q _j , the state likelihood is calculated by appropriately changing the number of frames K for which the state likelihood is calculated for each state. The amount of calculation processing can be further reduced. Therefore, the calculation of the acoustic likelihood can be further speeded up, and the speech recognition process can be further speeded up.

Hereinafter, although the example of 2nd embodiment is demonstrated with reference to FIG. 1, FIG. 4, only a different part from 1st embodiment is demonstrated, and duplication description is abbreviate | omitted about the part similar to 1st embodiment. FIG. 4 is a flowchart illustrating the process flow of the speech recognition apparatus according to the second embodiment.
In addition to the likelihood calculation rate calculation unit 92 and the likelihood calculation rate frame number determination unit 93, the frame number determination unit 90 of the speech recognition apparatus according to the second embodiment determines the number of self-transition probability frames indicated by a broken line in FIG. A unit 91 and an integrated frame number determination unit 94 are included, for example.

<Step S32 '>
Likelihood calculation rate frame number determination unit 93 determines an integer K _B (j) that is larger as likelihood calculation rate q _j is higher as in the first embodiment. K _B (j) is sent to the integrated frame number determination unit 94. Unlike the first embodiment, K _B (j) is not sent as it is to the state likelihood calculation unit 31 as K. That is, in the second embodiment, K = K _B (j) is not uniformly set, and K is determined by the process of step S7 described later.

<Step S6>
The self-transition probability frame number determination unit 91 of the frame number determination unit 90 uses the self-transition probability a _jj of the state j read from the acoustic model storage unit 40, and the higher the self-transition probability a _jj is, the larger the integer K _A (j) is determined. Information about K _A (j) is sent to the integrated frame number determination unit 94.

For example, _{a l} a 0 to 1 inclusive of a few, _{a h} a _{a l} 1 inclusive _number, the _{K min} 0 or an _{integer, K max} and _{K min} +1 or more integer, f (·) below the decimal point K _A (j) can be obtained by the following equation as a function that outputs an integer by rounding down. a _l , a _h , K _min, and K _max are numbers that are appropriately determined in accordance with the target speech, hardware performance, target speech recognition processing speed, and the like. For example, a ₁ is set to 0.2 to 0.3, a _h to 0.7 to 0.8, K _min to 3 to 4, and K _max to 10 to 12.

That is, if the self-transition probability a _jj is lower than a _1, K _A (j) = K _min , and if the self-transition probability a _jj is greater than or equal to a _{1 and} lower than a _h, K _A (j) = f (( _{K max -K min) a jj /} (a h -a l)) + ((K min a h -K max a l) / (a h -a l))) and then, self-transition probability _{a jj} is _{a h} If it is above, K _A = K _max .

<Step S7>
The integrated frame number determination unit 94 determines the number of frames K in consideration of both K _A (j) and K _B (j). The determined number K of frames is sent to the state likelihood calculating unit 31. For example, let f (•) be a function that outputs an integer by rounding down the decimal point of •, and the weighting coefficient λ is a predetermined number between 0 and 1, and the following K _A (j) and K _B (j ) K may be obtained based on the linear interpolation formula.
K = f ((1-λ) K _A (j) −λK _B (j))
λ is a weighting coefficient that adjusts how much confidence is placed on K _B (j). When K _B (j) is considered to be reliable due to the large amount of development data available, a value close to 1 is given to the weighting factor λ, and in the opposite case, the weighting factor λ Gives a value close to zero.

[Modifications, etc.]
In the above example, f (•) is a function that outputs an integer by rounding down the decimal point of •, but f (•) is a function that outputs an integer by rounding down the decimal point of •, or It is good also as a function which rounds off the decimal point of and outputs an integer.

In the above formula (5), when a _jj = a ₁ , K _A (j) = f ((K _max −K _min ) a _jj / (a _h −a ₁ )) + ((K _min a _h − K _max a _l ) / (a _h −a _l ))), but when a _jj = a _l , K _A (j) = K _min may be used. _Further, a jj = when _{a h,} K _{A =} is set to _{K max,} when _{a jj = a h, K A} (j) = f ((K max -K min) a jj / (a h _{-a l)) + ((K} min a h -K max a l) / (a h -a l))) may be.

Similarly, in the above formula (6), when q _j = q _l , K _B (j) = f ((K _max −K _min ) q _j / (q _h −q _l )) + ((K _min q _h −K _max q _l ) / (q _h −q _l ))), but when q _j = q _l , K _B (j) = K _min may be used. Further, when q _j = q _h , K _B = K _max , but when q _j = q _h , K _B (j) = f ((K _max −K _min ) q _j / (q _{h -q l)) + ((K} min q h -K max q l) / (q h -q l))) may be.

_K min in self-transition probabilities frame number determining portion _91, and _{K max, K} min at likelihood calculating rate frame number determination unit _93, also the same as the _{K max,} may be different.
When the above configuration is realized by a computer, the processing contents of the functions of each unit of the speech recognition apparatus are described by a program. Then, by executing this program on the computer illustrated in FIG. 5, the functions of the above-described units are realized on the computer.

  That is, when the CPU 1 sequentially reads and executes the program, the acoustic analysis unit 10, the feature vector storage unit 20, the search unit 30, the state likelihood calculation unit 31, the state likelihood reference unit 32, the fetch unit 60, the number of frames The functions of the determination unit 90, the self-transition probability frame number determination unit 91, the likelihood calculation rate calculation unit 92, the likelihood calculation rate frame number determination unit 93, and the integrated frame number determination unit 94 are realized. In this case, the CPU 1 functioning as each unit of the speech recognition apparatus performs processing on the data read from the memory 2 and the auxiliary storage device 3 such as a hard disk, and the data after the processing is stored in the memory 2 and the auxiliary storage. Store in device 3.

  In the example illustrated in FIG. 5, the auxiliary storage device 3 corresponds to the acoustic model storage unit 40, the grammar storage unit 50, and the state likelihood storage unit 80. The cache 1 a corresponds to the state parameter storage unit 70.

The program describing the processing contents can be recorded on a computer-readable recording medium. The computer-readable recording medium may be any medium such as a magnetic recording device, an optical disk, a magneto-optical recording medium, or a semiconductor memory. Specifically, for example, the magnetic recording device may be a hard disk device or a flexible Discs, magnetic tapes, etc. as optical discs, DVD (Digital Versatile Disc), DVD-RAM (Random Access Memory), CD-ROM (Compact Disc Read Only Memory), CD
-R (Recordable) / RW (ReWritable), etc., MO (Magneto-Optical disc), etc. as a magneto-optical recording medium, EEP-ROM (Electronically Erasable and Programmable-Read Only Memory), etc. as a semiconductor memory it can.

  The program is distributed by selling, transferring, or lending a portable recording medium such as a DVD or CD-ROM in which the program is recorded. Furthermore, the program may be distributed by storing the program in a storage device of the server computer and transferring the program from the server computer to another computer via a network.

  As an execution form different from the above-described embodiment, the computer may read the program directly from the portable recording medium and execute processing according to the program. Each time is transferred, the processing according to the received program may be executed sequentially. Also, the program is not transferred from the server computer to the computer, and the above-described processing is executed by a so-called ASP (Application Service Provider) type service that realizes the processing function only by the execution instruction and result acquisition. It is good. Note that the program in this embodiment includes information that is used for processing by an electronic computer and that conforms to the program (data that is not a direct command to a computer but has a property that is based on computer processing).

  In this embodiment, the present apparatus is configured by executing a predetermined program on a computer. However, at least a part of these processing contents may be realized by hardware.

  In addition, the various processes described above are not only executed in time series according to the description, but may be executed in parallel or individually according to the processing capability of the apparatus that executes the processes or as necessary. For example, in FIG. 2, the process of step S2 and the processes of steps S31 and 32 may be performed in parallel. In FIG. 4, the process of step S2 and the processes of steps S31 and 32 'may be performed in parallel. Further, in FIG. 4, the processes in steps S31 and 32 'and the process in step S6 may be performed in parallel.

  Needless to say, other modifications are possible without departing from the spirit of the present invention.

The functional block diagram of the example of the speech recognition apparatus of this invention. The flowchart which illustrates the flow of a process of the speech recognition apparatus of 1st embodiment of this invention. The figure for demonstrating the example of the method of determining the number K of frames. The flowchart which illustrates the flow of a process of the speech recognition apparatus of 2nd embodiment of this invention. The example of a functional block diagram in the case of implement | achieving the speech recognition apparatus of this invention with a computer. The example of the state likelihood table of this invention. The functional block diagram of the example of the speech recognition apparatus of a prior art. The figure for demonstrating the example of the state S. The figure for demonstrating the example of phoneme HMM. The example of a state likelihood table of a prior art.

Explanation of symbols

10 acoustic analysis unit 20 feature vector storage unit 30 search unit 31 state likelihood calculation unit 32 state likelihood reference unit 40 acoustic model storage unit 50 grammar storage unit 60 fetch unit 70 state parameter storage unit 80 state likelihood storage unit 90 frame Number determination unit 91 Self transition probability frame number determination unit 92 Likelihood calculation rate calculation unit 93 Likelihood calculation rate frame number determination unit 94 Integrated frame number determination unit

Claims (10)

An acoustic model storage unit for storing an acoustic model including a state parameter and a self-transition probability;
A state parameter storage unit faster than the acoustic model storage unit;
An acoustic analysis unit that obtains a feature vector for each frame of a certain time length from input speech and obtains a time series of the feature vector;
A feature vector storage unit for storing a time series of the obtained feature vectors;
State likelihood b _j (X _t ) is calculated by using j and t as arbitrary integers, and the state likelihood b _j (X _t ) as the probability that a certain state j outputs the feature vector X _t of frame t. Before, a fetch unit that reads the state parameter of state j from the acoustic model storage unit into the state parameter storage unit,
In speech recognition processing performed on speech having acoustic properties close to the target speech to be processed, the ratio of frames in which state likelihood calculations have been performed in all frames (hereinafter, likelihood calculation rate q _j And a likelihood calculation rate calculation unit for obtaining for each state;
A likelihood calculation rate frame number determination unit that determines a larger integer K _B (j) as the frame number K in a state where the obtained likelihood calculation rate q _j is higher;
The state likelihood b _j (X _t ) is calculated using the state parameter of the state j read from the state parameter storage unit and the feature amount vector X _t read from the feature amount vector storage unit, and the state parameter State likelihood b _j (X _{t + 1} ),..., B _j using the state parameter of state j read from the storage unit and the feature amount vectors X _{t + 1} ,..., X _{t + K} read from the feature vector storage unit. A state likelihood calculator for further calculating (X _{t + K} );
A state likelihood storage unit for storing the further calculated state likelihood b _j (X _{t + 1} ),..., B _j (X _{t + K} );
When any of the state likelihoods b _j (X _{t + 1} ),..., B _j (X _{t + K} ) becomes necessary, the state likelihood reference is obtained by referring to the state likelihood storage unit. And
A speech recognition apparatus. The speech recognition apparatus according to claim 1,
q _l is a predetermined number between 0 and 1; q _h is a predetermined number between q _{l and} 1; K _min is a predetermined integer greater than 0; and K _max is a predetermined number greater than K _min +1. As a fixed integer,
The likelihood calculation rate frame number determination unit,
If the likelihood calculation rate q _j is lower than q _{l, set} K _B (j) = K _min ,
If the likelihood calculation rate q _j is higher than q _{l and} lower than q _h, K _B (j) = f ((K _max −K _min ) q _jj / (q _h −q _l )) + ((K _min a _h− K _max q _l ) / (q _h −q _l )))
If the likelihood calculation rate q _j is above q _{h, set} K _B = K _max ,
If the likelihood calculation rate q _j = q ₁ , then K _B (j) = K _min or K _B (j) = f ((K _max −K _min ) q _j / (q _h −q _l )) + (( _{K min q h -K max q l} ) / (q h -q l))) and then,
If self-transition probability q _j = q _h , then K _B (j) = K _max or K _B (j) = f ((K _max −K _min ) q _j / (q _h −q _l )) + ((K _{min q h -K max q l)} / (q h -q l)))
Is the part,
A speech recognition apparatus characterized by that. The speech recognition apparatus according to claim 1 or 2,
_A self-transition probability frame number determination unit that determines a larger integer K _A (j) as the self-transition probability a _jj of the state j read from the acoustic model storage unit is higher;
A function that outputs an integer by rounding down, rounding up, or rounding off f (·) is rounded off, and weighting factor λ is a predetermined number between 0 and 1, and K = f ((1− (λ) K _A (j) −λK _B (j))
A speech recognition apparatus further comprising: The speech recognition apparatus according to claim 3,
a _l predetermined 0 to 1. number and _{a h} predetermined _{a l} 1 inclusive _number, 0 or an integer which is predetermined the _{K min, K max} and _{K min} +1 more pre As a function to output an integer by rounding off, rounding up, or rounding off the specified integer, f (•)
The self-transition probability frame number determination unit
If the self-transition probability a _jj is below a ₁ , set K _A (j) = K _min ,
If the self-transition probability a _jj is higher than a _{1 and} lower than a _h , K _A (j) = f ((K _max −K _min ) a _jj / (a _h −a ₁ )) + ((K _min a _h− K _max a _l ) / (a _h −a _l )))
If the self-transition probability a _jj is above a _h , then K _A = K _max
If self-transition probability _{a jj = a l, K A} (j) = K min or _{K A (j) = f (} (K max -K min) a jj / (a h -a l)) + (( _{K min a h -K max a l} ) / (a h -a l))) and then,
If self-transition probability a _jj = a _h , then K _A (j) = K _max or K _A (j) = f ((K _max −K _min ) a _jj / (a _h −a _l )) + (( _{K min a h -K max a l} ) / (a h -a l)))
Is the part,
A speech recognition apparatus characterized by that. The acoustic model storage unit is a storage unit that stores an acoustic model including state parameters and self-transition probabilities,
The state parameter storage unit is a storage unit faster than the acoustic model storage unit,
An acoustic analysis step in which an acoustic analysis unit obtains a feature vector for each frame of a fixed time length from the input speech, and stores a time series of the feature vector in the feature vector storage unit;
The state likelihood b _j (X _t ) is a state likelihood b _j (X _t ), where the fetch unit uses j and t as arbitrary integers, and a certain state j outputs the feature quantity vector X _t of the frame t. Fetching the state parameter of state j from the acoustic model storage unit into the state parameter storage unit before calculating
In the speech recognition processing performed by the likelihood calculation rate calculation unit on the speech having an acoustic property close to the target speech to be processed, the ratio of frames in which the state likelihood is calculated in all frames ( Hereinafter, the likelihood calculation rate q _j ) is calculated for each state.
A likelihood calculation rate frame number determination unit in which the likelihood calculation rate frame number determination unit determines a larger integer K _B (j) as the frame number K as the obtained likelihood calculation rate q _j is higher.
The state likelihood calculation unit calculates the state likelihood b _j (X _t ) using the state parameter of the state j read from the state parameter storage unit and the feature quantity vector X _t read from the feature quantity vector storage unit. The state likelihood b _j (X) is calculated using the state parameter of the state j read from the state parameter storage unit and the feature amount vectors X _{t + 1} ,..., X _{t + K} read from the feature amount vector storage unit. _{t + 1} ),..., b _j (X _{t + K} ) are further calculated, and the further calculated state likelihoods b _j (X _{t + 1} ),..., b _j (X _{t + K} ) are stored in the state likelihood storage unit. A state likelihood calculation step;
When the state likelihood reference unit needs any of the state likelihoods b _j (X _{t + 1} ),..., B _j (X _{t + K} ), the state likelihood storage unit refers to the state likelihood storage unit and determines the state likelihood. A state likelihood reference step for obtaining a degree;
A speech recognition method comprising: The speech recognition method according to claim 5,
q _l is a predetermined number between 0 and 1; q _h is a predetermined number between q _{l and} 1; K _min is a predetermined integer greater than 0; and K _max is a predetermined number greater than K _min +1. As a fixed integer,
The likelihood calculation rate frame number determining step includes:
If the likelihood calculation rate q _j is lower than q _{l, set} K _B (j) = K _min ,
If the likelihood calculation rate q _j is higher than q _{l and} lower than q _h, K _B (j) = f ((K _max −K _min ) q _jj / (q _h −q _l )) + ((K _min a _h− K _max q _l ) / (q _h −q _l )))
If the likelihood calculation rate q _j is above q _{h, set} K _B = K _max ,
If the likelihood calculation rate q _j = q ₁ , then K _B (j) = K _min or K _B (j) = f ((K _max −K _min ) q _j / (q _h −q _l )) + (( _{K min q h -K max q l} ) / (q h -q l))) and then,
If self-transition probability q _j = q _h , then K _B (j) = K _max or K _B (j) = f ((K _max −K _min ) q _j / (q _h −q _l )) + ((K _{min q h -K max q l)} / (q h -q l)))
Is the step
A speech recognition method characterized by the above. The speech recognition method according to claim 5 or 6,
A self-transition probability frame number determination unit that determines a larger integer K _A (j) as the self-transition probability a _jj of the state j read from the acoustic model storage unit is higher;
A speech recognition method, further comprising: The speech recognition method according to claim 7,
a _l predetermined 0 to 1. number and _{a h} predetermined _{a l} 1 inclusive _number, 0 or an integer which is predetermined the _{K min, K max} and _{K min} +1 more pre As a function to output an integer by rounding off, rounding up, or rounding off the specified integer, f (•)
The self-transition probability frame number determination step includes:
If the self-transition probability a _jj is below a ₁ , set K _A (j) = K _min ,
If the self-transition probability a _jj is higher than a _{1 and} lower than a _h , K _A (j) = f ((K _max −K _min ) a _jj / (a _h −a ₁ )) + ((K _min a _h− K _max a _l ) / (a _h −a _l )))
If the self-transition probability a _jj is above a _h , then K _A = K _max
If self-transition probability _{a jj = a l, K A} (j) = K min or _{K A (j) = f (} (K max -K min) a jj / (a h -a l)) + (( _{K min a h -K max a l} ) / (a h -a l))) and then,
If self-transition probability a _jj = a _h , then K _A (j) = K _max or K _A (j) = f ((K _max −K _min ) a _jj / (a _h −a _l )) + (( _{K min a h -K max a l} ) / (a h -a l)))
Is the step
A speech recognition method characterized by the above.   A speech recognition program for causing a computer to function as each part of the speech recognition apparatus according to claim 1.   A computer-readable recording medium on which the voice recognition program according to claim 9 is recorded.

Images