JPH08123472A - Speech recognition device and method for generating syntax control graph for the device - Google Patents

Abstract

PURPOSE: To prevent a phoneme string which is not prescribed in the syntax control graph from being recognized and to improve the estimation precision of a phoneme boundary by providing a boundary likelihood calculating means which calculates the boundary likelihood of a phoneme from an input speech and a model arithmetic means which selects an optimum phoneme model. CONSTITUTION: This device is equipped with the boundary likelihood calculating means 3 which calculates the boundary likelihood of the phoneme from the input speech and the model arithmetic means which selects the optimum phoneme model. In this case, an HMM arithmetic means 5 based upon viterbi algorithm wherein the calculation of the sum of HMM arithmetic based upon normal trellis algorithm is replaced with calculation of maximization is used as the model arithmetic means. Then the boundary likelihood calculating means 3 calculates the boundary likelihood of the phoneme from the input speech. Further, the model arithmetic means (HMM arithmetic means) 5 selects the optimum phoneme model only when the boundary likelihood of the phoneme calculated by the boundary likelihood calculating means 3 is larger than a specific value and the selected phoneme model meets restrictions as to >=3 phoneme strings.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a speech recognition apparatus for recognizing continuous speech and converting it into a phoneme sequence.

[0002]

2. Description of the Related Art A continuous speech is regarded as a concatenation of phonemes, an input speech is analyzed according to a phonological model series which is a concatenation of phonological models, and an optimum phonological model series applicable to the input speech is obtained by a model calculation means. In the speech recognition device for converting the phoneme sequence into the phoneme sequence of the optimal phoneme model sequence obtained in this way, apart from the calculation of the phoneme model sequence fitting, the boundary of the phoneme is directly detected in the input speech and the phoneme model sequence fitting is performed. At times, a speech recognition apparatus that improves recognition accuracy by limiting transitions between phonological models to the vicinity of a detected phonological boundary is published by the Acoustical Society of Japan, Proc. Transition bound type H
Phonological description by MM ". Furthermore, Proceedings of the Acoustical Society of Japan 2-P-11 published in March 1994
"Inter-state transition bound HMM considering reliability of boundary likelihood
"Phonological description by" describes a method of using a common codebook as a method of detecting a phonological boundary.

FIG. 10 shows a state-boundary transition-bound HM of this type.
It is a block diagram of the speech recognition apparatus by M (HMM / BT). Each part of FIG. 10 will be described below. Voice section detection means 1
Detects a voice section from the change shape of the short-term power of the input voice, cuts out the voice signal R1 in this voice section, and sends it to the feature extraction means 2. The feature extracting means 2 measures 10 m from the voice signal R1 in the voice section by 15th-order linear predictive mel cepstrum analysis using a time window having a length of 25.6 ms.
A feature parameter time series R2 consisting of mel-cepstral coefficients of 0th to 10th order is extracted for each s, and the boundary likelihood calculating means 3,
And sends it to the HMM calculating means 5a as the model calculating means.

The boundary likelihood calculating means 3 is a phonological boundary parameter R4 stored in the phonological boundary parameter storing means 4.
From the characteristic parameter time series R2, the time t =
For 1, 2, ..., T, a total of 0 to 7th-order mel cepstrum coefficients within a range of 10 frames with a time t as a center, and a total of 8
0 (= 10 frames x 8 dimensions) are extracted as one 80-dimensional vector (hereinafter referred to as a fixed length segment),
The likelihood (boundary likelihood) that a phoneme boundary in the input speech exists at the center of these fixed length segments is calculated. A phoneme i is placed at the center of a fixed-length segment (hereinafter referred to as Bt) at the central time t.
Boundary likelihood that there exists a phonological boundary ij between ij and j (hereinafter, Cij
(Denoted as (Bt)) is calculated according to equation (1). Here, the denominator of equation (1) is the likelihood when the boundary of the phoneme ij does not exist in the center of the fixed length segment Bt, and the numerator assumes that the boundary of the phoneme ij exists in the center of the fixed length segment Bt. Corresponding to the likelihood of, the expression (1) as a whole represents the boundary likelihood of the phoneme ij at the time t. Where Mb is the number of common element distributions (size of common codebook), N (Bt | μ
m, Σm) is a multidimensional normal probability density function of mean μm and variance Σm of the m-th element distribution, Pmij and Qmij are phonological boundaries ij
Is a polynomial coefficient obtained by learning in advance.

[0005]

[Equation 1]

Next, the HMM calculation means 5a will be described. FIG. 11 schematically shows the structure of the phoneme sequence HMM to be calculated by the HMM calculation means 5a. Book H
The MM has exactly the same number of states as the number of phonemes (let's say n), and consists of n states (n = number of phonemes), and each state is associated with one phoneme. The transition probability from state i to state j is
The output probability in the state j of the characteristic parameter xt at the time t is indicated by bj (xt). The output probability bj (xt) is represented by a Gaussian mixture distribution of M common element distributions common to all phonemes, and the m-th average vector μm
And the probability density function N of the element Gaussian distribution of the covariance matrix Σm
(xt | μm, Σm) and the branch probability λmj of the phoneme j are calculated by the equation (2).

[0007]

[Equation 2]

The HMM calculating means 5a is the boundary likelihood calculating means 3
The boundary likelihood R3 of the output and the HMM parameter R6 stored in the HMM parameter storage means 6 are referred to, and equations (3) and (4), which are recurrence equations based on the Viterbi algorithm, are given as equations (5) giving initial conditions. Calculate under. Where α
(j, t) is the probability of staying in state j (forward probability) at time t, and β (j, t) is a back pointer that represents the most suitable state number before reaching state j at time t. is there.

[0009]

(Equation 3)

As indicated by the above recurrence formula, the HMM computing means 5a performs the boundary likelihood Cij (of the phonological unit) at the time t during the state transition from the state i to the state j corresponding to the transition between the phoneme models. Bt) is compared with a threshold value θij depending on the phoneme boundary ij, and transition between states is allowed only when the boundary likelihood of the phoneme is larger than the threshold value θij (Cij (Bt)> θij). Therefore, the transition between states occurs only at the phoneme boundary estimated in the input speech, and the insertion error can be reduced. For the transition of states within the same phoneme (when i = j), the boundary likelihood Cij
No limit is set by (Bt).

The optimum state series detecting means 7a as the phoneme sequence converting means calculates the optimum state series R7 from the values of the forward probability α (j, t) and the back pointer β (j, t) obtained as the HMM calculation result R5. (Hereinafter referred to as β ^ (1), β ^ (2), ..., β ^ (T)) is output. The optimum state series R7 is obtained by calculating a recurrence formula (formula (6)) under initial conditions (formula (7)). The optimum state sequence R7 is a sequence of state numbers indicating the phoneme sequence of the recognition result.

[0012]

[Equation 4]

In the conventional speech recognition apparatus using the state-to-state transition bound HMM (HMM / BT) having the above-described configuration, the transition between states corresponding to the transition between phonemes is statistically estimated from the input speech. It is limited to the vicinity, and the state transitions other than the phoneme boundaries are suppressed by the phoneme boundary information obtained directly from the input speech, and as a result, the insertion error is suppressed. Therefore, although relatively high recognition accuracy can be obtained, some recognition errors still occur. When these recognition errors are analyzed, the recognition result shows that a phoneme sequence that is linguistically impossible as voice data, for example, [k, ts, sh, ts, sh].
(Hereinafter, the phoneme sequence is represented by enclosing the phonemes with [] in this way)
You can see that is included. Therefore, if such a phoneme string can be suppressed by linguistic knowledge, it is possible to further improve the recognition performance.

[0014]

In the conventional HMM / BT-based speech recognition apparatus, it is easy to make such a restriction on a sequence of two phonemes. That is, for example, if the phoneme sequence [p, q] resulting from the transition from the state i of the model of a certain phoneme p to the state j of the model of another phoneme q is linguistically impossible as speech data, Boundary likelihood Cij (B
By setting the threshold value θij for t) to ∞, such transition can be easily prohibited. However, in this method, the first utterance such as "Kusu", "Kushi", "Shitsu", etc.
Data [k, ts], [k, sh], [sh,
Since it is necessary to recognize phoneme sequences such as [ts], [k, ts], [t
Since it is not possible to prohibit transitions between states corresponding to phoneme strings such as s, sh] and [sh, ts], the above phoneme string, for example, [k, ts, sh, ts, sh] is the recognition result. Can occur as As described above, the conventional HMM / BT-based speech recognition device has a problem in that it is not possible to prohibit recognition of a phoneme string that is linguistically impossible as voice data for a phoneme string of length 3 or more. there were.

Conventionally, a speech recognition method using a statistical language model or an automaton control for a sequence of language symbols such as phonemes and syllables as a language model, and an acoustic model of a recognition unit such as phonemes and syllables as an acoustic model. There is. As this kind of speech recognition method, a statistical language model regarding the phoneme occurrence sequence as a language symbol is used, and a phoneme HMM is used as an acoustic model, and a kana as a language symbol and statistics concerning the kanji occurrence sequence are used. Method using a dynamic language model, using an HMM of a syllable corresponding to a kana as an acoustic model and an HMM corresponding to reading of kanji (for example, Japanese Patent Laid-Open No. 4-73694), and further, a syllable corresponding to Japanese as a language model. A method of using a phoneme HMM as an acoustic model for a directed graph that defines the order of occurrence of phonemes as linguistic symbols in a syllable automaton designed to allow the sequence of
Proceedings of the Acoustical Society of Japan 2-P-26, "phoneme recognition by HMM using syllable automata and speaker adaptation", published in March, 2002) is proposed. In addition, a statistical language model of syllable chain as a language symbol and a phoneme HM as an acoustic model.
A method using M (for example, Proceedings of the Acoustical Society of Japan 3-3-9 “Use of syllable chain statistical information in HMM phoneme recognition” published in March 1990) has been proposed. In particular, the method using the language model of syllable chain has less task dependency and can be expected to be strongly restricted.

By the way, in the above technique, the phoneme HM
M, syllable HMM, or HM for reading kanji
M is used as the acoustic model and these HMMs
The language model of the sequence of language symbols corresponding to is used. Further, as the HMM as an acoustic model, the one in which the number of states and the structure of transition between states are determined in advance is used.
On the other hand, in continuous speech, when syllables as linguistic symbols are realized as a time-series structure of acoustic phonetic phonological features,
The spectrum of each phonological feature section varies depending on the speech environment and individual differences, and due to the lack of phonological feature sections such as devoicing of vowels and drop of buzz bars, the phonological structure of the syllables formed by the phonological feature series in the syllable itself. It is known to fluctuate (for example, S8, the audio study group material published in 1984)
4-69 “Study of syllable variation for continuous syllable recognition”). Therefore, hereinafter, if the above phonological feature section is called a phonological section and its label is called a phonological symbol, an acoustic model of a syllable or a phoneme is represented by a predetermined state transition structure as in the conventional example. Also, it is considered appropriate to represent the phoneme network structure corresponding to the voice environment and individual differences.

For example, when observing in a speech database what kind of phoneme sequence a syllable "tsu" (/ cu / in phoneme notation) in continuous speech is realized, a vowel dropout or a consonant closed section dropout occurs. (Or fusion with the preceding phoneme), etc., as a phoneme sequence [ts], [ts, u], [c
It can be seen that it is realized as l, ts], [cl, ts, u], and so on. However, in the method using the syllable HMM, an HMM having a graph structure with a predetermined number of states is learned even for data having such a structural variation. Therefore, for example, syllable "tsu" (/ cu / as a phoneme string)
Is a phoneme sequence in the voice database, [ts], [ts,
u], [cl, ts], and [cl, ts, u], the syllable HMM having a graph structure with a predetermined number of states is learned for this. As a result, there is a problem that the accuracy of the acoustic model (syllabic HMM) deteriorates for unknown speech in which the phonological structure in the syllable is changed due to dropout of the phoneme as well as the fluctuation of the spectrum. This is the phoneme H
The same applies when using MM. For example, although the phoneme / c / representing the consonant part of "tsu" is realized as [ts], [cl, ts], etc. as a phoneme sequence in the speech database, the number of states is Phoneme H with the graph structure
In order to learn MM, not only the fluctuation of spectrum but also
The accuracy of the acoustic model (phoneme HMM) is reduced for unknown speech that has undergone deformation of the phoneme structure in the phoneme due to dropping of the phoneme. That is, in the conventional speech recognition method using the phoneme or syllable HMM and its language model, since the phoneme or syllable HMM having a predetermined state transition structure is used as the acoustic model in advance, the phoneme or the phonological spectrum inside the syllable is analyzed. There is a problem that the accuracy of the model deteriorates for unknown speech in which the phoneme structure inside the syllable or the phoneme changes due to the drop of the phoneme along with the change.

The present invention has been made to solve the above problems, and a first object thereof is to apply a phonological model sequence to an input voice and convert the input voice into an optimum phonological sequence. In the speech recognition device that binds the transition time between models to the vicinity of the phoneme boundary estimated from the input speech,
Introducing restrictions on the phoneme occurrence sequence for phoneme strings of length 3 or more to prevent recognition of phoneme strings that are impossible as speech data, and to limit the phoneme boundaries assumed for input speech. It is an object of the present invention to provide a speech recognition device having limited types and improved estimation accuracy of phonological boundaries and improved recognition accuracy. A second object of the present invention is to generate a syntactic control graph of a speech recognition apparatus that applies a phonological model sequence to an input speech according to a syntactic control graph and converts the input speech into an optimal phonological sequence. It is intended to provide a method for generating a syntactic control graph that models both a variation in the spectrum of a phonological segment and a variation in the phonological structure in a syllable or a phoneme due to dropout of the phonological segment.

[0019]

A speech recognition apparatus according to claim 1 of the present invention comprises a boundary likelihood calculating means for calculating a boundary likelihood of a phoneme from an input speech, and a boundary likelihood of the phoneme from a predetermined value. The model calculation means selects an optimum phoneme model only when the constraint is large and the phoneme sequence having a length of 3 or more is satisfied.

In the speech recognition apparatus according to the second aspect of the present invention, the boundary likelihood calculating means for calculating the boundary likelihood of the phoneme according to the kind of the phoneme boundary from the input speech, and the boundary likelihood of the phoneme are the phonemes. The optimum phoneme model is larger than the value set according to the type of boundary, and only when the phoneme occurrence order defined in the syntax control graph that defines the occurrence order of phonemes with a length of 3 or more in the phoneme sequence is followed. And model calculation means for selecting.

According to a third aspect of the present invention, a method for generating a syntax control graph of a speech recognition apparatus comprises a step of obtaining a syllabic syntax graph defining a syllable occurrence order from a text database,
The method includes a step of obtaining an intra-syllable phoneme graph that defines the order of occurrence of phonemes in a syllable from a voice database, and a step of substituting the in-syllable phoneme graph into a syllable equivalent part of the syllable syntax graph.

According to a fourth aspect of the present invention, a method of generating a syntax control graph for a speech recognition apparatus analyzes an input voice, regards the input voice as a concatenation of phoneme models, and determines the input voice according to the definition of the syntax control graph. As a method of generating the syntactic control graph of a speech recognition device that applies a sequence of phonological models and converts the input speech into an optimal phonological sequence, syllables and syllable contexts surrounding the syllables are extracted as syllable data from a text database. A step of obtaining a syllable syntactic graph that defines the occurrence order of syllable data; a step of obtaining an intra-syllable phoneme graph that defines the occurrence order of phonemes in a syllable for each syllable context of the syllable data from a speech database; and the syllable syntax graph. Of the syllable data corresponding to the syllable context of the syllable data corresponding to the syllable context of the syllable data. Comprising the step of assigning a.

The model operation means in the speech recognition apparatus according to claim 5 of the present invention uses a syntax control graph generated based on the syntax control graph generation method according to claim 3 or 4.

[0024]

In the speech recognition apparatus according to the first aspect of the present invention, the boundary likelihood calculating means calculates the boundary likelihood of the phoneme from the input speech. Further, the model calculation means is optimal only when the boundary likelihood of the phoneme calculated by the boundary likelihood calculation means is larger than a predetermined value and the selected phoneme model satisfies the constraint for the phoneme sequence having a length of 3 or more. Select a phoneme model.

In the speech recognition apparatus according to the second aspect of the present invention, the boundary likelihood calculating means calculates the boundary likelihood of the phoneme corresponding to the type of the phoneme boundary from the input speech. Further, the model calculation means is larger than the boundary likelihood of the phoneme according to the kind of the phoneme boundary calculated by the boundary likelihood calculation means and the value set according to the kind of the phoneme boundary, and the phoneme model to be selected is long. 3
The optimum phoneme model is selected only when the above restrictions on the phoneme sequence are satisfied.

In the method of generating a syntactic control graph of a speech recognition apparatus according to claim 3 of the present invention, the step of obtaining a syllable syntactic graph obtains a syllable syntactic graph corresponding to an syllable automaton that defines the syllable occurrence order from a text database. . In the process of obtaining the syllable in-syllable graph, the syllable sequence in the syllable section in the speech database is extracted, and the in-syllable phonological graph corresponding to the phoneme automaton that defines the occurrence order of the syllable in the syllable is obtained. Furthermore, in the last step, the intra-syllable phoneme graph is substituted into the syllable equivalent part of the syllable syntax graph.

In the method for generating a syntactic control graph of a speech recognition apparatus according to claim 4 of the present invention, in the step of obtaining a syllable syntactic graph, a syllable and a syllable context surrounding the syllable are extracted as syllable data from a text database. Find a syllable syntax graph corresponding to the automaton of. Further, in the process of obtaining a syllable in-syllable graph, a phonological string in a syllable section is extracted for each syllable context of the syllable data from the speech database and a phonological graph in the syllable for each syllable context is obtained. Furthermore, in the final step, the intra-syllable phoneme graph obtained from the syllable context that matches the syllable context of the syllable data is substituted into the syllable data equivalent portion of the syllable syntax graph.

In the speech recognition apparatus according to claim 5 of the present invention, the model calculation means uses a syntax control graph generated based on the syntax control graph generation method according to claim 3 or 4.

[0029]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below. In this embodiment, a phoneme HMM (phoneme HMM) that represents one phoneme in one state is used as an acoustic model, and a certain input speech is represented by a sequence of these phoneme HMMs. Also, the phoneme is
The consonant closed section is regarded as a phoneme different from the consonant burst part, as shown in FIG.
The system consisting of 29 kinds of phonemes shown in Fig. 2 is used, and each phoneme is numbered as shown in this figure. Hereinafter, the phoneme is referred to by this number. Further, in the present embodiment, since the phoneme HMM that represents one phoneme in one state is used as the acoustic model, the generation of the phoneme boundary appears as a transition between the states of the phoneme HMM. In addition, a phoneme H having a plurality of states in one phoneme
It goes without saying that the present invention is applicable even when using MM, in which case the phonological boundary generation is performed by the phonological HM.
Appears as a transition between states corresponding to a transition between M. Further, in this embodiment, as the model calculation means, the HMM calculation means based on the Viterbi algorithm in which the sum calculation in the HMM calculation based on the ordinary trellis algorithm is replaced with the maximization calculation is used. Needless to say, the present invention can be applied to the HMM calculation based on the ordinary trellis algorithm.

FIG. 1 is a block diagram of a voice recognition apparatus according to an embodiment of the present invention. Each part of FIG. 1 will be described below. The voice section detecting means 1 detects a voice section based on the change shape of the short section power of the input voice, and the voice signal R1 in this voice section.
Is cut out and sent to the feature extracting means 2. Feature extraction means 2
Is 25.6 ms in length from the voice signal R1 in the voice section.
The characteristic parameter time series R2 consisting of 0th to 10th mel-cepstral coefficients is extracted every 10 ms by the 15th-order linear predictive mel-cepstral analysis using the time window of B, the boundary likelihood calculating means 3 and the HMM as a model calculating means. It is sent to the calculation means 5.

The boundary likelihood calculating means 3 has a phonological boundary parameter R4 stored in the phonological boundary parameter storing means 4.
From the characteristic parameter time series R2, the time t =
For 1, 2, ..., T, a total of 0 to 7th-order mel cepstrum coefficients within a range of 10 frames with a time t as a center, and a total of 8
0 (= 10 frames x 8 dimensions) are extracted as one 80-dimensional vector (hereinafter referred to as a fixed length segment),
The likelihood (boundary likelihood) that a phoneme boundary in the input speech exists at the center of these fixed length segments is calculated. A phoneme i is placed at the center of a fixed-length segment (hereinafter referred to as Bt) at the central time t.
Boundary likelihood that there exists a phonological boundary ij between ij and j (hereinafter, Cij
(Denoted as (Bt)) is calculated according to equation (1). Here, the denominator of equation (1) is the likelihood when the boundary of the phoneme ij does not exist in the center of the fixed length segment Bt, and the numerator assumes that the boundary of the phoneme ij exists in the center of the fixed length segment Bt. Corresponding to the likelihood of, the expression (1) as a whole represents the boundary likelihood of the phoneme ij at the time t. Where Mb is the number of common element distributions (size of common codebook), N (Bt | μ
m, Σm) is a multidimensional normal probability density function of mean μm and variance Σm of the m-th element distribution, Pmij and Qmij are phonological boundaries ij
Is a polynomial coefficient obtained by learning in advance.

Next, the HMM calculation means 5 will be described. FIG. 3 schematically shows the structure of the HMM to be operated by the HMM operation means 5. The whole HMM is 1
Phoneme H as an acoustic model represented by one state per phoneme
MM (there are 29 types in total in this example) are connected by transitions between states according to a phoneme syntactic graph as a syntactic control graph. That is, the sequence of states obtained by transitioning the states in the HMM corresponds to a certain phoneme sequence that can be generated according to the phoneme syntactic graph. (The method of generating the phonological syntax graph will be described later.) In particular, this figure shows the state transition from the state p to the state q in order to explain the recurrence formula calculation of the HMM computing means 5.
Each state of the phonological syntax graph is associated with a certain phoneme, for example, in FIG. 3, the state q is associated with the phoneme j, and the characteristic parameter x at the time t in the state q.
The output probability of t is calculated as bj (xt) using the parameter of phoneme j. The transition from state p to state q is phonological i
Boundary likelihood Cij based on the phoneme boundary parameter of the phoneme j
It is possible when (Bt) is greater than the threshold θ ij and the transition between states from state p to state q is allowed in the syntax control graph (this is indicated by δpq = 1 in the recurrence equation). The transition probability from the state p to the state q is indicated by apq. The output probability bj (xt) is represented by a mixture distribution of M common Gaussian distributions in total, and the average vector μm and covariance matrix Σm of the mth common Gaussian distribution and the phoneme j
It is calculated by the equation (8) using the branch probability λmj of the parameter as a parameter. In the equation, N (xt | μm, Σm) represents a normal probability density function of mean μm and variance Σm. The parameters for calculating the output probability are stored in the HMM parameter storage means 6.

[0033]

(Equation 5)

The HMM calculation means 5 refers to the boundary likelihood R3 output from the boundary likelihood calculation means 3 and the HMM parameter R6 and the syntax control information R8 stored in the syntax control means 8 to perform the HMM calculation based on the Viterbi algorithm. The syntax control information R8 is composed of information about phoneme numbers corresponding to the respective states and information indicating connections between the states, as a result of converting the phoneme syntax graph as the syntax control graph. There are n states in total, and the phoneme number j corresponding to the state q is j as a function of q.
= F (q) Also, from state p to state q
The possibility of transition to is shown by δpq = 1. The HMM calculation in the HMM calculation means 5 is performed by the equation (9) and the equation (10).
Is calculated under the initial condition of equation (11). Here, n is the number of states of the phonological syntactic graph, and α (q, t) is the time t.
, The probability of staying in the state q (forward probability)
(q, t) is a back pointer that represents the optimum state number before the state q at time t.

[0035]

(Equation 6)

As indicated by the above recurrence formula, the HMM calculating means 5 makes a boundary likelihood Cij (of the phoneme) at the time t during the state transition from the state i to the state j corresponding to the transition between the phoneme models. Bt), the threshold θi depending on the phoneme boundary ij
Only when the boundary likelihood of the phoneme is larger than the threshold value θij (Cij (Bt)> θij) as compared with j and the transition is allowed in the phoneme syntax graph (δpq = 1). Since the transition between states is allowed, the transition between states occurs only at the phoneme boundary estimated in the input speech, and the state transition at the non-phoneme boundary is reduced. It reduces errors, prevents phoneme sequences that are not allowed as transitions in the phoneme syntactic graph from being selected as state sequences, and recognizes phoneme sequences that are linguistically impossible as speech data. . Regarding the transition of states within the same phoneme (when i = j), there is no restriction on the selection by the boundary likelihood Cij (Bt) and the phoneme syntax graph.

The optimum state series detecting means 7 as the phoneme sequence converting means calculates the optimum state series R7 from the values of the forward probability α (q, t) and the back pointer β (q, t) obtained as the HMM calculation result R5. (Hereafter written as β ^ (1), β ^ (2),…, β ^ (T))
Is output. The optimum state sequence R7 is a recurrence formula (1
2) is calculated under the setting of Expression (13) indicating the initial condition. The optimum state sequence R7 is a sequence of phonetic state numbers in the phoneme syntax graph representing the phoneme sequence of the recognition result, and the conversion from the optimum state sequence R7 to the phoneme sequence is a simple one-to-one function. Realized by relationships.

[0038]

(Equation 7)

Above, the description of the structure of the speech recognition apparatus has been completed, and the method for creating the phoneme syntax graph used in the speech recognition apparatus of this embodiment will be described below. FIG. 4 is an explanatory diagram of a phoneme syntax graph generation process in this embodiment. The process of generating a phonological syntactic graph is composed of processes I to III as a whole.

In step I, in the syllable chain extraction in the figure,
Extract a syllable string from a large amount of text database and create a directed graph (syllable syntax graph) with syllable branches that accept all the extracted syllable strings. This syllable syntactic graph is configured to accept a sequence of syllable trigrams (triads) in order to express language constraints strongly and to accept any sentence as much as possible without depending on a task. FIG. 5 exemplifies a process of generating a syllable syntactic graph that accepts a three-syllable chain from syllable text data. If the text data is "ε
# Saita Saita # Sakuraga Saita # ε ”, from this text data, as a triple depending on one syllable environment before and after, (ε) # (sa), (#) sa (a), (sa) i (ta) ,
(A) ta (sa), (ta) sa (a), (sa) ai (ta), (a) ta (#), (ta) #
(Sa), (#) Sa (ku), (sa) ku (la), (ku) la (ga), (la) ga (sa),
(Mo) sa (a), (sa) ai (ta), (a) ta (#), (ta) # (ε) can be extracted, and by removing the common triplet among them, the center of the figure We obtain a set of three syllables as shown in. By connecting these three sets under the condition that the speech environment (context before and after the syllable) matches, a graph like the one below is generated as a directed graph of syllables (syllable syntax graph). Here, “ε” and “#” indicate a blank character and a boundary between sentences or words, respectively. In the notation of syllable triples, the syllables in () on the left and right indicate the environment (context of syllables) of the central syllable.

In step II, first, a sequence of phoneme labels (phoneme sequence) of a section corresponding to a syllable (phoneme sequence) is extracted from a large amount of known speech data labeled in phonological units, and the correspondence relation between syllables and phonological sequences is determined. Ask. Next, a directed graph (in-syllable phoneme graph) having phonems as branches is created as an expression that comprehensively describes in what phonological sequence the inside of a syllable is realized from this correspondence. Here, by obtaining the correspondence relation between the syllable and the phoneme sequence for each syllable context, an intra-syllable phoneme graph depending on the environment of the syllable (syllable context) can be obtained. FIG. 6 shows a state in which the syllable-in-syllable graph is extracted from the description in the voice database for the sentence utterance "# tsukusiku # tsutsumu #", for example. First, the correspondence between each syllable section indicated by "syllable string" in the frame of "speech database" in the upper part of the figure and the partial phoneme string of "phoneme string" below it is found, and the partial phoneme corresponding to each syllable is obtained. Find the set of columns (middle row). Next, each syllable is converted into a directed graph (intra-syllable phoneme graph) having a phoneme as a branch by setting a common part in the syllable-in-syllable string set as a common branch (lower part of the figure). When the syllable context is not taken into consideration, the syllable in-syllable phoneme graph of the syllable "tsu" is extracted as a directed graph of four states and five branches as shown in the center of the lower part of the figure. When the syllable context of the preceding and following one syllable is taken into consideration, for example, the syllable “(U) tsu (ku)” is extracted as a three-state two-branch in-syllable phoneme graph as shown on the bottom left of the figure.

In step III, a phonological syntactic graph is obtained by substituting the in-syllable phoneme graphs obtained in step II for all syllable branches in the syllable syntactic graph obtained in step I. FIG. 7 shows a state in which an intra-syllable phoneme graph is substituted for a branch of a part of a syllable syntax graph in which the syllable context of one syllable before and after is considered. In this example, a phonological syntax graph in which the state s12 is newly inserted in the branch connecting the states s1 and s2 is generated. In this phonological syntactic graph, only [cl, ts] is allowed as a phonological sequence corresponding to the syllable "tsu" in the syllable context "(u) tsu (ku)". On the other hand, when the preceding and following syllable contexts are not considered, the syllable-intra-syllable graph of the syllable "tsu" shown in the center of the lower part of FIG. 6 is assigned to the branch corresponding to the syllable "tsu" in the syllable syntax graph to be generated. The phonological syntactic graph has [ts],
Phoneme strings such as [cl, ts] and [cl, ts, u] are allowed. As described above, when the syllable context graph is generated, considering the syllable context reduces the number of types of phoneme sequences to be recognized for the same syllable, and thus the effect of further improving the recognition performance can be expected. (This effect is shown in the experiment described later.)

By applying the phonological syntactic graphs obtained in the above steps I to III to the continuous speech recognition system based on the speech recognition apparatus having the above-mentioned configuration, when an unknown speech is input to this apparatus, the phonological sequence of the recognition result is obtained. The phonological sequences appearing in the phonological description are limited to those observed as phonological sequences inside the syllables in the speech database according to the syllable occurrence order in the text database. As a result, a phoneme string that cannot be linguistically included as a syllable string and that cannot be included as the voice data, for example, [k, ts, sh, ts, sh].
Can be prevented from being recognized. The speech recognition apparatus according to the present embodiment uses the one-pass phoneme HMM as an acoustic model.
It is similar in structure to a speech recognition device that uses the DP method (for example, “Speech recognition by probabilistic model” by Seiichi Nakagawa) for syntax control. However, in the speech recognition apparatus of the present embodiment, for grammatical control, the syllable syntactic graphs generated according to the occurrence order of syllables in the text database are observed in the speech database at the syllable equivalent part in this syllable syntactic graph. A phonological syntactic graph generated by substituting an intra-syllable phonological graph expressing a variation of a phoneme sequence in a syllable is used. Also, in the state transition between phoneme models as acoustic models, the transition time is bound based on the estimated value information (boundary likelihood) of the phoneme boundary obtained directly from the input speech. Therefore, the speech recognition apparatus of the present embodiment, by the action of the language knowledge of the syllable string and the knowledge of the variation of the phonological structure in the syllable,
The number of phoneme boundaries assumed for the input speech is reduced, and the resulting phoneme boundary estimation accuracy is improved. On the contrary,
Since the phonological boundaries of the input speech depend on the types of phonemes before and after, there is a feature that phonological sequences that consider the types of phonemes before and after the phonological boundaries are recognized in the generation of phonological sequences.
This has the effect of improving recognition accuracy.

Next, an evaluation experiment conducted on the voice recognition device of this embodiment will be described. Here, HMM / BT using a semi-continuous distribution model for phonemes and phoneme boundaries
We conducted a phoneme description experiment of an unspecified speaker using. The common experimental conditions are shown in FIG. Although general text data can be used as the language data, 4024 utterance texts (speech descriptions) composed of phonological balance sentences which are learning voice data are used here. Experiments were conducted for the case where the context was not considered as the syllable context in the generation of the syllable syntactic graph, the case where a syllable bigram (two sets) was used, and the case where a syllable trigram (three sets) was used. HMM / BT continuous speech recognition with phonologically-based syntactic control for each phonological graph is created by extracting phonological graphs in syllables and substituting them into syllable syntactic graphs with different dependences on syllable contexts. The phoneme recognition performance was obtained by the system. In addition, HMM / BT
Conventional HMM (1 phoneme 1 state) without phonological boundary constraint
Experiments were also performed when using.

FIG. 9 shows the experimental results. The figure shows the phoneme error rates under various experimental conditions for the HMM (HMM / BT) that constrains the phoneme boundaries and the HMM (HMM without BT) that does not constrain the phoneme boundaries. (In HMM / BT, the boundary likelihood threshold (θij)
Is shown as a constant value (θ) regardless of the type of phonological boundary). The phonological error rate shows the total error rate and the error rates of replacement, dropout, and insertion as a breakdown. The phoneme error rate is calculated as the ratio of the phoneme recognition errors of each error form to the number of phonemes of the input. The syllable context dependence considered in extracting the syllable-in-syllable graph is shown in the columns of the number of preceding syllables and the number of subsequent syllables. Further, as the syllable context surrounding the syllables of the syllable syntax graph when extracting the syllable syntax graph from the text data, as shown in the first column on the left side of the figure, a syllable bigram and a syllable trigram are represented. The experimental results are shown for the case. Still further, for reference, the test set perplexity of syllable and phonological syntactic graphs is shown. Generally, the larger the test set perplexity, the smaller the limitation by the syntax (the greater the degree of freedom of the syntax). From the experimental results, in the generation of syllable syntax graph,
From a context-independent syllable graph that is independent of the syllable context (that is, the number of preceding syllables and the number of subsequent syllables are both 0), regardless of whether a syllable bigram or a syllable trigram is used as the syllable context surrounding the syllable syntactic graph. Also, the phonological recognition error is less in the case of using the syllable context dependent phoneme graph, and the method of using the syllable context dependent phoneme graph has better recognition performance. This is because the syllable contextual syllable context-dependent graph narrows the range of variations in which syllables are realized as phonological sequences, so the number of phonological sequences assumed as recognition targets is substantially reduced, and recognition performance is improved. It is thought that it depends on what you did. This idea is
In fact, it is also explained by the fact that the phonological perplexity in the syllable context-dependent case is smaller than that in the syllable context-independent case, and thus the syntactic freedom is reduced. Also, HMM
/ BT compared with HMM (without BT), HMM / B
T has overwhelmingly less recognition errors, and when a syllable trigram with the number of succeeding syllables of 2 is used as the syllable context surrounding the syllable of the syllable syntax graph, the minimum total phoneme error rate is 10.
0% (bottom line) is obtained. This is HMM / BT
This is a significant improvement in recognition error compared to 54.0% (the second line from the top) when the conventional syntactic graph based on the phoneme trigram is used.

[0046]

As described above, according to the invention of claim 1,
A speech recognition apparatus that analyzes input speech, regards the input speech as a concatenation of phoneme models, applies a sequence of phoneme models to the input speech, and converts the input speech into an optimum phoneme sequence, And a syntax control graph that defines the order of occurrence of phonemes having a phoneme boundary likelihood greater than a predetermined value and a length of 3 or more in the phoneme sequence. The model operation means for selecting an optimal phoneme model is provided only when the phoneme sequence in which the phoneme sequence is generated is followed. As a result, the types of phonological boundaries that are assumed to be limited to the phonological boundaries in the phonological sequence specified in the syntax control graph are
There is an effect that the estimation accuracy of the phonological boundary is improved, and a speech recognition device with improved recognition accuracy is provided.

According to the second aspect of the present invention, the input voice is analyzed, the input voice is regarded as the concatenation of the phoneme models, the sequence of the phoneme models is applied to the input voice, and the input voice is converted into the optimum phoneme sequence. In a speech recognition device for conversion, a boundary likelihood calculating means for calculating a boundary likelihood of a phoneme from the input speech according to a kind of a phoneme boundary, and a boundary likelihood of the phoneme are set according to the kind of the phoneme boundary. Greater than the given value,
In addition, the model operation means for selecting an optimum phoneme model is provided only when the phoneme occurrence order specified in the syntax control graph that defines the occurrence order of phonemes of length 3 or more in the phoneme sequence is followed. In addition to preventing recognition of phoneme sequences that are not specified in the control graph, the types of phoneme boundaries assumed for the input speech are limited to the phoneme boundaries in the phoneme sequence specified in the syntactic control graph. As a result of being able to perform appropriate phoneme boundary estimation according to the type of phoneme boundary, there is an effect that the accuracy of phoneme boundary estimation is improved and a speech recognition apparatus with improved recognition accuracy is provided.

According to the third aspect of the present invention, as a method of generating the syntax control graph, a step of obtaining a syllable syntax graph that defines the occurrence order of syllables from a text database,
A method of generating a syntactic control graph having a process of obtaining an intra-syllable phonological graph that defines the order of occurrence of phonemes in a syllable from a speech database and a process of substituting the intra-syllable phonemic graph for each branch of the syllable syntactic graph is used. Therefore, the present invention provides a syntactic control graph in which variations in phonological structure within a syllable are considered together with the order of occurrence of syllables, and a speech recognition apparatus capable of modeling both variations in the spectrum of a phonological segment and variations in phonological structure within a syllable. Has the effect of providing.

According to the invention of claim 4, as a method of generating the syntactic control graph, a syllable is extracted from a text database as a syllable with a context together with a syllable context surrounding the syllable, and a syllable for defining the occurrence order of these syllables with a context. A step of obtaining a syntactic graph, a step of obtaining an intra-syllable phonological graph that defines the occurrence order of phonemes in a syllable for each syllable context of the syllable with context from a speech database, and a part corresponding to the syllable with context of the syllable syntactic graph. Since the method of generating the syntactic control graph including the step of substituting the syllable context graph in the syllable obtained from the syllable context matching the syllable context of the syllable with the context is used, the syllable that depends on the syllable context together with the syllable occurrence order. We provide a syntactic control graph that considers the variation of phonological structure in a syllable. There is an effect that both modeling of variation rhyme structure to provide a speech recognition apparatus capable.

According to the invention of claim 5, since the model operation means uses the syntax control graph generated based on the method of generating the syntax control graph according to claim 3 or 4, it follows the syllable occurrence order. Moreover, since only the phoneme model sequence according to the phoneme occurrence order in the syllable is the target of recognition, the types of phoneme boundaries assumed for the input speech are limited. As a result, there is an effect that the estimation accuracy of the phoneme boundary is improved, and a speech recognition device with improved recognition accuracy is provided.

[Brief description of drawings]

FIG. 1 is a configuration diagram of a voice recognition device according to an embodiment of the present invention.

FIG. 2 is an explanatory diagram of a phoneme system according to an embodiment of the present invention.

FIG. 3 is a diagram showing a structure of an HMM according to an embodiment of the present invention.

FIG. 4 is a diagram showing an overall process of generating a phoneme syntax graph according to an embodiment of the present invention.

FIG. 5 is a diagram illustrating a phoneme syntax graph generation process I according to an embodiment of the present invention.

FIG. 6 is a diagram exemplifying a phoneme syntax graph generation process II according to an embodiment of the present invention.

FIG. 7 is a diagram exemplifying a phoneme syntax graph generation process III according to an embodiment of the present invention.

FIG. 8 is a diagram showing conditions for evaluating one embodiment of the present invention.

FIG. 9 is a diagram showing an evaluation result of an example of the present invention.

FIG. 10 is a configuration diagram of a conventional voice recognition device.

FIG. 11 is a diagram showing a structure of an HMM in a conventional voice recognition device.

[Explanation of symbols]

 1 voice section detection means 2 feature extraction means 3 boundary likelihood calculation means 4 phonological boundary parameter storage means 5 HMM calculation means 6 HMM parameter storage means 7 optimal state sequence detection means 8 syntax control information storage means

Claims (5)

[Claims] 1. A speech recognition apparatus for analyzing an input voice, regarding the input voice as a concatenation of phoneme models, applying a sequence of phoneme models to the input voice, and converting the input voice into an optimum phoneme sequence, A boundary likelihood calculating means for calculating a boundary likelihood of a phoneme from the input speech, and a generation order of phonemes having a boundary likelihood of the phoneme larger than a predetermined value and having a length of 3 or more in a phoneme sequence. A speech recognition apparatus comprising: a model operation unit that selects an optimum phoneme model only when the phoneme occurrence order defined in the syntax control graph is followed. 2. A speech recognition apparatus for analyzing an input voice, regarding the input voice as a concatenation of phoneme models, applying a sequence of phoneme models to the input voice, and converting the input voice into an optimum phoneme sequence, Boundary likelihood calculation means for calculating a boundary likelihood of a phoneme in accordance with the type of a phoneme boundary from the input speech, and a boundary likelihood of the phoneme is larger than a value set according to the type of the phoneme boundary, and , Length in phoneme sequence 3
A speech recognition apparatus comprising: model operation means for selecting an optimum phoneme model only when the phoneme occurrence order defined in the syntax control graph defining the phoneme occurrence order is followed. 3. An input speech is analyzed, the input speech is regarded as a concatenation of phonological models, a sequence of phonological models is applied to the input speech according to the rules of a syntax control graph, and the input speech is converted into an optimum phonological string. As a method of generating the syntactic control graph of the speech recognition device, a process of obtaining a syllable syntactic graph defining a syllable occurrence order from a text database, and an intra-syllable phonological graph defining a syllable occurrence order from the speech database. A method of generating a syntax control graph for a speech recognition apparatus, comprising: a step of obtaining and a step of substituting the in-syllable phoneme graph into a syllable equivalent part of the syllable syntax graph. 4. An input speech is analyzed, the input speech is regarded as a concatenation of phonological models, a sequence of phonological models is applied to the input speech according to the definition of a syntax control graph, and the input speech is converted into an optimum phonological string. As a method of generating the syntactic control graph of the speech recognition apparatus, a process of extracting a syllable from a text database together with a syllable context surrounding the syllable as a syllable with context and determining a syllable syntactic graph that defines the occurrence order of these syllables with context, and A process of obtaining an intra-syllable phoneme graph that defines the occurrence order of phonemes in a syllable for each syllable context of the syllable with context from a speech database, and a syllable context of the syllable with context in a portion corresponding to the syllable with context of the syllable syntax graph. And a process of substituting the in-syllable phoneme graph obtained from the syllable context matching with the speech recognition device. A method of generating a syntax control graph for a table. 5. The model calculation means according to claim 3,
The speech recognition apparatus according to claim 1 or 2, wherein the syntax control graph generated based on the syntax control graph generation method according to claim 1 is used.