JP3403838B2 - Phrase boundary probability calculator and phrase boundary probability continuous speech recognizer - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a phrase boundary probability calculation device for calculating accent phrase boundary probabilities at each time of sentence speech, and use of phrase boundary probabilities for improving speech recognition accuracy using accent phrase boundary probabilities. It relates to a continuous voice recognition device.

[0002]

2. Description of the Related Art Generally, speech recognition uses a time series of spectral characteristic vectors obtained by performing frequency analysis of a speech signal at fixed time intervals as a characteristic parameter, and prepares standard patterns and patterns of recognition target words and sentences prepared in advance. It is realized by performing matching. However, it is considered that humans also recognize the voice by using the information of Accen and intonation. So in recent years, voice accent phrases,
That is, an attempt has been made to detect the boundary of a phrase having one accent nucleus and utilize it for speech recognition.

For example, FIG. 10 shows the document "Nakai, Shimodaira, Sagayama," Phrase boundary detection of unspecified speaker continuous speech based on clustering of pitch patterns ", IEICE Transactions,
A, Vol. J77-A, No. 2, pp206-2l
4 is a block diagram showing an example of a phrase boundary detection method described in "February 1, 1992".

In FIG. 10, l is a voice signal input terminal, 2
Is a voice signal input from the input terminal 1 of the voice signal, 3 is a pitch analyzing means for performing pitch division of the voice signal 2, 4 is a pause detecting means for detecting a pause section of the voice signal 2, and 5 is a pitch of an accent phrase. A pitch pattern template expressing a pattern, and 6 is a phrase boundary detecting means for detecting a phrase boundary time of the audio signal 2 using the pitch pattern template 5.

The pitch pattern template 5 in FIG. 10 uses a large number of pitch patterns of accent phrases in advance,
The pitch pattern is divided into several types and created as an average of these types. That is, the pitch pattern template 5 is a representative pattern of each type of pitch patterns of accent phrases.

The pitch pattern of the accent phrase is extracted from the sentence voice uttered by an unspecified number of speakers. Since there are a plurality of pitch patterns appearing in Japanese accent phrases, such as flat plate type, head height type, and middle height type, a plurality of pitch pattern templates are created.

As the above-mentioned pitch pattern, for example, a time series of logarithmic pitch frequencies is used.

The pitch pattern template 5 is created by the following procedure.

(Procedure 1 for creating a template) Pitch patterns of a large number of accent phrases are linearly expanded / contracted to have fixed time lengths, and the pitch patterns are made uniform in time length.

(Procedure 2 for creating a template) Clustering is performed by using pitch patterns that have the same time length.
Each pitch pattern is classified into n clusters. Here, the number n of clusters is, for example, 4 in consideration of the pattern of accent phrases appearing in Japanese.

(Procedure 3 for creating a template) For each of the clusters, the average of the pitch patterns aligned for the same time length is obtained and used as the pitch pattern template 5.

By the above procedure, n (= 4) pitch pattern templates are created.

The phrase boundary is detected as follows.

The voice signal 2 inputted from the input terminal 1 of the voice signal is inputted to the pitch analyzing means 3 and the pause detecting means 4.

The pitch dividing means 3 divides the pitch of the voice signal 2 at regular intervals to obtain a time series of the logarithmic pitch frequency of the voice signal 2. Here, for example, the lag window method is used for the pitch folding.

Further, the pause detecting means 4 obtains a time series of the power of the audio signal 2, extracts a section equal to or less than a predetermined power threshold, and a duration of the extracted section is equal to or more than a predetermined threshold. Is detected as a pause section, and the start time and end time of each pause section are output.

The phrase boundary detection means 6 performs pattern matching between the logarithmic pitch frequency time series output from the pitch folding means 3 and a template in which several pitch templates 5 are connected, and the connection boundary time of the pitch pattern template 5 is determined. Is output as the phrase boundary time.

That is, the phrase boundary detecting means 6 receives the logarithmic pitch frequency time series output from the pitch dividing means 3 and the start time and end time of the pause section output from the pause detecting means 4 as inputs, and pauses. For sections other than sections,
The phrase boundaries are detected by the following procedure.

(Phrase boundary detection procedure 1) One-Stage DP using n (= 4) pitch pattern templates 5 with the logarithmic pitch frequency time series as an input pattern.
Match.

(Phrase Boundary Detecting Procedure 2) After DP matching is completed, a back trace of the DP path is performed to obtain a connection sequence of the pitch pattern template having the smallest DP distance from the logarithmic pitch frequency time series, and the connection boundary time is calculated. Output as phrase boundary time.

[0021]

In the above-mentioned conventional phrase boundary detection method, it is determined only whether or not each time of the input voice is a phrase boundary. However, it is difficult to detect phrase boundaries by 100% and prevent false detection of non-phrase boundaries, and it cannot be assumed that the detection result is 100% correct. Further, with the above-mentioned conventional technique, the reliability of the detected phrase boundary cannot be quantitatively obtained. Therefore, it is difficult to use the detected phrase boundary for speech recognition.

The present invention has been made in order to solve the above problems, and does not judge whether or not each time of input speech is a phrase boundary by l and 0, but at each time of input speech. The purpose is to make it easier to use phrase boundary information for speech recognition. Further, another object of the present invention is to provide a method for improving the accuracy of speech recognition by utilizing the probability of phrase boundary likeness for speech recognition.

[0023]

A phrase boundary probability calculating device and a phrase boundary probability utilizing continuous speech recognizing device according to the present invention include one or a plurality of pitch patterns obtained by modeling a time series of pitch feature amounts of accent phrases. A pitch pattern model memory storing a model and a pitch forward probability and a pitch backward probability with respect to the time series of the pitch feature amount are calculated using the pitch pattern model, and the input is performed based on the pitch forward probability and the pitch backward probability. And a phrase boundary probability calculating means for calculating an accent phrase boundary probability at each time of speech.

In addition, an HMM having the same structure is learned for each time series of a plurality of vectors having different time lengths, and the average vector of each HMM obtained after learning is used as clustering data by a clustering method. Cluster pitch patterns and learn a pitch pattern model for each cluster.

The accent phrase boundary probability is provided with a weighting coefficient, and phrase boundary probability weighting means for weighting the accent phrase boundary probability is provided.

Further, a background model memory storing one or a plurality of background models that model the time series of the spectral feature vector of the voice, and the time series of the spectral feature vector of the input voice are input, Background model matching means for calculating a spectral feature forward probability and a spectral feature backward probability for the spectral feature vector time series using a background model, and an integrated forward which is a product of the phrase boundary probability and the spectral feature forward probability. Forward probability integrating means for calculating a probability, backward probability integrating means for calculating an integrated backward probability which is a product of the phrase boundary probability and the spectral feature backward probability, and a spectral feature vector of a phrase speech to be spotted A bunsetsu model that models a series And Le, said the time series of the spectral feature vectors of the speech and the integration forward probability and inputs the integrated backward probability, with a, a spotting means for performing spotting clause using the phrase model.

In addition, a chain of clause models is used as the background model.

Further, a sentence model network memory storing a sentence model network in which a time series of spectral feature vectors of a sentence speech to be recognized is stored, and a time series of spectral feature vectors of the input speech are input.
Using the sentence model network, the input speech is recognized, and a plurality of recognition result candidate sentences, spectral feature recognition scores of each recognition result candidate sentence, and a clause constituting the sentence for each recognition result candidate sentence are included. A continuous speech recognition unit that outputs a boundary time, the phrase boundary probability, the spectral feature recognition score of each of the plurality of recognition result candidate sentences, and the boundary time of the clauses that form each recognition result candidate sentence are input, and each recognition is performed. A probability integration unit that corrects the spectral feature recognition score using the phrase boundary probability at the boundary time of the clauses of the result candidate sentence and determines the recognition result candidate sentence based on the corrected recognition score.

[0029]

The pitch pattern model statistically models the time series of the pitch feature quantity of the accent phrase, and the phrase boundary probability calculating means uses the pitch pattern model to calculate the pitch forward probability and pitch for the time series of the pitch feature quantity. The backward probability is calculated, and the accent phrase boundary probability at each time of the input speech is calculated based on the pitch forward probability and the pitch backward probability.

Further, by learning the HMM having the same structure for each time series of a plurality of feature vectors having different time lengths, the time series of a plurality of feature vectors having different time lengths is non-linearly compressed, and Clustering is performed by aligning the time series of the individual feature vectors to the same time length.

Further, the phrase boundary probability weighting means is provided with a weighting coefficient for the accent phrase boundary probability and weights the accent phrase boundary probability, thereby integrating with the speech recognition score calculated from the spectral feature amount. Adjust the contribution rate of the accent phrase boundary probability in the case.

The forward probability integrating means calculates the integrated forward probability by calculating the product of the phrase boundary probability and the spectrum feature forward probability, and the backward probability integrating means calculates the product of the phrase boundary probability and the spectrum feature backward probability. By calculating the integrated backward probability, the spotting means performs spotting using the integrated forward probability and the integrated backward probability.

The background model constituted by a chain of bunsetsu models suppresses the spectrum feature forward probability and the spectrum feature backward probability at times other than the bunsetsu boundary time.

Further, the continuous speech recognition means recognizes the input speech by using the sentence model network, and recognizes a plurality of recognition result candidate sentences, the spectral feature recognition score of each recognition result candidate sentence, and each recognition result candidate sentence. And the boundary time of the bunsetsu that composes the sentence, and the probability integration means corrects the spectral feature recognition score using the phrase boundary probability at the boundary time of the bunsetsu of each recognition result candidate sentence.

[0035]

【Example】

Example l. FIG. 1 is a block diagram showing an example of the configuration of a phrase boundary probability calculation apparatus according to the invention described in claim 1. In FIG. 1, the same functional blocks as those in FIG. 10, which is an explanatory diagram of the conventional technique, are denoted by the same reference numerals, and description thereof will be omitted.

A characteristic point of this embodiment is that a pitch pattern model memory 7 for storing a plurality of pitch pattern models representing the pitch patterns of accent phrases and a pitch pattern memory for storing a network of these pitch pattern models. A model network memory 8 and a phrase boundary probability calculating means 9 for calculating a phrase boundary probability at each time for the voice signal 2 using the pitch pattern model network.
And to prepare.

The pitch pattern model described above represents the average of the types of accent phrases, and represents the representative pattern of the pitch patterns of accent phrases.

Before calculating the phrase boundary probability of a speech signal, it is necessary to create a pitch pattern model and a pitch pattern model network.

First, a method of creating a pitch pattern model will be described.

In this embodiment, a continuous pattern HMM (Hidden Markov Mode) is used as a pitch pattern model.
l, Hidden Markov Model). Further, as the parameter expressing the pitch pattern, a time series of logarithmic pitch frequency is used as in the prior art.

Here, the HMM is used as a pitch pattern model.
Is used to obtain the probability of phrase boundary at each time of the input voice as a probability. Like the prior art,
When the pitch pattern template is used, whether or not it is a phrase boundary can be obtained by 1 bit, and the probability of phrase boundary likeness cannot be obtained.

As shown in FIG. 2, the HMM is composed of several states (S ₁ , S ₂ , S ₃ , S ₄ , S ₅ , S ₆ ) and an arc connecting the states. A transition probability a _{ij of} transitioning each state through the arc and an output probability b _ij (x) of the voice feature vector x are given as parameters to each arc. Here, the subscript ij indicates the transition from the state S _i to the state S _j , and the transition probability a _ij is the state S _i to S.
_The probability that a transition to _j occurs. The output probability b _ij (x)
Is usually expressed by a multidimensional normal distribution, the average value and variance of the voice feature vector are given as parameters, and the probability that the voice feature vector x is output at the transition from the state S _i to S _j Shows the density.

The arc has an arc having only the transition probability a _ij as a parameter and contributing only to the transition between states without outputting the feature vector of the voice. The state transition by this arc is called null transition.

In the present embodiment, as the voice feature vector,
Since the logarithmic pitch frequency is used, the dimension number of the feature vector is one.

The HMM starts transition from a state (S ₁ in FIG. 2) called an initial state and reaches the final state (S ₆ in FIG. 2) by the state transition in the process of transition probability a _ij and output probability b. It is possible to generate a time series of various voice feature vectors with the probability calculated by _ij (x). That is, during the transition from the state S ₁ to the state S ₆ , the pitch pattern of the acceleration phrase is generated.

For example, when the logarithmic pitch frequency is used as the voice feature vector, the HMM can generate time series of various logarithmic pitch frequencies. In this case, H
It is possible to calculate the probability that the pitch pattern of each acceleration phrase as a time series of the logarithmic pitch frequency is generated by the MM. Therefore, the HMM can statistically model pitch patterns of various acceleration phrases.

The modeling of accent phrase pitch patterns by the HMM is performed by using a large number of accent phrase pitch patterns so that the probability of these pitch patterns being generated from the HMM is high, and the transition probability that is a parameter of the HMM is high. It is realized by estimating a _ij and output probability b _ij (x). This estimation method is called maximum likelihood estimation, the parameter estimation procedure is called HMM learning, and the data used for HMM learning is called learning data.

In this embodiment, the pitch pattern of the accent phrase used for learning is extracted from the sentence voice uttered by an unspecified number of speakers, as in the prior art.

The learning procedure is divided into two processes: pitch pattern clustering and HMM learning.

The procedures of clustering and learning will be described below.

First, the procedure of clustering is as follows.

(Clustering procedure 1) For each of a large number of pitch patterns of accent phrases, the HMM is learned by using each pitch pattern as learning data. That is, the HMM is learned for each pitch pattern. At this time, even if the sequence length of each accent phrase, that is, the length of the time series of the logarithmic pitch frequency is different, the structure of the number of states and transition of the HMM is common to each pattern. For example, the HMM structure shown in FIG. 2 is used.

(Clustering procedure 2) HM for each pitch pattern created by the above clustering procedure 1
From M, an average vector of output probabilities b _ij (x) is extracted. For example, when learning is performed using the HMM structure shown in FIG. 2, the output probabilities are b ₁₁ (X), b ₁₂ (X), b
₂₂ (X), b ₂₃ (X), b ₃₃ (x), b ₃₄ (x), b ₄₄
(X), b ₄₅ (x), b ₅₅ (x), b ₅₆ (x) total 10
Since there are such numbers, 10 average vectors are extracted for each pitch pattern.

(Clustering procedure 3) l for each pitch pattern created by the above clustering procedure 2
Clustering of each pitch pattern is performed by using 0 average vectors as clustering data and using, for example, the LBG algorithm. The number of clusters is, for example, 4 as in the conventional technique. Therefore, four clusters of pitch patterns are generated by this procedure.

Next, a learning procedure will be shown.

(Learning Procedure 1) For each cluster, the HMM is learned using the pitch pattern belonging to each cluster. However, for learning, the original pitch pattern before compression is used instead of using the data created by the clustering procedure 2. The data created by the above clustering procedure 2 is used only for clustering. For example, a forward / backward algorithm is used for learning. By this procedure, the HMM of the representative pitch pattern of each cluster, that is, the pitch pattern model is obtained. This pitch pattern model is stored in the pitch pattern model memory 7 shown in FIG.

As described above, in the data used for clustering, pitch patterns with different time lengths are HM
Non-linear compression is performed by an operation of learning M, and each pitch pattern is made to have the same data length. The average vector of each HMM is obtained by maximum likelihood estimation. Therefore, as compared with the case where the data length is made uniform by linear expansion and contraction as in the above-mentioned conventional technique, pattern distortion due to data expansion and contraction is suppressed to be small, and accurate clustering is possible.

The pitch pattern model network is
It is generated by connecting the pitch pattern models of each cluster as shown in FIG.

First, a network initial state S ₁ and a network final state S ₂₆ are newly generated. Next, the network initial state S ₁ and the initial state of each pitch pattern model are connected by a null transition. That is, the states S ₁ and S ₂ , the states S ₁ and S ₈ , the states S ₁ and S ₁₄ , and the states S ₁ and S ₂₀ are connected by null transitions. The null transfer is shown in dotted lines in FIG.

Next, S ₇ and S ₁₃ which are the final states of each model
And S ₁₉ and S ₂₅ are connected to the final state S _{26 of the} network with a null transition.

[0061] In addition to allow the loop from the final state S ₂₆ of the network to the initial state S _1, and generates a null transition in the initial state S ₁ f from the final state S _26. This completes the pitch pattern model network. This pitch pattern model network is stored in the pitch pattern model network memory 8 shown in FIG.

By constructing a pitch pattern model network having a loop from the final state S ₂₆ of the network to the initial state S _{1 in} this way, it is possible to make an arbitrary transition between the pitch pattern models of each cluster, and The logarithmic pitch frequency time series of speech can be expressed as a chain of pitch pattern models.

Next, a method of calculating the phrase boundary probability will be described.

The audio signal 2 input from the input terminal 1 of the audio signal is input to the pitch dividing means 3 and the pause detecting means 4.

Since the operations of the pitch folding means 3 and the pause detecting means 4 are the same as those of the prior art, the description thereof will be omitted.

The phrase boundary probability calculation means 9 receives the logarithmic pitch frequency time series output from the pitch analysis means 3 and the start time and end time of the pause section output from the pause detection means 4, and receives the pitch pattern model. Network memory 8
Using the pitch pattern model network of, the phrase boundary probability is calculated for the sections other than the pose section as follows.

The time series of the logarithmic pitch frequency is P ₁ P
₂ P ₃ , ..., P _T (subscript indicates time), the forward probability α (S _i , t) is first defined as in equation (1).

[0068]

[Equation 1] That α (S _i, t) starts a transition from the initial state S ₁ in the pitch pattern model network, the time series P ₁ P ₂ P ₃ logarithmic pitch frequency, ..., and outputs to P _t state It is the probability of reaching Si.

Further, the backward probability β (S _i , t) is set to (2)
Define like an expression.

[0070]

[Equation 2] That is, β (S _i , t) has a time axis in the opposite direction, starts transition from the final state S ₂₆ in the pitch pattern model network, and is P _T P _T-1 P _T- which is a backward time series of logarithmic pitch frequencies _{. 2} ,. ． ． , P _{t + 1} , and the probability of reaching the state S _i .

The forward probability α (S _i , t) can be calculated by the following recurrence formula.

[Initial value setting]

[Equation 3] [Recursion formula calculation for t = 1 to T, S _i (i = 1 to J)]

[Equation 4]

[Equation 5] The backward probability β (S _i , t) can be calculated by the following recurrence formula.

[Initial value setting]

[Equation 6] [Recursion Formula Calculation for t = T-1 to 1, S _i (i = J to 1)]

[Equation 7]

[Equation 8] The forward probability α (S _i , t) and the backward probability β (S
_i , t), the phrase boundary probability S at time t
_p (t) (t = 1 to T) is calculated by the equation (9).

[0074]

[Equation 9] Forward probability α (S _i , t) and backward probability β (S _i ,
From the definition of t), in the formula (9), the denominator is the time series P ₁ P of the pitch pattern when all state transitions are considered.
₂ P ₃ ,. ． ． , P _T is the probability of being generated. That is, it is the total probability that the time series is generated by the pitch pattern model network. Further, the numerator is the time series P ₁ P ₂ P ₃ ,. ． ． , P _T is the sum of the probabilities of being generated. Therefore, by taking the ratio of both, time t
, The probability of transitioning the initial state of each cluster HMM at
That is, the probability of being a phrase boundary can be obtained. This can be thought of as using the forward-backward algorithm in HMM learning for phrase boundary probability calculation.

Further, in the equation (9), it is possible to replace the probability sum of the numerator and the denominator with the maximum value selection. That is, the phrase boundary probability can also be calculated using the following formula (10).

[0076]

[Equation 10] In the equation (10), the time when the value of the equation becomes 1 is the boundary time of the model when the optimum series of the pitch pattern model is obtained, and phrase boundary detection equivalent to the above-mentioned conventional technique is also possible.

Example 2. FIG. 4 is a block diagram showing a configuration example of the phrase boundary probability calculation device according to the second embodiment of the present invention. 4, the same functional blocks as those in FIG. 1 which is an explanatory diagram of the embodiment 1 are denoted by the same reference numerals, and the description thereof will be omitted.

A characteristic point of this embodiment is that a phrase boundary probability weighting means 10 is newly added. The pitch pattern model and the pitch pattern model network are created in the same manner as in the first embodiment.

Next, the operation will be described.

The voice signal 2 inputted from the input terminal 1 of the voice signal is inputted to the pitch folding means 3 and the pause detecting means 4. Then, by the same operation as that of the example 1, the phrase boundary probability calculating means 9 causes the phrase boundary probability S _p (t) at the time t.
(T = 1 to T) is output.

The phrase boundary probability weighting means 10 receives the phrase boundary probability S _p (t) (t = l to T) as an input and uses the formula (11) to weight the phrase boundary probability S ′ _p (t) (t = 1). ~ T)
Is calculated and output. In the formula (11), w is a constant that determines the degree of weighting, and as will be described in a later example, when integrating the accent phrase boundary probability and the speech recognition score calculated from the spectral feature, the accent phrase boundary is used. This is for adjusting the probability contribution rate. By adjusting the contribution rate, the voice recognition can be performed with higher accuracy.

[0082]

[Equation 11] Example 3. FIG. 5 is a configuration block diagram showing a configuration example of a continuous speech recognition apparatus according to the third embodiment of the present invention. The continuous speech recognition apparatus according to the present invention comprises a phrase boundary probability calculation unit 11 and a spotting unit 19.

Phrase boundary probability calculation unit 1l in this embodiment
Since the phrase boundary calculation device described in the second embodiment is used, the same reference numerals are given and the description thereof will be omitted. The phrase boundary calculation device described in the first embodiment can be used as the phrase boundary probability calculation unit 11.

The spotting unit 19 is a spectrum dividing unit 12 for calculating the time series of the spectrum feature vector of the input voice, and a background model memory 13 for storing a background model that models the time sequence of the spectrum feature vector of the voice. And a background model matching means l4 for inputting a time series of the spectral feature vector of the input speech and calculating a spectral feature forward probability and a spectral feature backward probability for the spectral feature vector time series using the background model l3.
And a forward probability integrating means 15 for calculating a product of the weighted phrase boundary probability output from the phrase boundary probability calculating unit 11 and the spectral feature forward probability, and calculating a pre-integration probability.
Backward probability integrating means 16 for calculating a product of the weighted phrase boundary probability output from the phrase boundary probability calculation unit 11 and the spectral feature backward probability to calculate an integrated backward probability.
And a phrase model memory 17 for storing a phrase model that models a time series of spectral feature vectors of the phrase speech to be spotted, a time series of the spectrum feature vector of the voice, the integrated forward probability, and the integrated backward probability. It is composed of spotting means 18 for inputting and, and using the phrase model to spot a phrase.

Spotting is a technique for extracting a predetermined word or phrase from the input voice. For example, "Tomorrow,
Spotting the phrase "To Tokyo" from the voice that says "I'm going to Tokyo" means finding the start time, end time, and spotting score of the phrase "To Tokyo". is there.

Although there are several spotting methods, in the present embodiment, as will be described later, a model capable of expressing a feature vector time series of an arbitrary voice is connected before and after a phrase model to be spotted and input. A method of obtaining a matching section between the phrase model and the input speech is used in the process of performing pattern matching with the feature vector time series of all sections of the speech. Here, a model connected before and after the phrase model is called a background model.

In this embodiment, HMMs are used as both the background model and the clause model. The difference from the pitch pattern model is that the feature vector time series is not a pitch pattern but a time series of spectral feature vectors. Here, the time series of the spectrum feature vector is used because the phrase cannot be recognized unless the spectrum feature vector is used. On the other hand, the pitch pattern model is very useful for phrase boundary detection, and the pitch pattern model is also used in the phrase boundary probability calculation unit 11 in this embodiment.

As the background model, for example, a syllable model network as shown in FIG. 6 is used.
This is exactly the same as the method of constructing the pitch pattern model network described in the first embodiment. By preparing a syllable model for all syllables appearing in Japanese, it is possible to model the time series of spectral feature vectors of arbitrary utterances in Japanese.

In addition, the bunsetsu model is an independent word that constitutes a bunsetsu, for example, a model of "Tokyo", and an attached word, for example, "he".
A model in which several models are connected is used, and a bunsetsu model is prepared for all bunsetsu to be spotted.

Next, the operation will be described.

The voice signal 2 input from the voice signal input terminal 1 is supplied to the phrase boundary probability calculation unit 11 and the spotting unit 19.
Entered in.

The phrase boundary probability calculation unit 11 performs exactly the same operation as in the second embodiment, and the weighted phrase boundary probability S'at time t.
(T) (t = 1 to T) is output.

The speech signal 2 input to the spotting unit 19 is spectrally divided by the spectral dividing unit 12, and a time series X ₁ X of spectral feature vectors is obtained.
₂ X ₃ , ..., X _T. The spectral feature vector X is, for example, an LPC cepstrum.

The background model matching means 14 uses the time series of the spectrum feature vector output from the spectrum analysis means 12 as an input, and uses the background model from the background model memory 13 as follows. Probability S _fw (t) (t =
1 to T) and S, which is the backward probability of the spectral feature.
Calculate _bw (t) (t = 1 to T).

Spectral feature forward probability S _fw (t) (t
= L to T) is calculated by the equation (12).

[0096]

[Equation 12] That is, S _fw (t) starts the transition from the initial state (S ₁ ) in the syllable model network shown in FIG. 6 as a background model, and the time series of spectral feature vectors X ₁ X ₂ X ₃ , ..., It is the probability of outputting up to X _t and reaching the final state (S _J ).

However, [initial value setting]

[Equation 13] [Recursion formula calculation for t = 1 to T, S _i (i = 1 to J)]

[Equation 14]

[Equation 15] In addition, the spectral feature backward probability S _bw (t) is (l
6) Calculate with the formula.

[0098]

[Equation 16] That is, S _bw (t) has the time axis in the opposite direction, and the final state (S _bw (t) in the syllable model network shown in FIG.
Start the transition from _J), and reaches the initial state (S ₁₎ and outputs _{X T X T-1 X T} -2 is a retrospective time series of spectral feature vectors, ..., up to X t + ₁ It is a probability.

However, [initial value setting]

[Equation 17] [Recursion Formula Calculation for t = T-1 to S _i (i = J to 1)]

[Equation 18]

[Formula 19] The forward probability integrating means 15 outputs the weighted phrase boundary probability S ′ _p (t) which is the output of the phrase boundary probability calculating section 11, and the spectrum feature forward probability S _fw (t) which is the output of the background model matching means 14. ) As an input, S ′ _fw (t) (t
= 1 to T) is calculated.

[0100]

[Equation 20] The backward probability integrating means 16 outputs the weighted phrase boundary probability S ′ _p (t) output from the phrase boundary probability calculating section 11 and the spectral feature backward probability S _bw (t) output from the background model matching means 14. Input as (21)
_S'bw (t) which is the integrated backward probability according to the formula
(T = 1 to T) is calculated.

[0101]

[Equation 21] The spotting means 18 receives the time series of the spectral feature vector of the voice, the previous integrated forward probability and the integrated backward probability as inputs, and uses the phrase model according to equation (22) for each spotting target phrase. F ⁽ⁿ⁾ (t) (t = 1 to T), which is the spotting score of the phrase, is calculated. Here, the shoulder subscript (n) is the number of the bunsetsu model, and n = 1, 2, 3 ,. ． ． N (N: total number of target phrases for spotting).

Then, the spotting score F
^{When (n)} (t) is equal to or greater than a predetermined threshold value, the time t, the phrase model number n, and the spotting score F ⁽ⁿ⁾ (t) are output as the spotting result.
The time t at which the spotting score F ⁽ⁿ⁾ (t) exceeds a predetermined threshold is a time at which there is a high probability that it is a bunsetsu boundary.

[0103]

[Equation 22] However, [Initial value setting]

[Equation 23] [Recursion formula calculation for t = 1 to T, S _i (i = 1)]

[Equation 24] [Recursion formula calculation for t = 1 to T, S _i (i = 2 to _J , S _J : final state)]

[Equation 25]

[Equation 26] As is clear from the expressions (22) and (24), the spotting score becomes large in the sections where the integrated forward probability and the integrated backward probability are large. Therefore, the spectral feature forward probability calculated using the background model, and the spectral feature backward probability,
By multiplying the weighted phrase boundary probability, it is possible to suppress the spectral feature forward probability and the spectral feature backward probability at times other than phrase boundaries, and bunsetsu is mistakenly spotted at times other than phrase boundaries. Can be suppressed. In most cases, the phrase boundary coincides with the bunsetsu boundary, and in the end, it is possible to prevent the bunsetsu from being mistakenly spotted at a time other than the bunsetsu boundary.

Example 4. An example in which a phrase model chain is used as the background model in the speech recognition apparatus using phrase boundary probabilities described in the third example will be described.

FIG. 7 shows an example of the configuration of a background model dedicated to voice input regarding an action schedule, for example, when action schedule management is used as a recognition task.

In FIG. 7, a portion surrounded by a square represents a word model, and a bunsetsu model is constructed as a network of word models. By incorporating the models of all the clauses appearing in the recognition task into the background model, the time series of the spectral feature vectors for all the utterances can be represented by the background model if the utterance is in the recognition task.

The operation of spotting is exactly the same as that of the phrase boundary probability utilizing speech recognition apparatus described in the third embodiment except that the phrase model chain is used as the background model, and therefore the description thereof is omitted.

By using the bunsetsu model chain as the background model, as compared with the case of using the syllable chain as in the background model described in the third embodiment, the spectral feature forward probability S described in the third embodiment.
_{Since fw} (t) (t = 1 to T) and the spectral feature backward probability S _bw (t) (t = 1 to T) can be suppressed to smaller values other than the bunsetsu boundary time, at times other than the bunsetsu boundary time. It is possible to further prevent bunsetsu from being spotted by mistake.

Example 5. FIG. 8 is a diagram showing an example of the configuration of a continuous speech recognition apparatus according to the invention of claim 7. The continuous speech recognition apparatus according to the present invention is a phrase boundary probability calculation unit ll.
And the continuous speech recognition unit 23.

The phrase boundary probability calculation unit ll in this embodiment.
Since the phrase boundary calculation device described in the second embodiment is used, the same reference numerals are given and description thereof is omitted. The phrase boundary calculation device described in the first embodiment can also be used.

The continuous speech recognition unit 23 stores the spectrum analysis means 12 for calculating the time series of the spectral feature vector of the input speech, and the sentence model network that models the time series of the spectral feature vector of the sentence speech to be recognized. The sentence model network memory 20 and the time series of the spectral feature vector of the input speech are input, and the input speech is recognized using the sentence model network.
A continuous speech recognition unit 21 that outputs a plurality of recognition result candidate sentences, the spectral feature recognition score of each recognition result candidate sentence, and the boundary time of the clauses forming the sentence for each recognition result candidate sentence.
And using the weighted phrase boundary probabilities at the boundary times of the clauses of each recognition result candidate sentence, the spectral feature recognition score is corrected, and the final recognition result candidate sentence is determined based on this corrected recognition score. It is composed of probability integration means 22.

In this embodiment, an HMM is used as the sentence model network. The feature parameter modeled by the sentence model network is a time series of spectral feature vectors as in the phrase model described in the third embodiment. In continuous speech recognition, the total number of sentences to be recognized is very large. Therefore, instead of preparing a model for each sentence to be recognized, sentences formed by connecting word models as shown in FIG. Use a model network.

Further, as shown in FIG. 9, information on the bunsetsu delimiter position is added to the sentence model network.

Next, the operation will be described.

The voice signal 2 input from the input end 1 of the voice signal is input to the phrase boundary probability calculation unit 11 and the continuous voice recognition unit 23.

[0116] The phrase boundary probabilities calculator ll is exactly the same operation as in Example 2, the weighted phrase boundary probabilities at time t S _'p
(T) (t = 1 to T) is output.

The speech signal 2 input to the continuous speech recognition unit 23 is spectrally divided by the spectrum dividing unit 12, and the time series of spectral feature vectors X ₁ X ₂ X ₃ ,
..., converted to X _T. Spectral feature vector X
Is, for example, the LPC cepstrum.

The continuous speech recognition means 2l receives the time series of the spectrum feature vector output from the spectrum analysis means 12 as an input and uses the sentence model network from the sentence model network memory 20 to perform continuous speech by, for example, an N-best algorithm. The recognition is performed and the N recognition result candidate sentences and the spectral feature recognition score G of each recognition candidate sentence are recognized.
⁽ⁿ⁾ (n = 1 to N) and the boundary time t ⁽ⁿ⁾ _k (n = 1 to N, k = 1 ^{) of} the bunsetsu that composes each recognition result candidate sentence.
˜K ⁽ⁿ⁾ , where K ⁽ⁿ⁾ : the number of clause boundaries included in the n-th recognition result candidate sentence) is output.

The probability integrating means 22 is the phrase boundary probability calculating section 1
1, the weighted phrase boundary probability, the output of the continuous speech recognition means 2l, the spectral feature recognition score of each of the plurality of recognition result candidate sentences, and the boundary time of the clause forming the sentence for each recognition result signature. By inputting and, the corrected recognition score G ⁽ⁿ⁾ '(n = 1 ~
Calculate N). And as a recognition result, the 'a high order, and the recognition result candidate sentences, correction recognition score G ^(n)' Correction recognition score G ⁽ⁿ⁾ and outputs a.

[0120]

[Equation 27] As mentioned above, in addition to the spectral feature recognition score,
By integrating the weighted phrase boundary probabilities calculated from the pitch pattern, the probability of recognition result sentence candidates having incorrect phrase boundaries can be suppressed, and the accuracy of sentence speech recognition can be improved.

[0121]

As described above, according to the present invention, the time series of the pitch feature amount of the accent phrase is statistically modeled by the pitch pattern model, and the phrase boundary probability calculation means uses the pitch pattern model. The pitch forward probability and the pitch backward probability with respect to the time series of the pitch feature amount are calculated, and the accent phrase boundary probability at each time of the input speech is calculated based on the pitch forward probability and the pitch backward probability. It is possible to integrate the recognition score of the recognition result candidate of the voice recognition and the accent phrase boundary probability calculated from the pitch pattern, and determine whether or not each time of the input speech is a phrase boundary,
Compared with the conventional technique of determining by 0, there is an effect that the accent phrase boundary information is easily used for voice recognition.

Further, the data used for pitch pattern clustering for creating the pitch pattern model is such that pitch patterns having different time lengths are non-linearly compressed by an operation of learning HMM, and each pitch pattern has the same data length. Since the average vector of each HMM is obtained by the maximum likelihood estimation, the pattern distortion due to the data expansion and contraction can be suppressed to be smaller than that in the case where the data lengths are aligned by the linear expansion and contraction as in the above-mentioned conventional technique. , Accurate clustering is possible.

Further, since the phrase boundary probability weighting means is provided with a weighting coefficient for the accent phrase boundary probability and weights the accent phrase boundary probability, the accuracy of speech recognition by considering the accent phrase boundary probability. The contribution rate of the accent phrase boundary probability can be set to maximize the improvement.

Further, the forward probability integrating means calculates the integrated forward probability by calculating the product of the weighted phrase boundary probability and the spectral feature forward probability, and the backward probability integrating means calculates the weighted phrase boundary probability and the spectral feature backward probability. The integrated backward probability is calculated by obtaining the product of and, and the spotting means performs the spotting by using the integrated forward probability and the integrated backward probability.
Spectral feature forward probability at times other than phrase boundaries,
It is possible to reduce the backward probability of scan vector features to a small value, and to prevent bunsetsu from being mistakenly spotted at times other than phrase boundaries.

Further, by using the bunsetsu model chain as the background model, the spectral feature forward probability and the spectral feature backward probability can be suppressed to a smaller value at times other than the bunsetsu boundary time. Can be further suppressed from being spotted.

Further, the continuous speech recognition means recognizes the input speech by using the sentence model network, and recognizes a plurality of recognition result candidate sentences, spectral feature recognition scores of each recognition result candidate sentence, and each recognition result candidate sentence. The boundary time of the clauses that compose the sentence is calculated for each, and the probability integrating unit corrects the spectral feature recognition score using the weighted phrase boundary probability at the boundary time of the clauses of each recognition result candidate sentence. The score of the recognition result sentence candidate having an incorrect boundary can be suppressed, and the accuracy of sentence voice recognition can be improved.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration example of a phrase boundary probability calculation device according to a first exemplary embodiment of the present invention.

FIG. 2 is a diagram for explaining the configuration of an HMM.

FIG. 3 is a diagram for explaining a configuration of a pitch pattern model network.

FIG. 4 is a block diagram showing a configuration example of a phrase boundary probability calculation device according to a second exemplary embodiment of the present invention.

FIG. 5 is a block diagram showing a configuration example of a phrase boundary probability utilizing continuous speech recognition device according to a third embodiment of the present invention.

FIG. 6 is a diagram for explaining a background model of Example 3 configured by syllable chains.

FIG. 7 is a diagram for explaining a background model of Example 4 configured by a clause chain.

FIG. 8 is a block diagram showing a configuration example of a continuous speech recognition apparatus using phrase boundary probability according to a fifth exemplary embodiment of the present invention.

FIG. 9 is a diagram for explaining the configuration of a sentence model network.

FIG. 10 is a block diagram showing a configuration example of a phrase boundary detection device of a conventional technique.

[Explanation of symbols]

1 voice signal input terminal, 2 voice signal, 3 pitch analyzing means, 4 pause detecting means, 5 pitch pattern template, 6 phrase boundary detecting means, 7 pitch pattern model memory, 8 pitch pattern model network memory, 9
Phrase boundary probability calculation means, 10 Phrase boundary probability weighting means, 11 Phrase boundary probability calculation part, 12 Spectrum analysis means, 13 Background model memory, 14 Background model matching means, 15 Forward probability integration means,
16 backward probability integration means, 17 bunsetsu model memory,
18 spotting means, 19 spotting part,
20 sentence model network memory, 21 continuous speech recognition means, 22 probability integration means, 23 continuous speech recognition section.

─────────────────────────────────────────────────── ─── Continuation of the front page (51) Int.Cl. ⁷ Identification code FI G10L 15/14 G10L 3/00 531W 15/18 535Z 537E 537G 9/00 301A (56) Reference Patent 2793137 (JP, B2) Patent 3061292 (JP, B2) JP-A-4-66999 (JP, A) JP-A-60-229099 (JP, A) Hanazawa, Abe, Nakajima, Examination of bunsetsu spotting for semantic driven speech understanding system, Japan Proceedings of Autumn Meeting of ASJ, Japan, October 31, 1994, 1-Q-14, 169-170 Nakai, Shimodaira, Sagayama, Independent speaker continuous speech based on pitch pattern clustering Phrase boundary detection, IEICE Journal A, Japan, February 25, 1994, Vol. J77-A, No. 2, p. 206-214 Kitaoka, Kawahara, Doshita, Phrase spotting for free speech recognition and understanding, Technical Report of IEICE [Voice], Japan, December 10, 1993, SP93-116, NLC93-56 , P. 15-22 (58) Fields investigated (Int.Cl. ⁷ , DB name) G10L 15/00-15/28 JISST file (JOIS)

Claims (7)

(57) [Claims] 1. A pitch folding means for calculating a time series of pitch feature amounts of input speech, and a modeled time series of pitch feature amounts of accent phrases.
A pitch pattern model memory that stores one or a plurality of pitch pattern models, and a time series of the pitch feature amount as an input, and a chain of the pitch pattern models from the initial to a predetermined time of the time series of the pitch feature amount. A pitch forward probability that is the probability that the pitch feature amount is output, and a pitch backward probability that is the probability that the pitch feature amount from the end of the time series of the pitch feature amount to a predetermined time by the chain of the pitch pattern model is output. And, based on this pitch forward probability and pitch backward probability, the total probability that the time series of the pitch feature amount is generated by the chain of the pitch pattern models, and the pitch pattern model at the predetermined time. Depending on the ratio with the probability that the time series of the pitch feature will be generated after passing through the boundary, it will be determined at each time of the input speech. And a phrase boundary probability calculating means for calculating an accent phrase boundary probability, and a phrase boundary probability calculating device comprising: 2. A sequence characterized by learning an HMM having the same structure for each sequence of a plurality of feature vectors having different sequence lengths, and using an average vector for each HMM obtained after learning as clustering data. A clustering method for a series of feature vectors with different lengths. 3. The accent pattern pitch pattern is clustered by the clustering method according to claim 2,
2. The phrase boundary probability calculation device according to claim 1, wherein a pitch pattern model is created and used based on the clustering result. 4. The phrase boundary probability calculation device according to claim 1, further comprising a phrase boundary probability weighting means for weighting accent phrase boundary probabilities by providing a weighting coefficient for accent phrase boundary probabilities. 5. A spectrum analysis means for calculating a time series of a spectrum feature vector of an input voice, a background model memory storing a background model modeling a time series of a spectrum feature vector of the voice, and the input voice Spectral feature forward probability, which is the probability that a spectral feature vector time series is input, and the spectral feature vector from the initial of the spectral feature vector time series to a predetermined time is output by the background model chain, and the background. Background model matching means for calculating a spectrum feature backward probability which is a probability that a spectrum feature vector from the end of the spectrum feature vector time series to a predetermined time by a chain of models, and the spectrum feature forward probability, Claim l Alternatively, a forward probability integrating unit that calculates an integrated forward probability that is a product of the phrase boundary probability output from the phrase boundary probability calculating device according to claim 4, the spectrum feature backward probability, and the claim 1 or claim 4. A backward probability integration means for calculating an integrated backward probability that is a product of the phrase boundary probability output from the described phrase boundary probability calculator, and a phrase that models the time series of the spectral feature vector of the phrase speech to be spotted. A bunsetsu model memory storing a model, and a spotting means for spotting bunsetsu using the bunsetsu model by inputting the time series of the spectral feature vector of the voice, the integrated forward probability and the integrated backward probability A continuous speech recognition apparatus using phrase boundary probabilities, characterized in that 6. The continuous speech recognition apparatus using phrase boundary probabilities according to claim 5, wherein a chain of clause models is used as the background model. 7. A spectrum analysis means for calculating a time series of spectral feature vectors of input speech, a sentence model network modeling a time series of spectral feature vectors of sentence speech to be recognized, and a spectral feature vector of input speech. , The input of the time series, the input speech recognition is performed, the plurality of recognition result candidate sentences, the spectral feature recognition score of each recognition result candidate sentence, and the sentence for each recognition result candidate sentence. Continuous speech recognition means for outputting the boundary time of the bunsetsu that composes, the spectral feature recognition score of each of the plurality of recognition result candidate sentences, and the boundary time of the bunsetsu that composes each recognition result candidate sentence; The phrase boundary probability output from the phrase boundary probability calculation device according to item 4 is used as an input, and the phrase boundary probability at the boundary time of the phrase of each recognition result candidate sentence is determined. A phrase-boundary-probability-based continuous speech recognition apparatus comprising a probability integration unit that corrects a spectral feature recognition score using a rate and determines a recognition result candidate sentence based on the corrected recognition score.