JPH11237893A - Speech recognition system and phoneme recognizing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve the phoneme recognition rate by matching an input phoneme against a dictionary template and expanding a range in which an optimum start end is possibly present by making use of results up to a specific frame when the optimum start end is set for the 1st phoneme of the dictionary template in word spot calculation. SOLUTION: A feature extraction part 13 takes a frequency analysis of voice data inputted to a voice input part 12 to obtain a spectrum sequence, which is inputted to a phoneme recognition part 14. A phoneme sequence is obtained as its output and supplied to a matching part 16, which finds a cumulative distance of matching distances between the input phonemes and the dictionary template up to an (n)th frame of the input phonemes when matching the phoneme sequence against templates in the dictionary. Then the most similar word or word sequence is outputted as a recognition result. At this time, the path of DP at a word spot calculation part is changed so that the optimum start end can be selected out of even frames far from a tail end.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a phoneme recognition method in a continuous word speech recognition system having a phoneme recognition unit at a preceding stage.

[0002]

2. Description of the Related Art As an example of a speech recognition apparatus, there is a discrete word speech recognition system shown in FIG. This system is shown in FIG.
As shown in FIG. 1, voice data is input to a voice input unit 12 from a voice input device 11 such as a telephone or a microphone.
The audio data input to the audio input unit 12 is supplied to a feature extraction unit 13, where the audio data is subjected to frequency analysis. A spectrum sequence is obtained from the result of the frequency analysis, and the spectrum sequence is input to the phoneme recognition unit 14.
The phoneme recognition unit 14 is configured by a neural network (not shown) whose output is duplicated.

The above-mentioned neural network is composed of an input layer, a hidden layer, and an output layer.
The spectrum of the frame is input, and the phoneme corresponding to the center spectrum of the frame is transmitted according to the value of the unit in the output layer. Since the output units are duplicated, two units are associated with each phoneme category. On the other hand, as a result, two units are selected from those having the largest output value, and the phonemes corresponding to the two units are obtained as the first and second phoneme candidates.

[0004] The similarity between the recognized phoneme candidate sequence and the template 15 in the dictionary having the phoneme pattern of the vocabulary to be recognized is matched by the matching unit 16 by the DP (Dynamic Programming) method. Then, the matching unit 16 outputs the most similar word or word string as a recognition result.

Here, an outline of a general continuous word recognition algorithm will be described. Now, it is assumed that word connection conditions (grammar) are described by a finite state automaton shown in FIG. This serves to limit the number of recognized words, connections between words, and the like. Result of phoneme recognition of input pattern T =
{a (1), a (2), a (3),... a (t)} (the number of frames t) from the i-th frame to the j-th frame {ai,..., aj} and the dictionary word template Write the DP matching distance to n = {b (1), b (2), ..., bN (n)} as dist [n] [i] [j] (N (n): length of template n ). dist [n] [i] [j] is obtained by DP matching or the like. This value is also called the word spot value. Also,
Let i be the word spot start and j be the word spot end.

FIG. 5 shows an example of an automaton for processing two-digit numbers (the number of states = 3). In the case of a two-digit number, the second digit is output when transitioning to state 1, and the first digit is output when transitioning from state 1 to state 2. In other transitions (from state 0 to state 2), a two-digit number is not output.

Now, consider the case where the number of states is stat num, the number of input phoneme frames is len obj (that is, t = len obj), and the number of dictionary words is word num. Automaton state stat,
Cumulative matching distance frm s of input phoneme to frame k
If you want to find cr [stat] [k] (0 <= stat <last stat, 0 <
= k <last frm), which is expressed by the following equation (1).

Frm scr [stat] [k] = min {frm scr [from stat] [m] + dist [n] [m + 1] [k]} (1) where word n is generated from state p F (p, n) = q (2) indicates that from stat and n are 0 <n <word num, f (from sta
t, n) = stat, and 0 <m <k (actually, the range of m is further limited by the amount of calculation, etc.), and min is from stat, n, m To take the minimum when moved within that range.

From stat, which satisfies the above equations (1) and (2),
n, m are respectively frm stt [stat] [k], frm tpl [stat] [k],
frm frm [stat] [k] (stt: state, tpl: template, frm:
Frame). Such a procedure is called 0 <= k <len obj, 0 <=
Calculate about stat <stat num. In an actual program, the following processing is generally performed.

First, the algorithm of the cumulative distance in the k-th frame will be described. Execute the following for all states stat (0 <= stat <stat num) Execute the following for all dictionary words n (0 <= n <word num) Cumulative distance scr = min {frm scr [ from stat] [m] + dist [n] [m +
1] [k]}. Here, min indicates the minimum value when only m and from stat are operated. “From stat” satisfies the above expression (2). If frm scr [stat] [k]> scr, execute frm scr [stat] [k] = scr, frm tpl [stat] [k] = n, frm frm
[stat] [k] = (satisfies m), frm stt [stat] [k] = (satisfies from stat) Then, backtrace is performed as follows to obtain a recognized word string.

FIG. 6 shows a flowchart of the algorithm of the cumulative distance (input j-th frame). In FIG. 6, after performing the above processing in step S1, the processing in step S2 is performed. Step S2 executes the above processing. When the processing in step S2 has been executed, the above processing is executed in step S3. Thereafter, the above-described determination is performed in step S4. If "yes", the process in step S5 is performed, and if "no", the process proceeds to step S6. When the execution of step S5 is completed, it is determined in step S6 whether the dictionary word template n is larger than the dictionary word word num. If the result of this determination is "yes", it is determined in step S7 whether the automaton state stat is greater than the number of states stat num, and if "yes", the cumulative distance calculation ends. In steps S6 and S7, "n
If "o", the flow returns to the processing of steps S3 and S2.

The algorithm of the back trace is performed as follows. k = len obj, stat = stat num (len obj: number of input frames, stat num: final state number of finite state automaton) frm tpl [stat] [k] is output as a recognition result. afterwards,
Let k = frm frm [stat] [k] and stat = frm stt [stat] [k]. If k = 0, end; otherwise, go to above.

Next, an algorithm of the extended continuous DP used as a continuous word speech recognition algorithm will be described. In extended continuous DP, all
Instead of finding the word spot value dist [n] [i] [j] for (i, j), when (j, n) is given, minimize dist [n] [i] [j]. I, its value (i min) and dist [n] [i mi
Find only n] [j]. That is, the word spot value for only the optimal start (i min) with respect to the end j is used.

Now, dist [n] [i min] [j] is converted to aug dist [n] [j]
And the optimal starting point (i min) is aug ini [n] [j]. (Au
g: augmented, ini: initial, a variable for calculating the starting point) aug dist [n] [j] and aug ini [n] [j] are as follows. Here, for the sake of simplicity, the DP matching calculation path for designating the transition of the DP score is assumed to be a path having no inclination limit as shown in FIG. 7A. As shown, a path having a slope restriction is more commonly used.

Next, an algorithm for calculating an extended continuous DP matching distance (word spot) will be described. For all input frames j (0 <= j <len obj),
Execute the following for all dictionary templates n (0 <= n <word num) Execute the following for all template phonemes tpl (0 <= tpl <N (n)) of n When tpl = 0 Dist tmp [tpl] [j] = value (tpl, j) (tem: abbreviation for temporary) When ini tmp [tpl] [j] = j tpl> = 1, dist tmp [tpl] [j] = min {dist tmp [tpl-1] [k]} + value (tpl, j) (j-2 <= k <= j) If k that satisfies the above equation is min k, ini tmp [tpl] [j] = ini tmp [tpl-1] [min k] where value (tpl, j) is the distance between phonemes between the phoneme of the tpl frame of template n and the phoneme of the jth frame of the input frame. The inter-frame distance is, for example, when the score when the phonemes match is set to “0” and the score when they do not match is set to “1”, the j-th phoneme a (j) of the input frame
And the inter-frame distance between the i-th phoneme b (i) of the template n is defined as follows. aug dist [n] [j] = dist tmp [N (n) -1] [j], aug ini [n] [j] = ini tmp [N (n) -1] [j] aug dist [] above Using [], aug ini [] [], from scr [] []
However, since only the optimal starting point is obtained for frame j, when calculating the minimum value described in the above “Algorithm of Cumulative Distance in k-th Frame”, the word spot score around the optimal starting point is calculated. Is approximately obtained.

FIG. 8 is a flowchart of the algorithm for calculating the extended continuous DP matching distance (word spot). In this flowchart, first, in order to execute the above-mentioned processes in steps S1 to S3,
Set the initial values of j, n, tpl to “0”. Next, it is determined whether or not the template phoneme tpl is “0” in the above step S4, and the above processing is executed in steps S5 and S6. Thereafter, it is determined whether tpl> = N (n) in step S7, and if "n", the process in step S5 is repeated.
If "y", the above processing is executed in step S8. After execution of step S8, dictionary template n is stored in step S8.
In step 9, the input frame j is determined in step S10, and the processing of the word spot algorithm ends.

Next, the algorithm of the cumulative distance in the extended continuous DP will be described. Execute the following for all j (0 <= j <len obj) Execute the following for all states stat (0 <= stat <stat num) All n (0 <= n <word num ), Execute the following scr = min {frm scr [from stat] [aug ini [n] [j] + m-1] + ap
x scr [aug ini [n] [j] + m] Where min is m, fr
Shows the minimum value when only om stat is run. m is a value that moves in a predetermined range from APX MIN to APX MAX. (Ap
x: abbreviation for approximate) Also, from stat satisfies the expression (2) at the same time.

Apx scr [aug ini [n] [j] + m] is obtained as follows. apx scr [aug ini [n] [j] + m] = aug dist [n] [j] × (j- (aug i
ni [n] [j] + m) / (j-aug ini [n] [j]) This value is the word spot value at the beginning (aug ini [n] [j]),
By applying a coefficient proportional to the frame length, the beginning (aug i
The word spot value of ni [n] [j] + m) is approximately obtained. If frm scr [stat] [j]> scr, execute the following. frm scr [stat] [j] = scr, frm tpl [stat] [j] = n, frm frm [stat] [j] = aug ini [n] [j] + (m satisfying), frm stt [stat ] [j] = (satisfies from stat) Then, find the recognized word string by backtrace.

FIG. 9 is a flowchart of an algorithm for cumulatively calculating the extended continuous DP.
As in the case of FIG. 9, first, in order to execute the above processes in steps S1 to S3, the initial values of j, stat, n are set to “0”. Next, the above processing is performed in step S4, and in step S5, the above-described frmscr
If [stat] [j]> scr, after performing the above processing in step S6, the processing in steps S7 to S9 is performed and the processing ends.

[0020]

(1) Even if the word spot algorithm of the extended continuous DP is used as described above,
For example, when "good morning" is recognized, as shown in FIG. 10 (a), if all phoneme recognitions are successful, word spots can be accurately formed, but as shown in FIG. 10 (b). If an incorrectly recognized phoneme is included in the middle and the phoneme recognition is not performed successfully, only the underlined portions in the drawing are word spotted, and some of them cannot be recognized, resulting in missing. As described above, when an erroneously recognized phoneme is included in the middle, in a case where the inclination restriction path in FIG. 10B is used, there is a strong tendency to select a frame closer to the end as the optimum start. For this reason, there is a problem that the accuracy of the word spot is greatly reduced due to slight phoneme error recognition.

(2) The cumulative distance of the extended continuous DP is calculated by AP
The cumulative distance is calculated only in the range of X MIN <= m <= APX MAX.
Therefore, when the word spot does not fit well, for some stat, j, from scr [stat] [j]
However, there is a possibility that the initial value remains without being updated. If the value of from scr [stat num-1] [len obj-1] is left as the initial value, there is a problem that the recognition result is not obtained.

The present invention has been made in view of the above circumstances, and aims to improve the phoneme recognition rate by suppressing a decrease in accuracy of a word spot and by drastically reducing cases in which a phoneme recognition result cannot be output. It is an object to provide a phoneme recognition method in a voice recognition system.

[0023]

According to the present invention, in order to achieve the above object, a first invention is to provide a feature extraction unit to perform frequency analysis on voice data input to a voice input unit to obtain a spectrum sequence, The spectrum sequence is input to a phoneme recognition unit, and a phoneme sequence is obtained as an output. The phoneme sequence is supplied to a matching unit to be matched with a template in a dictionary. In a speech recognition system in which a cumulative distance of an input phoneme up to n frames is obtained, the most similar word or word string is output as a recognition result. The matching unit matches the input phoneme with a dictionary template, When setting the optimal starting point for the first phoneme of the dictionary template in the spot calculation, the optimal starting point is determined using the result up to two frames before. It is characterized in that so as to extend the range to obtain Ri.

The second invention uses an extended continuous DP matching process for the matching unit, and performs DP matching under specified conditions only for matching with the template first frame.
The feature is that the path is changed.

According to a third aspect of the present invention, the matching unit matches an input phoneme with a dictionary template, and when the cumulative distance calculation is completed for each input frame, checks the value of the cumulative distance. , An appropriate value is set.

[0026]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a flowchart showing a first embodiment of the present invention. In the first embodiment, the problem described in the section (1) of the “Problem to be Solved by the Invention” is that the optimum starting point is shifted from the end to the end. The path of the DP is changed by the word spot calculation unit so that the frame can be selected even from a distant frame, and the following processing is performed in FIG.

All input frames j (0 <= j <len obj)
(Step S1) For all dictionary templates n (0 <= n <word num), execute the following (Step S2) For all template phonemes tpl of n (0 <= tpl <N (n)) (Step S3) When tpl = 0 in Step S4, the following processing is executed in Step S5. dist tmp [tpl] [j] = value (tpl, j) ini tmp [tpl] [j] = j After processing in step S5, if j> = 2 (step S5)
11), the DP1 pass as shown in FIG. 2B is limited to the matching with the template first frame.
Allow under specified conditions. Then, dist tmp [tpl] [j-2] <OTHER VAL, which is the process of step S12.
(Maximum value of the distance between phonemes). If “y” in this process, the process of step S13 is performed. Dist tmp [tpl] [j] = dist tmp [tpl] [j] + dist tmp [tpl] [j-2] ini tmp [tpl] [j] = ini tmp [tpl] [j-2] is executed. FIG. 2 shows a process in the first frame of the template, and FIG. 2A shows a normal DP pass,
FIG. 2B shows an improved DP path used in the first embodiment.

On the other hand, the processing in step S12 is
st tmp [tpl] [j-2]> = OTHER VAL and step S14
Is dist tmp [tpl] [j-1] <OTHER VAL, the processing in the next step S15 is dist tmp [tpl] [j] = dist tmp [tpl] [j] + dist tmp [ tpl] [j-1] ini tmp [tpl] [j] = ini tmp [tpl] [j-1] is executed. After the processing in step S15 and step S13 is completed, step S7 shown in FIG.
Then, it is determined whether or not tpl> = N (n), and based on the result of the determination, the processing after step S8 is executed.

When tpl> = 1 in step S4, the process of step S6 described below is executed. dist tmp [tpl] [j] = min {dist tmp [tpl-1] [k]} + value (tpl, j) (j-2 <= k <= j) where k is min k Then ini tmp [tpl]
[j] = ini tmp [tpl-1] [min k]. Where value (tpl,
j) is the inter-phoneme distance between the phoneme of the tpl frame of the template n and the phoneme of the jth frame of the input frame. Also,
For example, when the score between phonemes is set to “0” and the score when they do not match is set to “1”, the frame-to-frame distance is calculated based on the j-th phoneme a (j) of the input frame and the template n. The distance between frames with the i-th phoneme b (i) is defined as follows. After the process in step S6, the determination process in step S7 is performed. If "y", the process in the next step S8 is performed. aug dist [n] [j] = dist tmp [N (n) -1] [j], aug ini [n] [j] = ini tmp [N (n) -1] [j] After execution of step S8 , The dictionary template n is determined in step S9, and the input frame j is determined in step S10, and the processing of the word spot algorithm ends.

FIG. 3 is a flow chart showing a second embodiment of the present invention. In the second embodiment, the problem described in the section (2) of the “problem to be solved by the invention” is the same as that of the first embodiment. Even if it is not done, it is avoided by substituting the most appropriate value. Therefore, in the second embodiment, in each frame, after the cumulative distance calculation is completed, if the cumulative distance is not updated, a process of setting a value is performed. Hereinafter, the algorithm of the cumulative distance calculation unit will be described in association with the flowchart of FIG. 3, and the same parts as those in FIG. 9 are denoted by the same reference numerals.

The following-is executed for all j (0 <= j <len obj) (step S1) The following-is executed for all states stat (0 <= stat <stat num) (step S2) The following − is executed for all dictionary templates n (0 <= n <word num) (step S3) scr = min {frm scr [from stat] [aug ini [n] [j] + m-1] + ap
x scr [aug ini [n] [j] + m] Where min is m, fr
Shows the minimum value when only om stat is run. m is a value that moves in a predetermined range from APX MIN to APX MAX. Also,
from stat satisfies the above expression (2) at the same time.

Apx scr [aug ini [n] [j] + m] is obtained as follows. apx scr [aug ini [n] [j] + m] = aug dist [n] [j] × (j- (aug i
ni [n] [j] + m) / (j-aug ini [n] [j]) This value is the word spot value at the beginning (aug ini [n] [j]),
By applying a coefficient proportional to the frame length, the beginning (aug i
The word spot value of ni [n] [j] + m) is approximately obtained. (Step S4) If frm scr [stat] [j]> scr, the following is executed.
(Step S5) frm scr [stat] [j] = scr, frm tpl [stat] [j] = n, frm frm [stat] [j] = aug ini [n] [j] + (m satisfying), frm stt [stat] [j] = (satisfies from stat) (step S6) After the execution processing in step S6, the dictionary template n
Is larger than word num in step S7. As a result of the determination, if “y”, the determination process of step S10 is performed, and if “n”, the process returns to step S4.

In step S10, the following determination processing is performed. When from scr [stat] [j] is the initial value MAX VAL and from scr [stat] [j-1] <MAX VAL-OTHER VAL,
That is, when "y", the following (step S11) is executed. Here, OTHER VAL is the maximum value of the inter-phoneme distance (the maximum value taken by value (a, b)).

From scr [stat] [j] = from scr [stat] [j-
1] + OTHER VAL, from frm [stat] [j] = from frm [stat] [j-1], from tpl [stat] [j] = from tpl [stat] [j-1], from stt [stat ] [j] = from stt [stat] [j-1] Note that when “n” in step S10,
After the execution of, after all states stat are determined in step S8, all input frames j are determined in step S9, and the process is terminated.

The following Table 1 shows the phoneme recognition rate when the first and second modes of processing are used together and the word recognition rate when the conventional extended continuous DP is used, obtained by experiments. . The experimental conditions are as follows: A: phoneme learning data is uttered twice by three speakers, and 101 words are uttered. B: Experimental data is 101 words of the three speakers (learning speakers) +6 speakers (evaluation speakers). With one utterance, the phoneme recognition unit used was learned in A.

[0036]

[Table 1]

In the above experiment, the DP path shown in FIG. 7B was used. From Table 1, it is clear that the phoneme recognition rate is significantly improved for both the learning speaker and the evaluation speaker.

[0038]

As described above, according to the present invention,
A reduction in the accuracy of word spots due to misrecognition of phonemes can be suppressed, thereby improving the phoneme recognition rate. In addition, the phoneme recognition rate can be improved by drastically reducing cases in which a phoneme recognition result cannot be output. And the like.

[Brief description of the drawings]

FIG. 1 is a flowchart showing a first embodiment of the present invention.

FIG. 2 is a characteristic diagram of a normal DP path and an improved DP path in processing in a template first frame.

FIG. 3 is a flowchart showing a second embodiment of the present invention.

FIG. 4 is a block diagram of a discrete word speech recognition system.

FIG. 5 is an explanatory diagram of a finite state automaton.

FIG. 6 is a flowchart illustrating an algorithm of a general accumulation calculating unit.

FIG. 7 is a characteristic diagram of a DP matching calculation path.

FIG. 8 is a flowchart of an extended continuous DP word spot algorithm.

FIG. 9 is a flowchart of a cumulative calculation algorithm of the extended continuous DP.

FIG. 10 is an explanatory diagram of an example including a misrecognized phoneme.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 11 ... Voice input device 12 ... Voice input part 13 ... Feature extraction part 14 ... Phoneme recognition part 15 ... Dictionary template 16 ... Matching part

Claims (3)

[Claims] 1. A feature extraction unit frequency-analyzes voice data input to a voice input unit to obtain a spectrum sequence, inputs the spectrum sequence to a phoneme recognition unit, obtains a phoneme sequence at its output, and obtains a phoneme sequence. When the sequence is supplied to the matching unit and matched with the template in the dictionary, after calculating the cumulative distance of the matching distance between the input phoneme and the dictionary template up to n frames of the input phoneme, the most similar word or word sequence is determined. In the speech recognition system configured to output as a recognition result, the input phoneme and the dictionary template are matched by the matching unit, and when the optimum starting point of the dictionary template in the first phoneme is set in the word spot calculation, the result up to two frames before is set. The speech recognition system is characterized by using Kicking phoneme recognition method. 2. The method according to claim 1, wherein the matching unit uses an extended continuous DP matching process to change a DP path only under matching with a template first frame under designated conditions. Item 1. A phoneme recognition method in the speech recognition system according to Item 1. 3. A speech sequence input to a speech input unit is subjected to frequency analysis by a feature extraction unit to obtain a spectrum sequence, the spectrum sequence is input to a phoneme recognition unit, and a phoneme sequence is obtained at an output thereof. When the sequence is supplied to the matching unit and matched with the template in the dictionary, after calculating the cumulative distance of the matching distance between the input phoneme and the dictionary template up to n frames of the input phoneme, the most similar word or word sequence is determined. In the speech recognition system configured to output as a recognition result, the input unit is matched with the dictionary template by the matching unit, and when the cumulative distance calculation is completed in each input frame, the value of the cumulative distance is checked, and the initial value is checked. A phoneme recognition method in a speech recognition system, wherein an appropriate value is set if it remains as it is.