JPH0336437B2 - - Google Patents

Description

[Detailed Description of the Invention] [Summary] In a continuous speech recognition device, a plurality of standard patterns are divided into first half and second half word standard patterns,
Continuous DP matching is performed on the first half word standard pattern of the speech input pattern, reverse time continuous DP matching is performed on the second half word standard pattern, and from the results, each frame of the input pattern is compared to the boundary of each half word standard pattern. Select multiple standard patterns such that DP processing is performed to minimize the sum of the partial interval matching distances for each candidate to obtain recognition results.

[Industrial application field]

The present invention is a continuous speech recognition device that recognizes speech in which multiple words are uttered continuously, and in particular recognizes continuous speech using dynamic programming (hereinafter referred to as DP), efficiently and with a high recognition rate. This paper relates to a continuous speech recognition device that has been improved to be able to perform continuous speech recognition.

[Conventional technology]

Various methods have been proposed for recognizing continuous speech formed by continuously uttering a plurality of words, and the following methods are typical.

The first method divides continuous words that form continuous speech into individual words using characteristics such as a drop in voice power at the boundaries of each word (this operation is called segmentation). This method recognizes each word by comparing it with a standard pattern.

The second method is a method in which all sets of sections of continuous words are extracted and compared with a standard pattern, and an optimal combination of sections is determined and recognized from the matching results.

[Problem that the invention seeks to solve]

Among the continuous speech recognition methods, the above-mentioned first method has problems in that it is very difficult to segment continuous words and it is difficult to ensure a high recognition rate.

On the other hand, the second method achieves a high recognition rate, but the amount of calculation is much larger than the first method.
There was a problem that continuous speech recognition with fast response time was not possible.

Furthermore, since the joined part of each word in continuous speech is easily deformed due to the influence of the words before and after it, it is difficult to specify its position accurately. For this reason, DP
An error occurs in the end candidate or the start end candidate for each frame of the audio input pattern found by matching,
There was a problem that the recognition rate decreased.

Therefore, in order to solve these problems, we divided the multiple standard patterns into half-word standard patterns for the first half and the second half, and performed reverse time continuous DP matching of the voice input pattern for the first half word standard patterns.
From the results, for each frame of the input pattern, select one standard pattern that makes that frame the boundary of each half-word standard pattern and use it as a center candidate, and move the given center candidate from the frame where that candidate is located to the starting point. Inverse time DP matching is performed in the direction, DP matching is performed in the terminal direction, and DP is calculated so that the sum is the minimum from the partial interval matching distance for each given candidate.
A continuous speech recognition device (hereinafter referred to as the original invention) that obtains recognition results without performing any processing has been proposed by the same applicant.

This original invention selects the center candidate by continuous DP matching, which can significantly reduce the amount of calculation, and also extracts standard pattern candidates for each word in the stable part (for example, the center part). As a result, candidates can be found correctly and a high recognition rate can be obtained.
However, since only one center candidate is selected, it is often the case that it deviates from the correct center candidate. Therefore, there is a problem that the recognition rate decreases. Note that since the content of the original invention and the content of the invention of the present application have many parts in common, they will be appropriately explained in connection with the embodiments of the present invention later in the description of the embodiments.

The present invention significantly improves processing efficiency by reducing the amount of calculations during continuous speech recognition processing, and also achieves a higher recognition rate than the original invention by correctly determining candidates for each frame of the speech input pattern. It is an object of the present invention to provide a continuous speech recognition device that can be obtained.

[Means for solving problems]

Measures taken by the present invention to solve the above-mentioned problems in conventional continuous speech recognition devices will be explained with reference to FIGS. 1 to 3.

FIG. 1 is a block diagram illustrating the configuration of the present invention.

In FIG. 1, 110 is a standard pattern dictionary section in which standard patterns of a plurality of spoken words are stored. When reading, each standard pattern is divided into a first half word standard pattern for the first half word and a second half word standard pattern for the second half word.

120 is a half-word continuous DP matching unit that performs matching (continuous
DP matching) is performed, and the input pattern is matched using inverse time continuous dynamic programming (inverse time continuous DP matching) for each second half word standard pattern among a plurality of standard patterns.

Reference numeral 130 denotes a center candidate selection unit which selects a plurality of standard patterns for each frame of the input pattern based on the results of the half-word continuous DP matching unit 120, and selects a plurality of standard patterns in which that frame is the boundary of each standard pattern of the first half word and the second half word. Candidate.

Reference numeral 140 is a word start/end level DP processing unit that performs matching by inverse time dynamic programming (inverse time DP matching) from the frame in which the center candidate is located in the direction toward the start end, and also performs matching (inverse time DP matching) by dynamic programming in the direction toward the end. DP matching) to find the subinterval matching distance.

Reference numeral 150 denotes a sentence level DP processing unit, which calculates the combination that minimizes the sum of the subsection matching distances for each digit of the input pattern based on the subsection matching distance inputted from the word start/end level DP processing unit 140. Based on the results, it is determined how many digits the input pattern contains, and a recognition result is obtained.

Note that "digit" expresses the positional relationship of each word, and each word of a continuous word is expressed as the first digit, second digit, . . . xth digit from the beginning.

[Effect]

The operation of FIG. 1 will be explained with reference to FIGS. 2 and 3.

Figure 2 is an explanatory diagram of continuous DP matching and inverse time continuous DP matching between the input pattern and each half-word standard pattern before and after, and examples of their matching routes. time dp
It is an explanatory diagram of collation.

In the case of continuous word speech, as mentioned above, the joined part of each word is easily deformed due to the influence of the adjacent words before and after it, so it is difficult to specify its position accurately, so DP An error occurs in the end or start end candidates for each frame of the input pattern found through matching, reducing the recognition rate.

Therefore, in the case of continuous word speech, the beginning and end of each word are easily deformed due to the influence of the words before and after it, but the center part is less affected by the words before and after it. Focusing on stability, the standard pattern candidates for each word in continuous speech are extracted from the stable portion of that word.

A plurality of standard patterns are stored in the standard pattern dictionary section 110, but when read out,
Divide each standard pattern into two half words, the first half and the second half.
The first half word standard pattern for the first half word and the second half word standard pattern for the second half word are read out.

In the method of dividing each standard pattern into half words, the first half and the second half, the central part is generally stable, so it is divided into two at the central part, but it is also possible to divide it into two at a stable part other than the central part.

As shown in FIG. 2, the half-word continuous DP matching unit 120 performs continuous DP matching on each of the first half word standard patterns of a plurality of standard patterns of the input pattern, and performs inverse time continuous DP matching on each second half word standard pattern. I do. Then, the continuous DP matching distance and the inverse time continuous DP matching distance, which are the matching results, are output.

Based on the continuous DP matching distance and the inverse time continuous DP matching distance input from the half-word continuous DP matching unit 120, the center candidate selection unit 130 converts each frame of the input pattern into half-word standards for the first half and the second half. A plurality of standard patterns are selected as pattern boundaries, and each is used as a center candidate. By selecting a plurality of center candidates, it is possible to obtain more accurate recognition results in subsequent processing than in the original invention.

For each center candidate selected by the center candidate selection section 130, the word start/end level DP processing section 140
As shown in Figure 3, from the frame where the center candidate is located (indicated by k), reverse time DP matching is performed in the direction of the start end, and DP matching is performed in the direction of the end, and the starting center matching distance and Find the center-end matching distance. Next, a partial section matching distance is determined based on both matching distances.

The sentence level DP processing unit 150 finds a combination that minimizes the sum of the subsection matching distances for each digit of the input pattern based on the subsection matching distances input from the word start/end level DP processing unit 140, and calculates the combination that minimizes the sum of the subsection matching distances for each digit of the input pattern. The recognition result is determined by determining how many digits the input pattern contains.

As described above, since the center candidate is found by continuous DP matching and subinterval candidates are selected, subinterval candidates are selected by extracting all subinterval combinations from the input pattern and comparing them with multiple standard patterns. Conventional method to seek (second method mentioned above)
The amount of calculation can be significantly reduced compared to . In addition, since standard pattern candidates for each word of continuous speech are extracted from stable parts of the word and a plurality of central candidates are selected, a higher recognition rate than the original invention can be obtained.

〔Example〕

An embodiment of the present invention will be described with reference to FIGS. 2 to 9.

Fig. 4 is an explanatory diagram of the configuration of an embodiment of the present invention, Fig. 5 is an explanatory diagram of DP verification with fixed start and end ends, Fig. 6 is an explanatory diagram of continuous DP verification, and Fig. 7 is an explanatory diagram of reverse time continuous DP verification. FIG. 8 is an explanatory diagram of the continuous DP calculation method, and FIG. 9 is an explanatory diagram of the center candidate selection operation.
2 and 3 are as already explained.

(A) Configuration of the embodiment In FIG. 4, the standard pattern dictionary section 11
0, half-word continuous DP matching section 120, central candidate selection section 130, word beginning/end level DP processing section 14
0 and the sentence level DP processing unit 150,
This is as explained in FIG.

In the half-word continuous DP matching unit 120, 12
1 is a continuous DP matching unit that performs continuous DP matching on each first half word standard pattern of a plurality of standard patterns to obtain a continuous DP matching distance (Gf (k, n)). 122 is backward time continuous
The DP matching section performs reverse time continuous DP matching on each of the second half word standard patterns of the input pattern to obtain a reverse time continuous DP matching distance (Gb (k, n)).

In the word start/end level DP processing unit 140,
Reference numeral 141 denotes a reverse time DP matching unit, which performs reverse time DP matching for each center candidate input from the center candidate selection unit 130 in the direction of the start end from the frame in which the center candidate is located, and calculates the start center matching distance (Gs
(k, l)). 142 is the DP verification section,
For each center candidate, DP matching is performed from the frame in which the center candidate is located toward the end to obtain a center-end matching distance (Ge (k, m)). 1
43 is a partial section candidate selection unit, which selects the start end center matching distance Gs (k, l) and the center end matching distance Ge.
(k, m) based on subinterval matching distance (D
(l, m)) and the corresponding standard pattern (Ns(l, m))
Find m)).

In the sentence level DP processing unit 150, 151
is the sentence level DP matching part, and the word start/end level DP
The standard pattern Ns(l, m) corresponding to the subsection matching distance D(l, m) input from the processing unit 140 is
m) to find the optimal combination of words by DP matching. A recognition result processing unit 152 determines how many digits the input pattern contains based on the result of the sentence level DP matching unit 151 to obtain a recognition result.

160 is a microphone; 170 is a voice feature extraction unit that extracts the features of the input continuous voice and converts it into an input pattern for verification; and 180 is an input pattern buffer in which this input pattern is stored.

(B) Operation of the embodiment The microphone 160 extracts continuous speech consisting of continuous words uttered by the user from the speech feature extraction unit 17.
Enter 0. The audio feature extraction unit 170 extracts the features of the input continuous audio, converts it into an input pattern for matching, and stores it in the input pattern buffer 18.
Store at 0.

The audio input pattern A at this time is expressed as follows.

A=a(1),a(2),a(3),...a(m)...a
( ) Here, a(m) is a parameter in frame m, and for example, when the frequency direction is divided into 16 channels, it is a vector quantity represented by a power spectrum value in each division. The last I is the end frame of input pattern A and means the word length of input pattern A. Note that a frame is a voice section (more accurately, a voice section cut out using a window function).

Similarly, the standard pattern of the standard pattern dictionary section 110 is expressed as follows, where B(n) is the standard pattern of word n.

B(n) = b(1, n), b(2, n)...b(j,
n)...b(J(n), n) Here, b(j, n) is a parameter of word n in frame j, and similarly to input pattern A, it is expressed by, for example, the power spectrum value of 16 channels. It is a vector quantity. J(n)
is the final frame of word n and represents the word length of standard pattern B(n).

In the present invention, half-word matching is performed using continuous DP matching and reverse time continuous DP matching for the first half word standard pattern and the second half word standard pattern of the input pattern. This process is called word center level DP.

(B-1) Operation of continuous DP matching unit 121 In normal DP matching, the start and end of the input pattern are fixed, and as shown in Figure 5, one optimal matching path passing through the start and end is given. Then, the matching distance of the route is determined.

On the other hand, continuous DP matching, as shown in FIG. 6, refers to DP matching that is free from the start and ends at any frame of the input pattern. As a result, the optimal corresponding section existing in the input pattern A is automatically extracted, and the continuous DP matching distance is determined. Based on this result, it can be easily determined whether the standard pattern B(n) is likely to exist in the input pattern A, and if so, which frame of the input pattern A should be the end.

(B-2) Operation of the reverse time continuous DP verification unit 122 Reverse time continuous DP performed by the reverse time continuous DP verification unit 122
Verification is performed using input pattern A as shown in Figure 7.
This method performs continuous DP matching in the reverse time direction. This makes it easy to determine whether the standard pattern B(n) is likely to exist in the input pattern A, and if so, which frame of the input pattern A should be the starting end.

There are various methods of continuous DP calculation performed in the above-mentioned continuous DP matching and inverse time continuous DP matching, depending on how to take the matching path, etc., but here we will use a symmetric type as shown in Figure 8. do.

In FIG. 8, if the distance between the vectors of the standard pattern B and the input pattern A on the grid point (i, j) is d(i, j), then the matching distance gf(i, j ) is required.

When j = 1, gf (i, j) = gf (i, 1) = d (i, 1) When i = 1, j > 1, gf (1, j) = d (1, j) + gf (1,j-
1) When j > 1, gf (i, j) = mind (i, j) + gf (i-1, j) 2d (i, j) + gf (i-1, j-1) d (i, j )+gf(i,j-1) This gf(i,j) is the one that minimizes the sum of distances from point (1,1) to point (i,j).

Reverse time continuous DP matching is a method in which the input pattern is reversed and continuous DP matching is performed in the reverse time direction.
The calculation method is the same as the calculation method used during continuous DP verification described above, so the explanation will be omitted.

Through this continuous DP matching and reverse time continuous DP matching, the first half section and the second half section of standard pattern B are respectively matched as shown in FIG.

Let Gf(k) be the continuous DP matching distance on k frames of the input pattern, and let Gb(k) be the inverse time continuous DP matching distance. Furthermore, this Gf(k) and Gb
(k) is Gf for N standard patterns.
(k, n) and Gb (k, n), and the continuous DP matching unit 121 and the inverse time continuous DP
Each is output from the matching unit 122.

(B-3) Operation of center candidate selection unit 130 The center candidate selection process in the center candidate selection unit 130 is performed as follows.

Gf (k, n) and Gb (k, n) input from the half-word continuous DP matching unit 140 are accumulated in the center candidate selection unit 130 as shown in the following formula, and the cumulative half-word continuous DP matching distance DC (k,
n).

DC (k, n) = Gf (k, n) + Gb (k, n) DC (k, n) is a short one depending on the individual word length J (n) of the standard pattern B (n) as it is. Since the value tends to become smaller, time normalization is performed by dividing the value of DC(k, n) by the sum of each pattern length of input pattern A and standard pattern B(n). This time normalized
If DC (k, n) is expressed as DR (k, n), then
DR(k,n) is expressed by the following formula.

DR (k, n) = DC (k, n) / 2J (n) The reason for using 2J (n) here is that if input pattern A is correctly matched with standard pattern B (n), the corresponding input This is because the pattern length is considered to take a value close to J(n). If you use some method to
Starting point L(k,n) for l,m of (k,n)
If the terminus M(k,n) is known, DR(k,n) is obtained by the following equation.

DR (k, n) = DC (k, n) / {J (n) + M (k, n) - L (k, n) + 1} This DR (k, n) is as shown in Figure 9. ,
A different distance pattern is shown for each n.

Select C (2 in the example in Figure 9) of these N distance patterns from the smallest value for each k, and set that n to NC (k, c) as a candidate centered on frame k. .

The original invention includes a half-word continuous DP matching unit 12.
The processing up to 0 is the same as in the present invention, but the ninth
DR the N distance patterns in the figure for each k
Select the smallest (k, n) value and
Candidate whose center is k frame (center candidate)
is set to NC(k). As described above, in the original invention, since there is only one center candidate, there is a possibility that a correct center candidate may be omitted from selection, resulting in an inconvenience that erroneous recognition may occur.

On the other hand, in the present invention, since a plurality of center candidates are selected, the possibility that a correct center candidate is not selected as in the original invention is reduced, and the recognition rate can be increased.

Note that if the value of front C is small, the recognition accuracy will decrease, and if it is large, the amount of calculation will increase, so the selection is made with both factors taken into consideration.

(B-4) DP matching section 142 and reverse time DP matching section 1
41 operations (word start/end level DP) The center candidate indicated by NC(k, c) is
The DP matching distance between the start and the end is determined by the DP matching by the matching unit 142 and the reverse time DP matching by the reverse time DP matching unit 141.

There are various calculation methods for the DP used here, depending on how to take matching paths, etc., but here, a symmetric type is used like the continuous DP described above. That is, when i=1, j=1: ge(1,1)=d(1,1) When i=1, j>1: ge(1,j)=d(1,j)+ge( 1,j-
1) When i>1, j=1: g(i,1)=d(i,1)+ge(i-1,
j) When i > 1, j > 1: ge (i, j) = mind (i, j) + ge (i-1, j) 2d (i, j) + ge (i-1, j-1) d (i, j)+ge (i, j-1) This ge (i, j) is the one that minimizes the sum of distances from the grid point (1, 1) to the (i, j) point.

The DP matching distance ge(i,
j) is found, but the one at the end of the standard pattern is
Let it be Ge(i).

In addition, the inverse time DP matching distance gs(i, j) simply inverts the pattern and matches it,
The calculation method is the same as that for the DP matching distance ge(i,j), so the explanation thereof will be omitted. Let Gs(i) be the starting edge of the standard pattern.

Through these DP matching and reverse time DP matching,
The matching distance in the direction of the end and start end of the word center candidate NC (k, c) is determined.

Figure 3 shows the reverse time from the frame where the center candidate is located (indicated by k) in the direction of the starting point.
This is an example in which DP verification is performed, DP verification is performed in the direction of the end, and the start frame l and end frame m for that frame are determined. Note that H(NC(k,c)) is the word length of the standard pattern of the central candidate.

The center-end matching distance obtained by the DP matching unit 142 through DP matching is Ge(k, c, m),
In addition, the starting end center matching distance obtained by the inverse time DP matching unit 141 by the inverse time DP matching is expressed as Gs(k,
c, l).

(B-5) Operation of subsection candidate selection unit 143 The subsection candidate selection unit 143 selects the input
The partial pattern length (m-l+
1) and the standard pattern length J(NC(k,c)) to time normalize it.The DP matching distance of this time normalized subinterval is DI(k,c,
l, m), it is determined by the following formula. That is, DI (k, c, l, m) = Ge (k, c, m) + Gs (k, c, l)/J (NC
(k, c))+m−l+1 Next, the partial section candidate selection unit 143 selects this
Select the minimum value of DI (k, c, l, m) for C and set it as DK (k, l, m), and set the corresponding standard pattern number to NK (k, l, m). That is, DK(k,l,m)= min c {DI(k,c,l,m)} NK(k,l,m) =NC[K,argmin{DI(k,c,l,m) }] Here, "argmin c" is an operator that has the function of selecting c that satisfies the minimization condition in { }.

Furthermore, the minimum value of k is selected and similarly set to D(l,m) and NS(l,m). That is, D(l,m)=min k {DK(k,l,m)} NS(l,m) =NK[argmin k {DK(k,l,m)}l,m] This D(l , m) and NS (l, m), the partial pattern (l-m) of the input pattern
The optimal matching distance (D(l, m)) and matching partner (NS(l, m)) are determined and output from the partial section candidate selection unit 143.

(B-6) Operation of the sentence level DP matching unit 151 The sentence level DP matching unit 151 uses the D input from the word start/end level DP processing unit 140.
Using (l, m) and NS (l, m), find the optimal combination of words by DP matching.

Here, each word in the continuous word is 1 from the beginning.
The positional relationship will be expressed using expressions such as digit, second digit, and so on.

(b) Setting the 1st digit The 1st digit is the distance of the partial section of m frames from the start frame of the input pattern, T(m,
l). That is, T (m, 1) = D (1, m) Also, set where the beginning of the standard word is in LT (m, 1). That is, LT (m, l) = 1 Here, D where there is no subinterval candidate
No processing is performed for (1, m), and NT
Set "0" to (m, 1) and the maximum value to T(m, 1).

(b) Setting the xth digit In the xth digit, the position of l is undetermined, so the next
Determined using the DP recurrence formula. That is, T(m,x) l<m= min l [D(l,m)+T(l,x-1)] NT(m,x) l<m=NS[ argmin l {D(l,m ) +T(l,x-1)},m] LT(m,x) l<m= argmin l [D(l,m) +T(l,x-1)] In this case too, D(l,m) If there is no subinterval candidate for and m, do not process,
Set NT(m,x) to "0", T(m,x) to the maximum value, and LT(m,x) to "1" (although an indefinite value is fine).

(B-7) Operation of recognition result processing unit 152 The recognition result processing unit 152 uses T(m,x) and NT sent by the sentence level DP matching unit 151.
T(m,x) in (m,x) and LT(m,x)
First, the number of digits X of the input word is found from the following equation.

X=argmin l [T(I,x)/x] where I is the terminal frame of the input pattern.

Next, the recognition result processing unit 152 obtains the recognition result of X digits from LT (m, x) and NT (m, x) using the following (a) and (b). Here, let NR(x) be the recognition result of x digits, and let L(x) be the starting frame of NR(x).

(b) Recognition result of X-th digit NR (x) = NT (I, X) L (x) = LT (I, X) (b) Recognition result of X-y-th digit NR (X-y) = NT [L(X-y+1),X
-y] L(X-1)=LT[L(X-y+1),X-
y] In the above manner, the recognition result NR(x) is
This can be determined with a significantly smaller amount of calculation and a higher recognition rate than conventional methods.

〔Effect of the invention〕

As explained above, according to the present invention, the following effects can be obtained.

(b) Since the central candidate is determined by continuous DP matching and the partial interval candidates are selected, the amount of calculation can be significantly reduced compared to the conventional method.

(b) Since candidates for the standard pattern of each word are extracted from the stable part of that word, candidates can be found correctly and a high recognition rate can be ensured.

(c) Since a plurality of center candidates are selected, the correct center candidate is not dropped from selection unlike in the original invention where there is only one center candidate, and the recognition rate can be improved.

[Brief explanation of drawings]

Fig. 1: An explanatory diagram of the configuration of the present invention; Fig. 2: An explanatory diagram of continuous DP matching of the first half word standard pattern, reverse time continuous DP matching of the second half word, and examples of their matching routes; Fig. 3: Word center candidate FIG. 4 is an explanatory diagram of the configuration of an embodiment of the present invention, FIG. 5 is an explanatory diagram of DP verification with fixed start and end ends, and FIG. An explanatory diagram of continuous DP matching, Fig. 7... An explanatory diagram of reverse time continuous DP matching, Fig. 8... An explanatory diagram of the continuous DP calculation method, Fig. 9
Figure: An explanatory diagram of the center candidate selection operation. 1 and 4, 110... standard pattern dictionary section, 120... half word continuous DP matching section, 1
30... Center candidate selection section, 140... Word start/end level DP processing section, 150... Sentence level DP processing section.

Claims (1)

[Claims] 1. A speech feature extraction unit converts speech in which a plurality of words are continuously uttered into an input pattern, compares the input pattern with a plurality of standard patterns, selects candidates for each section, and In a continuous speech recognition device that obtains recognition by finding the one with the minimum matching distance between intervals, (a) the input pattern is matched by continuous dynamic programming for each first-half word standard pattern among multiple standard patterns; , a half-word continuous DP matching section 120 that performs matching of the input pattern using inverse time continuous dynamic programming for each second-half word standard pattern among a plurality of standard patterns; (b) From the results of the half-word continuous DP matching section 120; (d) a center candidate selection unit 130 that selects a plurality of standard patterns in which each frame of the input pattern is used as a boundary between standard patterns of first-half words and second-half words, and selects each standard pattern as a center candidate; (d) the center candidate; From the frame where is located, matching is performed using inverse time dynamic programming in the direction of the start end, and matching is performed using dynamic programming in the direction of the end to find the subinterval matching distance at the word start/end level.
DP processing unit 140; (d) Find a combination that minimizes the sum of subinterval matching distances for each digit of the input pattern based on the subinterval matching distance, and determine how many digits the input pattern is from the result. A continuous speech recognition device comprising: a sentence level DP processing unit 150 that determines a recognition result.