JPS62144200A - Continuous voice recognition equipment - Google Patents

Abstract

(57) [Summary] This bulletin contains application data before electronic filing, so abstract data is not recorded.

Description

[Detailed Description of the Invention] [Summary] In a continuous speech recognition device, a plurality of standard 4t patterns are divided into first half and second half word standard patterns, and the speech input pattern is subjected to continuous DP matching with respect to the first half word standard patterns. , Reverse time series Vt D for the second half word mark single pattern
P matching is performed, and from the result, for each frame of the input pattern, a standard pattern is selected that makes that frame the boundary of each half-word standard pattern, and the center candidate given by this is selected from the frame in which that candidate is located. Perform reverse time DP verification in the direction of the start end, perform DP verification in the direction of the end,
A recognition result is obtained by performing DP processing to minimize the sum of partial section matching distances for each given candidate.

[Industrial application field]

2. The present invention provides a continuous speech recognition device that recognizes speech in which a plurality of words are continuously uttered, and in particular a continuous speech recognition device that recognizes speech made by continuously uttering a plurality of words.
The present invention relates to a continuous speech recognition device that has been improved to efficiently recognize and recognize continuous speech using c programming (hereinafter referred to as DP) with a high recognition rate.

[Conventional technology]

Various methods have been proposed for recognizing continuous speech formed by uttering a plurality of words in succession, and the following methods are representative.

The first method is to segment continuous words that form continuous speech into individual words using characteristics such as a drop in speech power at the boundaries of each word (this operation is called segmentation). This method recognizes each word by comparing it with a standard pattern.

The second method is a method of extracting all sets of intervals of continuous words, comparing them with a standard pattern, and finding and recognizing the optimal combination of intervals from the results of the comparison. [Problem to be solved by the invention] Continuous Among the voice recognition methods, the first method mentioned above is
There have been problems in that it is very difficult to segment continuous words and it is difficult to ensure a high recognition rate.

On the other hand, the second method has the problem that although a high recognition rate can be obtained, the amount of calculation is much larger than that of the first method, and continuous speech recognition with a fast response time is not possible.

Furthermore, since the joined part of each word in continuous speech is easily deformed due to the influence of the words before and after it, it is difficult to specify its position accurately. For this reason, there is a problem in that an error occurs in the end candidate or the start end candidate for each frame of the audio input pattern obtained by DP matching, which lowers the recognition rate.

The present invention reduces the amount of calculations during continuous speech recognition processing to significantly improve processing efficiency, and also ensures a high recognition rate by correctly determining candidates for each frame of the speech input pattern. The purpose is to provide a recognition device.

[Means for solving problems]

The means taken by the present invention to solve the above-mentioned problems in the conventional continuous speech recognition device will be explained with reference to FIG.

FIG. 1 is a block diagram illustrating the configuration of the present invention.

In FIG. 1, 110 is a standard pattern dictionary section in which standard patterns of a plurality of spoken words are stored. When reading, each standard pattern is divided into a first half word standard pattern for the first half word and a second half word standard pattern for the second half word.

Reference numeral 120 denotes a half-word continuous DP matching unit, which performs matching (continuous DP matching) using continuous dynamic programming for each first-half word standard pattern among a plurality of standard patterns with respect to the input pattern, and also matches the input pattern with a plurality of standard patterns. Verification using inverse time continuous dynamic programming (inverse time continuous DP matching) is performed for each standard pattern of the second half of the words.

130 is a central candidate selection unit, and a half-word continuous DP matching unit 12
From the result of 0, a standard pattern is found for each frame of the input pattern, with that frame as the boundary between the standard patterns of the first half word and the second half word, and a central candidate is selected from the standard pattern.

Reference numeral 140 denotes a word start/end level DP processing unit that performs matching by inverse time dynamic programming (inverse time DP matching) from the frame in which the center candidate is located toward the start end, and also performs matching (inverse time DP matching) by dynamic programming in the direction toward the end. DP matching) is performed to find the partial interval matching distance.

150 is a sentence level DP processing unit, which processes word start and end level DP.
Based on the partial interval matching distances input from the processing unit 140, a combination that minimizes the sum of the partial interval matching distances is determined for each digit of the input pattern, and based on the result, it is determined whether the input pattern is a diffraction input. Obtain recognition results.

Note that "digit" expresses the positional relationship of each word, and each word in a continuous word is placed in the first digit, second digit, etc. from the beginning.
Expressed in the X digit.

[For production]

The operation of FIG. 1 will be explained with reference to FIGS. 2 and 3.

Figure 2 is an explanatory diagram of continuous DP matching and inverse time continuous DP matching between the input pattern and standard patterns of front and rear words, and examples of their matching routes. It is an explanatory diagram of time DP verification.

In the case of continuous word speech, as mentioned above, the joined part of each word is easily deformed by the influence of the adjacent words before and after it, so it is difficult to specify its position accurately.
Therefore, an error occurs in the end or start end candidates for each frame of the input pattern obtained by DP matching, reducing the recognition rate.

Therefore, in the case of continuous word speech, the beginning and end of each word are easily deformed due to the influence of the words before and after it, but the center part is less affected by the words before and after it. Focusing on stability, the standard pattern candidates for each word in continuous speech are extracted from the stable portion of that word.

A plurality of standard patterns are stored in the standard pattern dictionary section 110. When reading out each standard pattern, each standard pattern is divided into two parts, the first half word and the second half word. The second half of a half word is divided into standard patterns and read out.

The method of dividing each standard pattern into the first and second half words is as follows:
Generally, the center part is stable, so it is divided into two parts at the center part, but it is also possible to divide it into two parts at a stable part other than the center part.

As shown in FIG. 2, the half-word continuous DP matching unit 120 performs continuous DP matching on each first-half word standard pattern of a plurality of standard patterns of the input pattern, and performs inverse time continuous DP matching on each second-half word standard pattern. I do. Then, each matching result is the continuous DP matching distance and the inverse time continuous D
Output the P matching distance.

The central candidate selection unit 130 includes the half-word continuous DP matching unit 120
Based on the continuous DP matching distance and the inverse time continuous HDp matching distance input from Select the center candidate from the pattern.

As shown in FIG. 3, for each center candidate selected by the center candidate selection section 130, the word start/end level DP processing unit 140 calculates the word start/end level DP from the frame in which the center candidate is located (indicated by k) in the direction of the start end. Inverse time DP verification is performed in the direction of the end, and DP verification is performed in the direction of the end to determine the start-end center verification distance and center-end verification distance, respectively. Next, a partial section matching distance is determined based on both matching distances.

Sentence level DP processing section 150 finds a combination that minimizes the sum of subsection matching distances for each digit of the input pattern based on the subsection matching distances input from word start/end level DP processing section 140, and calculates the combination that minimizes the sum of subsection matching distances for each digit of the input pattern. A recognition result is obtained by determining whether the input pattern is a diffraction input.

As described above, since the center candidate is found by continuous DP matching and subinterval candidates are selected, subinterval candidates are selected by extracting all subinterval combinations from the input pattern and comparing them with multiple standard patterns. The desired conventional method (
The amount of calculation can be significantly reduced compared to the above-mentioned second method).

Furthermore, since the standard pattern candidates for each word of continuous speech are extracted from the stable portion of that word, the candidates can be found correctly and a high recognition rate can be ensured.

〔Example〕

An embodiment of the present invention will be described with reference to FIGS. 2 to 9.

Fig. 4 is an explanatory diagram of the configuration of an embodiment of the present invention, Fig. 5 is an explanatory diagram of DP verification with fixed start and end ends, Fig. 6 is an explanatory diagram of continuous DP verification, and Fig. 7 is an explanatory diagram of reverse time continuous DP verification. FIG. 8 is an explanatory diagram of the continuous DP calculation method, and FIG. 9 is an explanatory diagram of the center candidate selection operation.

2 and 3 are as already explained.

(A) Configuration of the embodiment In FIG. 4, a standard pattern dictionary section 110, a half-word continuous DP matching section 120, a central candidate selection section 130, a word start/end level DP processing section 140, and a sentence level DP processing section 15 are shown.
0 is as explained in FIG.

In the half-word series fiDP matching unit 120, reference numeral 121 is a continuous DP matching unit that performs continuous DP matching on each of the first half word standard patterns of a plurality of standard patterns for the input pattern, and calculates the continuous DP matching distance (Gf (k, n) ). Reference numeral 122 is a reverse time continuous DP matching unit, which performs reverse time continuous DP matching on each second half word standard pattern of a plurality of standard patterns of the input pattern to obtain a reverse time continuous DP matching distance (Gb (k, n)). .

In the word start/end level DP processing unit 140, reference numeral 141 is a reverse time DP matching unit that performs reverse time DP matching for each center candidate input from the center candidate selection unit 130 from the frame in which the center candidate is located in the direction of the start end. to find the starting end center matching distance (Gs(k, It)). 142 is DP
The matching unit performs DP matching for each center candidate in the direction of the end from the frame in which the center candidate is located to obtain a center-end matching distance (Ge (k, m)).

Reference numeral 143 denotes a partial section candidate selection unit, which selects the start end center matching distance G s (k y It ) and the center end matching distance Q
Based on e(k, m), subinterval matching distance (D(It,
m)) and the corresponding standard pattern (Ns(ff, m>
).

In the sentence level DP processing m15o, 151 is a sentence level DP matching unit that calculates a standard pattern Ns (e, m) corresponding to the subsection matching distance D (l, m) inputted from the word start/end level DP processing unit 140. The optimal combination of words is determined by DP matching. A recognition result processing unit 152 determines whether the input pattern is a diffraction input based on the result of the sentence level DP matching unit 151 and obtains a recognition result.

160 is a microphone; 170 is a voice feature extraction unit that extracts the features of the input continuous voice and converts it into an input pattern for comparison; and 180 is an input pattern buffer in which this input pattern is stored.

<B) The operational microphone 160 of the embodiment inputs continuous speech consisting of continuous words uttered by the user to the speech feature extraction unit 170. The audio feature extraction unit 170 extracts the features of the input continuous audio, converts it into an input pattern for verification, and stores it in the input pattern buffer 180.

The audio input pattern A at this time is expressed as follows.

A= a(11, a(2), a(31...a(
m)...a (1).

Here, a (m) is a parameter in frame m, and for example, when the frequency direction is divided into 16 channels, it is a vector quantity represented by a power spectrum value in each division. The last I is the end frame of input pattern A and means the word length of input pattern A. Note that a frame is a voice section (more accurately, a voice section cut out by a window function).

Similarly, the standard pattern of the standard pattern dictionary section 110 is
The standard pattern of word n is expressed as B (nl) as follows.

B(n)=b(1,n), b(2,n)・b(j,n
)-b(J(n), n) where b(j, n) is the parameter of word n in frame j and input pattern A
Similarly, it is a vector quantity expressed by, for example, power spectrum values of 16 channels. J (nl is word n
and represents the word length of the standard pattern B(n).

In the present invention, the input pattern is converted into continuous DP for the first half word standard pattern and the second half word standard pattern of the standard pattern.
Half-word matching is performed by matching and reverse time continuous DP matching.

This process is called word center level DP.

(B-1) Operation of continuous DP matching unit 121 In normal DP matching, the start and end of the input pattern are fixed, and as shown in FIG. 5, one optimal matching path passing through the start and end is given. Then, the matching distance of the route is determined.

Continuous DP matching, on the other hand, refers to P matching where the starting end is free and the end is any frame of the input pattern, as shown in FIG. As a result, the optimal corresponding section existing in the input pattern A is automatically extracted, and the series [Dp matching distance is determined. Based on this result, it can be easily determined whether the standard pattern B (n) is likely to exist in the input pattern A, and if so, which frame of the input pattern A should be the end.

(B-2) Operation of the reverse time continuous DP verification unit 122 The reverse time continuous DP verification performed by the reverse time continuous DP verification unit 122 is performed by the seventh
As shown in the figure, the input pattern A is continuously DP in the reverse time direction.
This is for verification. This makes it easy to determine whether the standard pattern B (n) is likely to exist in the input pattern A, and if so, which frame of the input pattern A should be the starting point.

There are various methods of continuous DP calculation performed in the above-mentioned continuous DP matching and inverse time continuous DP matching depending on how to take matching paths, etc., but here, a symmetric type as shown in FIG. 8 is used.

In Figure 8, standard pattern B on grid point (1, J)
Let d(i, j) be the distance between the vectors of input pattern A and input pattern A, then the matching distance gf (i, j) can be found using the following recurrence formula.

When j=1, g f (xyj) = g f (i・1) = d (i・
1) When i=1, j>1, gf(1,j)=d(1・j) 10g' (19J-
1) When j>1, this gf(iyj) is the one that minimizes the sum of distances from point (1, 1) to point (i, j).

Reverse time continuous DP matching is performed by reversing the input pattern and performing continuous DP matching in the reverse time direction, and its calculation method is the same as the calculation method for continuous DP matching described above, so a description thereof will be omitted.

Through this continuous DP matching and reverse time continuous DP matching, the first half section and the second half section of the standard pattern B are respectively matched as shown in FIG.

Let G f (k) be the continuous DP matching distance of the input pattern on the frame, and G b (k) be the inverse time continuous DP matching distance. Furthermore, G f (k) and G b (kl are G for N standard patterns.
They are expressed in the form of f(k, n) and Gb(k, n), and are output from the continuous DP matching section 121 and the inverse time continuous DP matching section 122, respectively.

(B-3) The center candidate selection process in the motion center candidate selection section 130 of the center candidate selection section 130 is performed as follows.

Gf(k,
n) and Gb(k, n) are accumulated in the center candidate selection unit 130 as shown in the following equation, and the cumulative half-word consecutive DP matching distance DC(k) is accumulated.

n).

D C (k, n) = G r (k, n)
+ G b (k, n) DC (k, n) is the individual word length J (n
), the shorter the value, the smaller the value.
Time normalization is performed by dividing the value of DC (k, n) by the sum of each pattern length of input pattern A and standard pattern B (n).

This time-normalized DC(k, n) is converted to DR(k,
n), DR(k, n) is expressed by the following formula.

DR (k, n) - DC (k, n)/2 J (n) The reason for using 2 J (n) here is that input pattern A is a standard pattern! 3fn), the corresponding input pattern length is considered to take a value close to J(n). If the starting point L(k, n) and ending point M (k, n) for l, m of DC(k, n) are known by some method, DR
(k, n) is determined by the following formula.

D R(k, n) = D C(k, n)/(J
(n)+M(k,n) L(k,n)+1) This DR
As shown in FIG. 9, (k, n) shows a different distance pattern for each n.

For each of these N distance patterns, select the one with the smallest value, and as shown in the lower part of Figure 9, select n as a candidate centered on the frame (center candidate) and convert it into N C (k). Set. That is, N C(kl= arg man (D R
(k, n)) For example, in FIG. 9, in ki frames, DR(k, 2) of n=2 is the minimum, so n=
The standard pattern of No. 2 is set in NC(2) as a candidate centered on the ki frame (center candidate).

Through the above processing, the number of frames Bfn) in which each input pattern A is centered is limited to -.

Since the one with the smallest DR value is considered to be the most suitable candidate, by limiting the number of center candidates for each frame to one (NC(g)), it is possible to correctly select the center candidate for each frame. In addition, the amount of calculation can be further reduced.

(B-4) DP matching section 142 and reverse time DP matching section 14
Operation 1 (word start/end level DP) The center candidate indicated by NC(k) is subjected to DP matching by the DP matching unit 142 and reverse time DP matching by the inverse time DP matching unit 141 to calculate the DP matching distance of the start and end. demand.

There are various methods for calculating the DP used here depending on how to take matching paths, etc., but here, a symmetric type is used like the continuous DP described above. That is, when i=l, j=1: ge (1, 1) = d (1, 1)i=l, h
>l: ge(Lj)=d(1,j)+ge(Lj-1)i
>1, when j=1; g(i,1)=d(i,1)+g e(i4,1)s
> l t j > 1: This ge(i, j) is the transition from the grid point (1, 1) to (i,
j) Minimize the sum of distances to the point.

DP matching distance ge(i.

j) is found, but the one at the end of the standard pattern is Ge (i
Let it be f.

In addition, the inverse time DP matching distance gs (i, j) is to match by inverting the pattern, and its calculation method is DP
Since it is the same as the matching distance gc(t+J), its explanation will be omitted. Let Gs(1) be the starting end of the standard pattern.

Through these DP matching and inverse time DP matching, matching distances in the direction of the end and start end of the word center candidate NC(k) are determined.

In Figure 3, reverse time DP matching is performed from the frame where the center candidate is located (indicated by k) in the direction of the start end, DP matching is performed in the direction of the end, and the starting end frame l is compared to that frame. This shows an example of finding the final frame. Note that H(NC(kl)) is the word length of the standard pattern of the center candidate.

The center-end matching distance obtained by the DP matching unit 142 by DP matching is Ge(k, m), and the inverse time DP matching unit 1
The starting end center matching distance obtained by 41 by inverse time DP matching is defined as Qs(k, I).

(B-5) Operation of the partial section candidate selection section 143 The partial section candidate selection section 143 selects the input Ge(k, m) and Gs
The partial pattern length (m-6
+l) and the standard pattern length J (NC(k)) for time normalization. D of this time-normalized subinterval
Letting P matching distance be DI (k, j2.m), it can be determined by the following equation. That is, next, the partial section candidate selection unit 143 selects this DI(k, j!
, m) with respect to k and D(It, m
) and set the corresponding standard pattern to NS(j2.m).

D(j!, m)=min(DI(k,A,m)1N
S (1, m) = NC(argmin (DI(k,
12, m)) where “argmin” is ()
This is an operator that has the function of selecting k that satisfies the minimum Hojo condition within.

If m has no termination candidate, N5CI.

m) is set to "0", and D(#, m) is set to the maximum value.

Based on this D (' + m) and N S (19m), the optimal matching distance (D (l, m)) and matching partner (NS (1)
, m)) are determined and output from the partial section candidate selection unit 143.

(B-6) Action sentence level D of sentence level DP matching unit 151
The P matching unit 151 uses the word start/end level DP processing unit 140
D(β.

m) and NS (j2.m), find the optimal word combination by DP matching.

Here, the positional relationship will be expressed by expressing each word in the continuous words as the first digit, second digit, . . . X digit from the beginning.

(a) Setting of 1st digit The 1st digit sets the cline of a subsection of m frames from the start frame of the input pattern to T(m, 1). That is, T (m, 1) = D (1, m) Also, where the beginning of the standard language is, LT (m, 1
). That is, LT (m, 1) = 1 Here, no processing is performed for D (1°m) for which there is no subsection candidate, and NT (m, 1) is set to "0", and T (
m, 1) to the maximum value (.

(b) Setting the X digit In the X digit, the position of l is undetermined, so it is determined using the following DP recurrence formula. In other words, + T (It y of,
The maximum value is set for T(m, x), and "l" is set for LT(m, x) (although an undefined value may be used).

(B-7) The motion recognition result processing unit 152 of the recognition result processing unit 152 receives the T
(m, x), NT (m, x) and LT (m, x)
First, the number of digits X of the input word is found from T(m, x) in the following equation.

X = argmin (T(1, x)/X) where,
■ is the final frame □- of the input pattern.

Next, the recognition result processing unit 152 converts the recognition result of X digits into LT
(m, x) and NT (m, x) to the following (a) and (b)
Find it by Here, the recognition result of X digits is N R(X
), and the starting frame of N R (Xl is L (X)).

(b) Recognition result of X-th digit NR (X) = NT (1, X) IjX1 = LT (I, -y+1), X-y
)L(X-1) =LT CL(X-y+1),X
-y) In the above manner, the recognition result N R(X) can be obtained with a significantly smaller amount of calculation than conventional methods and with a high recognition rate.

For example, in the case of the amount of calculation, in the conventional method, subinterval candidates are obtained by extracting all combinations of subintervals from the input pattern and comparing them with multiple standard patterns. , becomes as follows. now,
Input pattern length is 200 frames, standard pattern length is 5
Assuming that the window width for DP matching of 0 frame and subinterval is 20 frames and the number of standard patterns is 100, the number of operations of the DP recurrence formula is 200X 100X50X20 = 20.0000,0
00 times.

On the other hand, in the method of selecting center candidates according to the present invention, the result is 200X50X 100+200X50X20 under the same conditions. Therefore, the amount of calculation is only about 1/17, and a significant reduction in the amount of calculation can be achieved.

〔Effect of the invention〕

As explained above, according to the present invention, the following effects can be obtained.

(a) Since the center candidate is determined by continuous DP matching and the partial interval candidates are selected, the amount of calculation can be significantly reduced compared to the conventional method.

(b) Since candidates for the standard pattern of each word are extracted from the stable part of that word, candidates can be found correctly and a high recognition rate can be ensured.

[Brief explanation of drawings]

Figure 1: A detailed explanatory diagram of the present invention. Figure 2: An explanatory diagram of continuous DP matching of the first half word standard pattern, reverse time continuous DP matching of the second half word, and examples of their matching routes. ...Explanatory diagram of DP matching and reverse time matching in the direction of the end and beginning of a word center candidate, FIG. 4...Explanatory diagram of the configuration of an embodiment of the present invention, FIG. An explanatory diagram of DP verification, Fig. 6... An explanatory diagram of continuous DP verification, Fig. 7... An explanatory diagram of reverse time continuous DP verification, Fig. 8...
・Explanatory diagram of continuous DP calculation method, FIG. 9...Explanatory diagram of center candidate selection operation. In FIG. 1 and FIG. 4, 110... Standard pattern dictionary section, 120... Half word continuous DP matching section, 130... Center candidate selection section, 140
. . . Word start/end level DP processing section, 150 . . . Sentence bell DP processing section.

Claims (1)

[Claims] Speech in which a plurality of words are continuously uttered is converted into an input pattern in a speech feature extraction section, and the input pattern is compared with a plurality of standard patterns to select candidates for each section. In a continuous speech recognition device that obtains recognition by finding the one with the minimum matching distance, (a) the input pattern is matched by continuous dynamic programming for each first-half word standard pattern among a plurality of standard patterns; Half-word continuous DP in which input patterns are matched using inverse time continuous dynamic programming for each second-half word standard pattern among multiple standard patterns.
(b) From the results of the half-word continuous DP matching unit (120), a standard pattern is determined for each frame of the input pattern, with that frame as the boundary of each standard pattern of the first half word and the second half word; a center candidate selection unit (130) that selects a center candidate from the standard pattern; (d) a word start/end level DP processing unit (140) that performs matching according to the method and calculates a subinterval matching distance; A sentence level DP processing unit (150
), a continuous speech recognition device comprising: