JPH024919B2 - - Google Patents

Description

[Detailed description of the invention] In the speech recognition method, first, the phoneme is recognized,
After that, the obtained phoneme string and each word in the word dictionary expressed by the phoneme string are matched at the phoneme level based on the degree of similarity between phonemes, and the word that gives the greatest degree of similarity is taken as the recognition result. There is something. In this case, in order to recognize phonemes, a standard pattern of phonemes is registered in advance, and the input audio signal is compared with this standard pattern to perform phoneme recognition. The present invention relates to a method for determining such a standard pattern of phonemes. An example of a conventional speech recognition device will be described below with reference to the drawings.

FIG. 1 is a block diagram showing the configuration of a speech recognition device based on the above-mentioned phoneme recognition. Reference numeral 1 denotes an audio signal input terminal, into which an audio signal converted into an electrical signal from a microphone (not shown) or the like is input. 2
is a feature extraction unit that converts the input audio signal into a time series of feature parameters. For example, if a filter bank has 20 channels, the input audio signal represents the output magnitude of each of the 20 band filters arranged on the frequency axis.
Converted to a time series of 20 sets of numbers (20-dimensional vectors). 3 is a phoneme standard pattern holding unit in which each phoneme to be recognized is held as a 20-dimensional vector. This 20-dimensional vector is extracted for each phoneme by the feature extraction section 2 in the same manner as described above, and is prepared in advance. 4 is a phoneme recognition unit that compares each phoneme in the phoneme standard pattern holding unit 3 with the feature vector input via the switch 11 at regular intervals, and recognizes the phoneme standard pattern with the highest degree of similarity, that is, the closest distance. Output as result. A word dictionary 5 stores words to be recognized as representations of phoneme sequences. 6 is the word recognition section,
The output phoneme string of the phoneme recognition unit 4 and each word of the word dictionary 5 are compared, and the one with the highest degree of similarity is outputted to the terminal 7 as a recognized word. In this case, the degree of similarity between the input phoneme string and each word in the word dictionary is calculated based on the degree of similarity between phonemes. This degree of similarity between phonemes is determined experimentally in advance. For example, in a 20-dimensional vector space, the point represented by the average vector of vectors representing phoneme X and the average vector of vectors representing phoneme Y from a large number of data. The similarity between phoneme X and phoneme Y is obtained by linearly transforming the distance between the points represented by the vectors. Note that 8 is a phoneme standard pattern creation unit, and a switch 11 is configured so that the output of the feature extraction unit 2 is input to the phoneme pattern creation unit 8 when creating a phoneme pattern, and input to the phoneme recognition unit 4 during recognition. works.

In the above-described configuration, conventional phoneme standard patterns have been created by simply averaging a large number of vectors representing the same phoneme. That is,
Even if the phoneme change the speaker,
The average of many vectors for the phoneme X obtained by changing the phonemes that follow before and after is taken. Expressed mathematically, the k-dimensional vectors for phoneme X are 〓 ₁ , 〓 ₂ , ..., 〓 _o ; 〓 _i = (x _i1 , x _i2 , ...
_{x ik} ⁾ _, ^{then the} _{standard pattern vector of} phoneme /n,..., ( _o 〓 ⁱ⁼¹ x _ik )/n) is given.

In FIG. 2, the horizontal axis represents the channel number of the band filter constituting the filter bank, and the vertical axis represents the output intensity of the phoneme. A, B
are spectra for the phoneme X uttered by different speakers, and C is its average spectrum. As is clear from the figure, in each of A and B, the two peaks are noticeable when taken alone, but in C, these features are considerably diminished. This is A
This is because the positions of the peaks of and B with respect to the frequency axis are different, and the resonance frequencies are different mainly due to the difference in the vocal tract lengths of the speakers. According to the knowledge of phonetics, the appearance of this mountain (formant) is closely related to the type of phoneme, and it carries extremely important information in discriminating phonemes. Therefore, it is not desirable that characteristics such as C are diminished by the averaging operation. The above explanation was only about the average of the two, but if more spectra are averaged as described above, there is a risk that the characteristics will further decrease.

The present invention provides a speech recognition device that eliminates the above-mentioned drawbacks and obtains appropriate standard patterns.

The basic idea is to bring pattern A closer to pattern B by nonlinear expansion and contraction of the frequency axis, and then average it. Next, a method of expanding and contracting the frequency axis in an embodiment of the present invention will be described.

FIG. 3 is a so-called lattice graph, in which the horizontal axis is the frequency axis of pattern B, and the vertical axis is the frequency axis of pattern A. 13 and 14 or 13 and 15
The area surrounded by is a matching window, and a route can only be selected within this window. Note that (A) is a case in which the frequency axis of pattern A is shifted to the higher frequency side with respect to pattern B, and (B) is a case in which the frequency axis is shifted to the lower frequency side, and in both cases it is assumed that the frequency axis can be shifted by one channel.

FIG. 4 shows the conditions for route selection, where i is the coordinate corresponding to the channel of pattern B,
j indicates the coordinates corresponding to the channel of pattern A, indicating a possible route to point (i, j). However, the numbers 1 and 2 above each line segment are examples of the weighting coefficient k that is multiplied when that route is selected. In other words, when only the three route choices shown in Figure 4 are given, (i-1,
The weighting factor k when selecting a route from j) to (i, j) is k=1, but from (i-1, j-1) to (i,
The weighting coefficient k when selecting the route to j) is k=2
However, when selecting a route from (i, j-1) to (i, j), the weighting coefficient k is assumed to be k=1. Similarly, if only the three route choices shown in Figure 4 are given, (i-1,
j), (i-1, j-1) and (i-1, j-2)
Loading coefficient k when selecting route from to (i, j)
Assume that k=1 is imposed on all cases.

Now, the vector of pattern A using the three route selections and the weighting coefficient k shown in Figure 4 is a.
= (a ₁ , a ₂ , ..., a _I ), the vector of pattern B is
When = (b ₁ , b ₂ , ..., b _I ) (where I is the number of channels) and d (i, j) = |b _i −a _j |, well-known dynamic programming is applied. In other words, by solving the following recurrence formula, the distance between pattern A and pattern B can be defined.

Below, we will divide the frequency axis of Pattern A shown in Figure 3 into two cases: one in which the frequency axis of Pattern A is shifted to the higher frequency side relative to Pattern B, and the other in which the frequency axis of Pattern A is shifted to the lower frequency side relative to Pattern B. A detailed explanation will be given below.

(b) When the frequency axis of pattern A is shifted to the higher frequency side relative to pattern B, and when the route selection conditions are applied as shown in Figure 4.

Assuming the initial value g(1,1)=d(1,1), g(i,i)=ming(i,i-1)+d(i
, i) g (i-1, i-1) + kd (i, i) g (i, i-1) = ming (i-1, i-2)
+ kd (i, i-1) g (i-1, i-1) + d (i, i-1) (k = 2) _. ,B)=g(I,I)/2I is the distance between pattern A and pattern B. Furthermore, if i, j≦0, g(i, j)=∝, and g(i, i)=ming(i, i-1)+d(i
, i) g(i-1,i-1)+kd(i,i) and g(i,i)=ming(i-1,i-2)+kd
(i, i-1) g(i-1, i-1)+d(i, i-1) In the formula, min shall be the smaller value in [ ]. Also, since I is constant, it can be defined as D _H (A, B) = g (I, I).

(b) When the frequency axis of pattern A is shifted to the lower frequency side relative to pattern B, and when the route selection conditions shown in Figure 4 are applied.

Assuming the initial value g(1,1)=d(1,1), g(i,i)=ming(i-1,i)+d(i
, i) g(i-1,i-1)+kd(i,i) g(i-1,i)=ming(i-2,i-1)k
d (i-1, i) g (i _- 1, i-1) + d (i-1, i) (k = 2). ,B)=g(I,I)/2I is the distance between pattern A and pattern B.If i, j≦0, then g(i,j)=∝, and g(i,i)= ming(i, i-1)+d(i
, i) g(i-1,i-1)+kd(i,i) and g(i,i)=ming(i-1,i-2)+kd
(i, i-1) g(i-1, i-1)+d(i, i-1) In the formula, min shall be the smaller value in [ ]. Also, since I is constant, it can be defined as D _L (A, B) = g (I, I). As described above, the distance D _H in case A in Figure 3
(A, B) and distance D _L (A, B) in case of Figure 3 B
In the process of finding the optimal routes, the routes shown in FIG. 5A and FIG. 5B are found, respectively. That is, if the starting point is (I, I) and the above recurrence formula is traced in reverse according to each case, the route becomes clear.

Now, using the above recurrence formula, calculate the distances D _H (A, B) and D _L of patterns A and B in the case of I=20.
(A, B) respectively, D _H (A, B) = 146 D _L (A, B) = 187, D _H < D _L , and as is clear from Figure 5 A and B, the 5th The method shown in Figure 3B, in which the frequency axis of Pattern A is shifted to the lower frequency side relative to Pattern B, is better than the method shown in Figure 3, in which the frequency axis of Pattern A is shifted to the higher frequency side relative to Pattern B. desirable.

In Figure 6, A' moves the frequency axis of A to pattern B according to the route determined above.
The spectrum obtained by shifting to the lower frequency side is
C' indicates the average of spectrum A' and spectrum B. However, for example, when channel 3 of pattern B corresponds to channel 2 and channel 3 of channel A at the same time, in this case, channel 3 of pattern B and channel 2 and channel 3 of pattern A The spectral intensity is averaged. Now, the spectrum C' obtained in this way preserves the spectral characteristics well.

On the other hand, using the three route selections and load coefficient k shown in Figure 4 as route selection conditions,
The distance between pattern A and pattern B can also be defined by solving the following recurrence formula.

The case where the frequency axis of pattern A is shifted to the higher frequency side with respect to pattern B, as shown in Fig. 3 below,
A specific explanation will be given separately for the case where the frequency axis of pattern A is shifted to the lower frequency side with respect to pattern B.

(b) When the frequency axis of pattern A is shifted to the higher frequency side relative to pattern B.

As the initial value g (1, 1) = d (1, 1), g (i, i) = ming (i-1, i-1) + d
(i, i) g(i-1, i-2) + d(i, i) g(i, i-1) = ming(i-1, i-1)
Find g(i, j) sequentially from d(i, i-1) g(i-1, i-2) + d(i, i-1), and D _H (A, B)=g(I , I)/2I is the distance between pattern A and pattern B. Note that if i, j≦0, g(i, j)=∝, and g(i, i)=ming(i-1, i-1)+d
(i, i) g(i-1, i-2) + d(i, i) and g(i, i-1) = ming(i-1, i-1)
+d(i, i-1) g(i-1, i-2)+d(i, i-1) In the formula, min shall be the smaller value in [ ]. Also, since I is constant, it can be defined as D _H (A, B) = g (I, I).

(b) When the frequency axis of pattern A is shifted to the lower frequency side relative to pattern B.

As the initial value g (1, 1) = d (1, 1), g (i, i) = ming (i-1, i-1) + d
(i, i) g(i-1, i) + d(i, i) g(i-1, i) = ming(i-2, i-1)
+d (i-1, i) g (i-2, i-2) + d (i-1, i) Find g (i, j) sequentially, and D _L (A, B) = g (I , I)/2I is the distance between pattern A and pattern B. Note that if i, j≦0, g(i, j)=∝, and g(i, i)=ming(i-1, i-1)+d
(i, i) g(i-1, i) + d(i, i) g(i-1, i) = ming(i-2, i-1)
+d(i-1,i) g(i-2,i-2)+d(i-1,i) In the formula, min shall be the smaller value in [ ]. Also, since I is constant, it can be defined as D _L (A, B) = g (I, I).

As described above, the distance D _H in case A in Figure 3 is
(A, B) and the distance D _L (A,
When calculating B), D _H > D _L. Therefore, as shown in FIG. 7, it is desirable to select item A in FIG. 3, which shifts the frequency axis of pattern A to a higher frequency range than pattern B.

In Fig. 8, A″ indicates the spectrum obtained by shifting the frequency axis of pattern A to the higher frequency side relative to pattern B according to the route determined as above.The characteristics of this route selection method are , in the determined route, the horizontal axis (frequency axis of pattern B)
Since there is no perpendicular part to , two frequency axes of pattern A will never correspond to one frequency of pattern B at the same time. However, as seen for example in the 2nd, 3rd, and 4th channels of pattern B,
There may be two routes, and depending on which route is selected, the channel of pattern A that corresponds to a certain channel of pattern B is different. C″ is
In such a case, the average spectrum is obtained by calculating the average of the intensity of that channel of pattern B and the value of each channel of pattern A corresponding to each route. In this case as well, the spectral features are well preserved.

Generally speaking, the above is as follows.

The p feature spectra for phoneme X obtained as above from a large number of speakers and contexts are 〓 ₁ , 〓 ₂
..., 〓 _p However, when 〓 _e = (x _e1 , x _e2 , ..., x _eo ), the reference vector 〓 _r = (x _r1 , x _r2 , ..., x _ro ) is determined. Then, any vector 〓 ₁ ,…, 〓 _p 〓 _n = (x _n1 ,…,
x _nj , ..., x _no ) and the reference vector 〓 _r ,
When constructing the lattice graph and setting the intersection of i(k) and j(k) as C(k) = (i(k), j(k)),
The distance d (C(k)) between x _r,i(k) and x _n,j(k) , the load factor w(
k)
For weighted average The point sequence C(1)C(2)...C(k)...C is minimized.
(k), and the component x _n,i(k) of the vector 〓 _n is
Find the vector 〓′ _n converted to x _n,i(k) . In this way, the said vector 〓 ₁ ,..., 〓 _p becomes 〓' ₁ ,...,
〓′ _p , and the average vector of 〓′ ₁ , . . . , 〓′ _p is used as the standard pattern of the phoneme X.

Note that when calculating the weighted average, as shown in the example, it is common to provide a matching window or limit the route selection method to prevent unnatural deformation of the vector 〓 _n . Of course, these limiting methods are not limited to those shown in this embodiment, and various methods may be used.

FIG. 9 shows the configuration of the phoneme standard pattern creation section 8 in the speech recognition device which creates standard patterns for each phoneme by expanding and contracting the frequency axis using the method described above and then taking the average.

80 is a memory for storing vectors 〓 ₁ , 〓 ₂ , ..., _〓 o obtained by the feature extraction unit 2 of phoneme X when there are various phonemes before and after a large number of speakers; 85 is a memory for storing 85
The vector stored in 0 is any one of the vectors 〓 ₁ , ..., 〓 _o 〓 _i (where |≦i≦n
a reference pattern memory that stores vectors that give the most standard spectral pattern if possible;
81 is the vector 〓 ₁ stored in the memory 80,
..., 〓 Stretchable pattern memory that temporarily stores _o in sequence, 82 is the reference vector stored in the reference pattern memory 〓 For _i , the frequency axis of the vector stored in the Stretched pattern memory 81 〓 _j 83 is a memory for storing the outputs 〓 ₁ , 〓 ₂ , . _. . , 〓 _o of the spectral frequency axis expansion and contraction section 82; Reference numeral 84 denotes an averaging unit that takes ^the _average of _the vectors ₁ , _.

The average vector obtained by such a device〓
is stored as a standard pattern of the phoneme X in the phoneme standard pattern holding unit 3 via the line 10.

By creating a standard phoneme pattern using the method described above, an ideal standard phoneme pattern can be obtained.

In addition, during recognition, the standard phoneme pattern obtained in this way is used, and the phoneme recognition unit 4, which recognizes the input phoneme of Patent Application 1983-109145, expands and contracts the frequency axis of the input phoneme vector for each standard phoneme. The method of calculating the distance between the patterns after performing the same process as described above is an ideal normalization method for the speaker, and significantly improves the accuracy of phoneme recognition.

[Brief explanation of drawings]

FIG. 1 is a block diagram showing the configuration of the speech recognition device, FIGS. 2 and 6 are explanatory diagrams showing the output strength of phonemes, and FIG. 3 is a diagram for explaining the lattice graph.
Fig. 4 is an explanatory diagram showing an example of the route selection method, Figs. 5 and 7 are diagrams showing route selection based on concrete calculations, Fig. 8 is a spectrum diagram of the same, and Fig. 9 is an explanatory diagram showing an example of the route selection method. FIG. 2 is a block diagram showing main parts based on one embodiment. 2...Feature extraction unit, 3...Phonological standard pattern holding unit, 80...Memory, 81...Stretched pattern memory, 82...Frequency axis expansion/contraction unit, 83...Spectral memory, 84...Averaging unit, 85...Reference pattern memory.

Claims (1)

[Scope of Claims] 1. means for converting an input speech signal into a time series of n-dimensional feature vectors, and a phoneme standard pattern using n-dimensional feature vectors prepared in advance corresponding to each phoneme to be identified; In a speech recognition device including means for converting the time series of the feature vectors into a phoneme series by comparing each vector in the time series of the feature vectors with each of the phoneme standard patterns, when determining a standard pattern for a phoneme X, , the feature vectors obtained from a large number of speakers and contexts collected for phoneme X are 〓 ₁ , 〓 ₂ , ...,
〓 _p ; 〓 _e = x _e1 , x _e2 , ..., x _eo ), and the reference vector 〓 _r = (x _r1 , ..., x _ri , ..., x _ro ),
Let any vector of the feature vectors 〓 ₁ , ..., 〓 _p be 〓 _n = (x _n1 , ..., x _nj , ..., x _no ) i−j
Lattice point C(k) on the plane = (i(k), J(k)), x _r
i(k)
The distance d(c(k)) between and x _nj(k ), and the load factor w
( _k ), the _point sequence c( ^{1 )} c( 2) By means of determining c(3)...c(k)...C(k) and by converting the component x _ni(k) of the vector 〓 _n into x _nj(k) according to the point sequence. , means to obtain the transformed vector 〓′ _n , and 〓′ ₁ , 〓′ ₂ ,...,
A speech recognition device characterized in that it has means for taking an average of 〓′ _{r ,} .