JPS6073698A - Pattern comparator - Google Patents

Abstract

(57) [Summary] This bulletin contains application data before electronic filing, so abstract data is not recorded.

Description

DETAILED DESCRIPTION OF THE INVENTION Field of the Invention The present invention relates to a pattern comparison device that automatically recognizes continuous speech patterns or the like as a series of patterns.

Conventional configuration and its problems A general configuration of a speech recognition device using pattern matching is as follows.

A feature extraction means converts an input speech signal into a series of feature vectors by using a filter bank, frequency analysis LPC analysis, etc., and a series of feature vectors uttered in advance and extracted by the feature extraction means are used as a standard pattern for all recognized words. A standard pattern storage means to be registered, an input pattern uttered to be recognized and extracted by the feature extraction means, and similarity as a series of feature vectors to all standard patterns stored in the standard pattern storage means. a pattern comparison means for calculating degree or distance;
It consists of a determining means for determining and outputting a word corresponding to the standard pattern with the highest degree of similarity (smallest distance) as a recognition result as a result of pattern comparison.

At this time, even if the same speaker utters the same word, the duration of the utterance differs each time, so when comparing the standard pattern and the input pattern using the pattern comparison means, the time axis of both is It is necessary to expand and contract the pattern lengths of the two to make them the same and compare them. At this time, since the change in utterance time length does not occur uniformly in each part of the uttered word, it is necessary to expand and contract each part non-uniformly. The best results have been obtained when the expansion/contraction is performed as described below using distance, which maximizes the similarity between the two patterns to be compared (minimizes the distance). (hereinafter, this matching is referred to as DP matching).The DP matching method can be explained using a lattice graph.Figure 1 is a lattice graph, in which the horizontal axis Cover turn T
=a1 a2... represents the i coordinate corresponding to aX, the i coordinate of 5 pages. To match the input cover turn T and the standard pattern by nonlinearly expanding and contracting the time axis, on this grid graph, a path (1) indicating the correspondence between the feature vectors of both patterns is determined by some kind of standard, and this The goal is to evaluate the distance between both patterns with respect to the path. When determining this route, limiting conditions are set in consideration of the nature of the voice. FIG. 2(a) is an example of restrictive conditions for route selection.

That is, in this example, the path to point ('yj) is point (i-
2,1-1) to the point (i-1,j) (or,
The point (i -1, j -1) or the path (3) or the point (i
-1, j-2) to point (i, j-1)'f: Path passing through (
→It means that only one of the following can be taken.

At this time, if the condition is that the starting and ending ends of the input cover pattern and the standard pattern must correspond, the matching path is limited to the shaded area in FIG. This restriction is to prevent extreme correspondence six page assignments from occurring due to the fact that no matter how much the time axis expands or contracts, it is unlikely that the same word will expand or contract to that extent.

If the distance between the vectors ai and bfl is dn (i, j), then the distance along the path between the input cover turn T and the standard pattern Rn is dn (i, j) along that path.
is defined as the weighted average of a, b on the route in Figure 2
, c, and d are the loads when the corresponding paths are selected. In order for DP matching to be applicable, the method for determining these loads is such that no matter what path is selected on the grid graph under the above-mentioned limiting conditions, It may be determined so that the sum of the loads along the path is constant. 'a=c=e=2.1)=d'
= 1, the sum of these loads is I + 1'', a
= b = c = 1.

If d = e = 0.5, the sum of this load I, a
==b=o, 5. If c=d=e=1, the sum of these loads becomes 0, which is constant regardless of how the path is selected.

Both of these are commonly used. Furthermore, by making this load a function of j under the condition that the sum of the loads is constant, it is also possible to perform operations such as increasing the load on a portion of the path that is to be matched with greater emphasis.

The distance between the page 7 human cover turn T and the standard pattern Rn is defined as the minimum value of the weighted average under the limiting conditions.

That is, by solving the following recurrence formula, the minimum value of the weighted average and the path that provides the minimum value can be determined.

Initial condition qn (1, 1) = dn (1, 1) where D (
T, R'') is the distance between the input cover turn T and the standard pattern Rn.

There are various other possible route selection conditions as shown in Figure 2 (
bl~(i) etc. are other examples. In addition to this, various other modifications can be considered. Of course, the recurrence formula can be rewritten into a corresponding one depending on the selection conditions of these routes.

When recognizing words that were uttered in isolation, of course they were uttered consecutively (uttered without any breaks between words).
DP matching has also achieved good results when recognizing speech.

The problem of continuous word speech recognition is formulated as follows.

The number of frames of the input butter is 11. The feature vector of the i-th frame is ais the standard number of word n.
The standard pattern Rn of word n is expressed as follows: OR" = b seconds"-'-b"j -"b'i'nThen, the combination of standard patterns corresponding to X word strings, □=Rq (1)R2)1. , Hql→=bq(1)b
qε1)...b, q(, enter) b(] Curse 2) bq! ,
2),,・b? (Wrinkled cover turn T = a 1 a
2...ai...11 Word string q(1) where the distance between the vector series is minimum q(2)...q(x)9
Turn the page.

If we try to solve the above calculation using DP matching in the same way as in the case of isolated words, for example, when we have a word with 1o digits as a standard pattern, it takes 163 digits to recognize the sound of 3 digits uttered consecutively. -1ooO types of standard patterns must be matched. As the number of standard patterns increases, the number of combinations quickly becomes prohibitive.

Therefore, in order to apply DP matching to continuous word recognition, we set the conditions for path selection so that the normalization coefficient of the cumulative distance of matching (the sum of the loads) depends only on the number of input frames. Then, as shown below, dynamic programming can be applied to word combinations of standard patterns, and the amount of calculation can be significantly reduced.

There are generally conditions for selecting a route as shown in FIGS. 3(a) to 3(e). The numerical value shown on the route is the load factor when that route is selected.

Let dR be the interframe distance (vector distance) between the i-th frame (characteristic ventle of the incoming cover turn T, hereinafter referred to simply as frame) ai and the j-th frame b of the continuous standard pattern H consisting of X standard 10-page pattern concatenations. ('*
i), and the time function that matches the input cover pattern with the continuous standard pattern (the matching path) is (i), and the next cumulative distance measured along this time function (the sum of the weights of the interframe distances) is ) D(TtR)
(denoted as ``intermediate''). That is, D(T,R)=min[! dR(itu(i)):l
・・・・・・(31u(i)
(i −1)≦2 j u(1)=1? u(I)
= JR. Also, m1n(f(z)) is minimized with respect to 2 f(z), argmin(f(z
)) means the value of 2 that minimizes f(z).

Equation (4) can be solved when the number of words is known, when it is unknown, or when automaton control is incorporated. Various methods have already been proposed, and some examples have been commercialized. .

Among these, the present application relates to a device that recognizes continuously uttered speech by solving equations incorporating automaton control.

Next, we will explain the conventional method of solving equation (3) by incorporating automatic control.

When we actually utter words in succession, their order is often fixed. Therefore, the input cover turn (
The input sentence) is a sentence generated by a regular grammar equivalent to the finite state automaton α, and among all the word sequences accepted by the automaton a, the word sequence (q(1)) that minimizes the expression (Q).

The recognition rate can be improved by increasing q(J,...tq(→).Here,
Word string (q(1) , q(2!l ,..., q(x
)) is accepted by automaton a if Δ(qo, q
(1))=q,C8゜Δ(qi,q(gon=qjC8,・
..., Δ(qkyq(''-'))=S is a finite set of states q [q6q□yq1 ?..., q ).

-1-1 Σ is a finite set of input words n (nl-1, 2,..., N
).

Δ is the state transition function, S×Σ→S, (Δ(qi t”)
-q' l y] qo is q in the initial state. 4S, F is the final state set FC8
is o coco-'rq(1) p-qcxi e i
1t2p-Nl.

The difference from normal automaton recognition problems is that the frame number representing time is also included as a variable, and the degree of acceptability (cumulative distance) is output instead of simply accepting or rejecting. The minimum cumulative distance among all word strings, N
q, (i) as Dq, the last unit of the word string corresponding to (1)]] Base・Bq, 0) as Nq, starting position of (t)'
Ina7” (the last frame of the word before N1(i),
(referred to as back pointer), Qq, (i) state transition to q5], the state name that satisfies Dq, (1), that is, Δ(Q
When qj (i), Nqj (i)) - qj, by solving the following recurrence formula, the solution to the equation ('4) by automaton control can be obtained. In other words, Initial condition on page 13 Dgo (o) - 01B9allow)=0 and q=
Δ(qkyn) ・・・・・・(4Nq(i)=n, B
9(i)-m, Q9(i)=qk. If this calculation is performed until i = I, the words will be rounded to 0 in reverse order from the last word as follows:
i), q16F, ■ n=Nq(i) ■ B q (1)\○, then ' = Bq(') t
Set q=%(i) and go to ■. If Bq(i)=0, end.

FIG. 6 is a flowchart.

In addition, D"(m+1: i) is defined by the following formula,
In the same way as the DP matching of isolated words mentioned above,
Ru.

D"(m+1:t)'('4:d"(k, u
(k)):)-(a)expi)"k=m+1 Here, in the case of the route shown in Fig. 3(a), o<:u(k)-u (k-1)≦:2.u (m+1
)=1, u(i)=1”14 pages.

Formula ('4 is D'' (n juice 1:i) for a predetermined range of m from 11 to 12, and satisfies the condition for each m) and n, and let this be Dq(i). However, as a way to reduce the amount of calculation, after i=m+1, word n
Then, when the next state is q, (ttt) = (
Cumulative distance from 1y1) to (' t ) = ("y ]ri woD"(",i')
D as the minimum value for n, qk of tT'')
A method of calculating q(z) has been proposed.

The problem with these methods is that although the amount of calculation can be further reduced depending on the structure of the automaton, this point has not been taken into consideration in the past. The purpose of this paper is to propose a pattern comparison device that significantly reduces the amount of data required.

Page 15 Structure of the Invention The present invention is based on DP matching of continuous patterns by automaton control.
qkt''), then for pattern n, state q
When there are multiple k, if the minimum cumulative distance in state 1 for each state is /4, then the cumulative distance to the last △ pattern up to state q is the previous state By calculating as if only qk exists, the calculation of cumulative distance, which was conventionally performed for all qk, is reduced.

Description of examples The principle and embodiments of the present invention will be explained below.

FIG. 6 is a diagram for explaining the present invention in detail.
a) is a finite automaton representation for the word written in (bl).In order to simplify the explanation here, a single syllable is used instead of the word.In the calculation in the conventional example above, all state transitions are In FIG. 7, the number is 0, which had to be calculated for 22), but in the example of FIG. 7, state S12 has state S6. S7. S8
However, the same “n′” is included. In such a case,
State S6. If S7 and S8 are temporarily degenerated, only one calculation is required for S'' in state s12. This also applies to YA''' in state S. The present invention 4 is a pattern comparison device that utilizes this principle to reduce the amount of calculation.

In the example of FIG. 7, q=$12. If n0=n0 (no is the number attached to the end of the syllable), then for q=812, the formula ('4 means DB (i):=min [:DnO(fW)+D”) O
(m+1:i)]12 qk, m (Ik. In other words, in Fig. 5, $6.s7゜S
When emitting 1 digit from 8 and transitioning to 12, DB12
In order to minimize (i), DB6(m), DB7(
m).

1 to transition from the smallest state of Ds, (m)
7 b]! It means that it will become. If we write this in general, qo
As the initial state, Dqo(O)=0. Bqo(O) △ Nq(i)=n, Bq(i)=m, Qq(i)=arg
min(D (Q):]k.In other words, if there are states immediately before state q in frame i that have the same syllable (or word) n, the process continues until those states are reached. This means that the state continues from the state where the cumulative distance of is the minimum.

Using this fact, in the case of the example in Figure 6, the above can be said about ``hi'' and YA'', so the formula (field calculations will be 19 times, reduced by 3 times.D'' (m-)1: i
) and solve the equation (when directly solving the field, this amount can be almost ignored, but when using the high-speed calculation method mentioned above, it makes a big difference. Also, depending on the task, a large reduction in the amount of calculation can be expected. Ru.

An embodiment in which the present invention is applied to one of the above-mentioned high-speed calculation methods will be described.

According to the path restriction conditions shown in Figure 3(b) on page 18, D”(m-1-
1: i) is along the path inside the diagonal line in Fig. 6 from ('p J) - (m, 1) to ('tj) - ('y'
) is the cumulative distance. Here, (@ is the slope %, (η is a straight line with slope 2.) Now, the path (@ is (i, 1) = (
1,1) to (i?i)""('?I"), then according to the principles of dynamic programming, from (ig) = (1t1) to the path (→ The upper point ('
9] )=("p j') is exactly the same as the path to ('yN=(",j') at ('yN=(", j'). Therefore, D"(m- 1-1: i
) can be determined as Dq (i) = Dg (I P J di. However, the initial value of recurrence formula (9) is "(-1tj)-" (j = 0,1s" y”)
Dq (0, O) = 0 19 pages D" (0, j ) = oo (j = -1, 1, 2, ,,
, J"D: (i, -1) = oO (n = 1.2...,
I) Da(i, 0)-Dqk(i-1) (qk is q=U
(qk, which satisfies n) The back pointer gate (l, J) corresponding to Dq(', l) is BH(itt) when
-B, :(i-2tj-1) D revision i? +)=@ When li(itj)-Bq('-'?]-1)Dq(', j
) When mute Bη('tJ)=Bq('-1y+-2
). The initial value of Bq('tl) is B:(i,
1)=t −1 .

Figure 6 shows 'q (i, i) on i = i.
explains what it means. (8
), (10).

The straight line (12) has a slope of %, (9), (11), (1
3) The straight line has a slope (→, (itT)−(it
The path leading to T'') (point 26) is included in the area between straight lines (E) and (c4), and the matching path passing through point 06 is included in the area between straight lines (10) and (11). , point (
The matching path passing through 17) is a straight line (12) and (13)
In other words, the path passing through point 16 is between 18 and 190, the end is between 20 and 21, and the path passing through point 17 is between 18 and 190, and the path through point 17 is between 20 and 21. is between 22 and 230, and the end is between 24 and 26. After all, Dq on itj (itj)
When the slope of the a straight line 28 is expressed as %, the cumulative distance Dn (t // ) −
This represents the cumulative distance along the way with respect to Dn (t“t”).

(Hereinafter, "a (' t ]) will be referred to as the intermediate cumulative distance.) Also, as is clear from the recurrence formula (c4), Dq
(' t )), frame i-2 and i-
Only the intermediate cumulative distance at frame 1 and the interframe distance at frames i-1 and i need to be known, and the values before that may be forgotten.

To summarize the above, to calculate Dq(i),
For each frame i, n=1.2. ...2 pieces.

n=1, 2. °'tNtq=q1sq2t"'yqj,
When D3 (' t ]) is calculated for j-1 and 21 pages Qq(i)=qk Nq(i)=n. In this way, each grid point ('
The calculation of d''(i, j) in y5) only needs to be done once for each word n, and the calculation of ``(' y ]) requires q, n, Δ(q
It only needs to be done once for qk, where kt n )=q, and dn(i'
t5L'' (m+1: i) is required to calculate the equation (the amount of calculation is greatly reduced compared to the method of directly solving the field).

In this method, processing is performed for each frame i of the input pattern, but it is also possible to process each frame j of the standard pattern. That is, as is clear from the recurrence formula (@, in order to calculate D: (i, j) even in the j direction, the inter-frame distance between frame i-1 and frame j "('yj)" and frame j Since it is sufficient to know the intermediate cumulative distance between -2 and frame j-1, i
It is also possible to calculate Dq('s+) on tj.

The method on page 22 and above is based on the automaton control clock synchronous propagation type D.
This is called the P method (F3ACWDP). At this time, to introduce the idea of formula (→), we need to use the recurrence formula (c
In the calculation of 4, the initial value of DH (i, j) may be set as follows. That is, when q=Δ(qk, n) is satisfied, the following may be used.

FIG. 8 is a block diagram showing one embodiment of the present invention. 1
oO is an input terminal for audio signals. A feature extraction unit 101 converts an input audio signal into a series of feature vectors. 102 is a standard pattern storage unit in which each word (syllable) to be recognized is stored as a series of similar feature vectors; 103 is an interframe distance calculation unit;
01 output feature vector and standard pattern storage unit 102
Calculate the difference between the feature vector and the feature vector.

23BEF 104 is an interframe distance storage unit that temporarily stores the calculated interframe distance. 1o5 is an automaton control rule storage section. 106 is a cumulative distance storage unit in which cumulative distance Dq (1) is stored for each frame. Reference numeral 107 denotes an initial value calculation unit, which calculates the initial value in the calculation of rumors according to formula (10). This is a part that calculates the recurrence formula (10) based on
This part calculates the final value for each frame from the results of step 8. Reference numerals 113, 114, and 115 are a last word storage unit, a previous state storage unit, and a back pointer storage unit, respectively, which are determined by the final value determination unit 109.
The last word Nq(i), the state Q9(i) immediately before q, and the back pointer Bq(i) are stored for each frame. 112 is a part that determines the state of the input audio in the final frame.

That is, when qf F, the state q of the final frame I is determined as. 11o is a voice section detection section, and 111 is a frame number counting section. 116 is a backtrace control unit, which includes the last word storage unit 113 and the immediately preceding state storage unit 1.
14 and the contents of the back pointer storage unit 116, the fourth
The recognition results are output from the last word storage unit 113 in the reverse order according to the flowchart shown in the figure.

FIG. 9 is a chart showing an example of the case where the operation of the above embodiment is realized by software.

Steps 200 to 204 are the parts that initialize the entire process, steps 205 to 214 are the parts that perform processing in frame i, and steps 200 to 214 are the parts that perform processing for frame i.
6 to step 212, each standard pattern n (n=1
, 2 , ..., N), page 26 Dn(i), request part, steps 213 to 214 are the last word (syllable) assuming that the input speech ends at frame i. This is the part that stores the name, the corresponding cumulative distance, the back pointer, and the previous state for each state. Step 208 corresponds to the operations of the interframe distance calculation section 1o3 and the interframe distance storage section 104. Step 210 corresponds to the operation of the initial value calculation section 107.

Steps 211 and 212 are performed by the recurrence formula calculation unit 108.
corresponds to the behavior of Steps 213 and 214 correspond to the operations of the final value determining section 109. Step 215
- Step 217 is a process of determining the recognition result in the reverse order starting from the last frame of the input audio, and includes the final frame state determining section 118, the last word storage section 113, the immediately preceding state storage section 114, and the back pointer storage section 115. , and the backtrace control unit 116.

Another way to directly obtain and solve ``i (it]) is Automaton Control Level 1. Building
26 pages of Ash (FSALB) are known. The idea of the present invention can also be introduced to this, and the amount of calculation can be significantly reduced.

FIG. 10 shows the automaton control when reading a six-digit number as 0,000,000, for example, 73,200 to koana. In contrast, traditional FSA
Table 1 compares the calculation amount and work memory storage amount for a device using CWDP, F, 5ALB and a device using degenerate FSACWDP and degenerate FSALB according to the present invention. According to this table, in the case of the task shown in FIG. 10, the amount of calculation is significantly reduced and the amount of memory is also less than the same, so it is practically effective.

Table 1 27 Effects of Bacona's Invention As mentioned above, the control rule for the automaton can be expressed as q=Δ(
qk, n), then for pattern n, state qk
When there are multiple patterns, conventionally all the cumulative distances for the last pattern n were calculated for the immediately preceding state qk, but if some of the states immediately before becoming state q have the same pattern n, then By taking advantage of the fact that times are continuous from a state with the minimum cumulative distance to the previous state, it has become possible to significantly reduce the amount of calculation for some tasks.

[Brief explanation of the drawing]

Figures 1 to 3 are diagrams explaining the basic principle of DP matching, Figure 4 is a flowchart showing the procedure for segmentation and recognition word determination in continuous word speech recognition by automaton control, and Figure 5 is a detailed explanation of the present invention. A diagram showing an example of an automaton expression to be explained, FIGS. 6 and 7 are diagrams explaining the principle of an embodiment of the present invention, FIG. 8 is a block diagram showing an embodiment of the present invention, and FIG. Chart when the functions of the same embodiment device are realized by software, No. 10
The figure is a diagram showing the effect of the same embodiment. 101...Feature extraction unit, 1o2...Standard pattern storage unit, 1o3...Interframe distance calculation unit, 104...Interframe distance storage unit, 1
05...Automaton control rule storage section, 106
... Cumulative distance storage section, 107 ... Initial value calculation section, 108 ... Recurrence formula calculation section, 109.
...Final value determining section, 110... Voice section detecting section, 111... Frame number counting section, 112
...Final frame state determination section, 113...
... Last word storage section, 114 ... Immediate state storage section, 115 ... Back pointer storage section, 11
6...Backtrace control section. Name of agent: Patent attorney Toshio Nakao and 1 other person No. 1
Figure 2, α, (b. (C) (dl (C) (f) (7) (h) (2) (j) Figure 3, ω) tb) (C) (cb Figure 4 open d Shi=I D glance (shi) n=Nth (wa8g(il-0 1z84ri+ Figure 5(b) ~AeOHIJKuX NAcrOYA HUKIJSIMA NAGANOHIJKLJYAMhaNAt3AO'KA OγHAMA Sr;ILIOKA OKAYAMA suKuOKA wAKAYAM Ha 6th Figure 9: Figure 9 - A, 2.---.lδl-1 n-4,2,---, tJ 20' j-i, 2.---.Jl+202 Dg(O, jl-", 'J'-i5ha-” 20
3B Jin0.夛)～の ,Bε＜−rj)−ωDo(o)
-o 204 Goro 4.2. ---. r n-i, 2. ---. ~2θ6 1-i, 2. ---. JQ207 d' [2, jl Shibajumero 2θ8 φ-i, 2. ---. l5l-t20'? ';i,m
,Lr#,,1LrlCD6Ci -1+1 krFL
, bll)-d(k, n+krt'pDε(ε-+, o
)−Dε−(2−11,1)? LL-2. D no = OOB
j! t-zu0]! nii + vj (=-t, -1=
Φp i, 2. ---, :In" Di r&, 〆+=mtn Djr;-f, 7-IM'
f#,,)l−Dl(b−1,t−2)+d”(i,1
) −−−−−B@+i−t, i−n Di[A, j)
-■4h! 212 material (shi, me) wealth +1 Jit-inade" Dε (vessel, j): 8εtB-t when ■, multi-i'
l Dε (shi, 3) ~■ Nosa 6 = 1.2, -, IJ
I 213 π pass r furnace shi r) [1) j (i, J'l] ~ shi (1 + 6 JP-A-Sho GO-73698 (10) Figure 10 A/ ds = (-ys, double 7 −, 90,000 L dq=(+, 2 ima r ---19,000) d3=(white, 200, −, 9,000),
d2=(-r,=shi,---,9t)dzu=(-12,
---1ri14)

Claims (1)

[Claims] The input signal is defined as a feature vector a 1t &2 /...a l
4. Feature extraction means for converting into a series of feature vectors; bn,...bn...bfn
Standard pattern Rn (where n=1.2...
·, N), and the distance between the feature vectors a1 and b5 in the i-th frame of the input pattern ``(', 1)woj = 1t 2t''' t
"j" = 122 g..., an inter-frame distance calculation means that calculates N, and in the i-th frame of the input pattern %] = 1, 2, -, J"; n = 1.2
,..., Tweet N, assume that the state is q, then calculate the intermediate cumulative distance Dq (it5) from the state q in the i-1st frame of the state qk immediately preceding q in pattern n.
The cumulative distance D9k (i-1) along the path leading to k is determined as the continuation of the minimum value, and the intermediate back pointer Bq (' t j) along the path leading to it is calculated as the cumulative 2
Page distance calculation means and the last pattern n when assuming that it ends in state q in the first frame are calculated by intermediate cumulative distance 1
)n(i, )n) is determined as n that minimizes (i))
When we choose qk that minimizes 'Qq(i)=q
k, final value determining means, last pattern storing means for storing these values for each frame, back pointer storing means, immediately preceding state storing means, and when the input is completed at the last frame I of the input pattern, Letting the cumulative distance Dqf(I) to the state qf allowed as the state be the state q, and from the contents of the back pointer storage means, the previous state storage means, and the last word storage means, Bq(1)=o. up to n=Nq(i). 1=Bq(i), q=09(1) and outputs the recognition results in the reverse order.
A pattern comparison device characterized by the following features: