JPH0355836B2 - - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION [Industrial Field of Application] The present invention relates to a continuous speech recognition device, and particularly to a continuous speech recognition device with a high recognition rate that rapidly recognizes sentence speech continuously uttered according to grammar.

[Conventional technology]

Among voice recognition devices, devices that recognize sentence sounds uttered according to grammar can recognize computer programs, limited business texts, and commands for air traffic control and control of various equipment, and have a wide range of applications. There is. Given grammatical constraints,
It is known in principle that misrecognition can be prevented by using the grammar rules. In particular, when there is a restriction on the number of digits in input speech in continuous number recognition, the recognition rate can be improved by regularizing the restriction.

A method for recognizing sentence sounds continuously uttered according to such a grammar was proposed in a patent application filed in 1983 by the same applicant.
It is described in the specification 239303 "Continuous speech recognition device". The principle of this blockwise DP matching method is roughly as follows. The grammar is expressed by an automan α, and the automan α is defined as follows.

α=<K, Σ, Δ, P ₀ , F> ...(1) Here, K: set of states p {p|p=1, 2,
...π} Σ: Set of input words n {n|n=1, 2, ...N} Δ: State transition rule {(p, q, n)} Here, (p, q, n) is pn -→ It means q state transition.

P ₀ : Initial state, hereinafter indicated as p=0.

F: Final state set Fn -→ K Next, according to the automaton α, word nn -→ Σ
A voice pattern A obtained by continuously uttering is expressed as A=a ₁ , a ₂ , ...a _i , a _I ...(2), and is called an (unknown) input pattern. A standard pattern B ⁿ =b ₁ ⁿ , b ₂ ⁿ , ... b _j ⁿ , ... b ⁿ _Jo (3) is prepared for each word nn -→ Σ, and this is called a word standard pattern. Continuous speech standard pattern C= obtained by connecting this word standard pattern B ⁿ according to Automan α
Perform DP matching between B ⁿ¹ , B ⁿ² , ... B ^nx and the input pattern A, calculate the amount representing the degree of mutual difference between the two patterns (hereinafter referred to as the degree of dissimilarity), and give the minimum degree of dissimilarity. The word sequence is the recognition result.

Here, the minimum degree of dissimilarity is found using the following dynamic programming method. Initial conditions are T(0, 0)=0 T(i, q)=∞, i≠0, q=0 G ₁ (p, n, j)=∞, G ₂ (p, n, j)=∞ ...(4), and sequentially from b=1 to I/BL (Here, I/BL is assumed to be divisible for ease of explanation)
Based on the boundary conditions of equations (5) and (6), the recurrence equations of equations (7) and (8) are expressed as (p, q, n) for all pairs (p, n) such that n -→ Δ
Calculate about. In other words, the boundary condition with state p is i =: _s ,...i For _e , g(i-1, 0) = T(i-1, p) h(i-1, 0) = i-1...(5) However, i _s = (b-1)・BL+1, i _e =b・BL, and the boundary conditions with block b-1 are i=1,
..., J ⁿ for each standard pattern time j g(i _s -1, j) = G ₁ (p, n, j) h (i _s -1, j) = H ₁ (p, n, j) g (i _s −2, j)=G ₂ (p, n, j) h(i _s −2, j)=H ₂ (p, n, j) …(6), and the recurrence formula g(i, j)=d(i,j)+mind(i-
1, j) + g (i-2, j-1) g (i-1, j-1) g (i-1, j-2) ...(7) h (i, j) = h (i^, j^) ...(8) However, ((i^, j^) is (i, j) that gives the minimum g(i, j) on the right side of equation (7).

is calculated from input pattern time i=i _s to i _e . Here, d(i, j) is the distance between the feature vector a _i of the input pattern at time i and the feature vector b ⁿ _j of the standard pattern at time j.

d(i, j)=Dis(a _i , b ⁿ _j ) (9) Store the boundary value of the calculation result for the calculation of the next block.

G ₁ (p, n, j) = g (i _e , j) H ₁ (p, n, j) = H (i _e , j) G ₂ (p, n, j) = g (i _e , 1 , j) H ₂ (p, n, j) = h (i _e , 1, j) ...(10) (6), (7), (8), (10) are calculated at standard pattern time i =
After finishing up to J ⁿ , if T( _i , q)>g(i, j _n ⁾ then T(i, q)=g(i, J ⁿ ) N(i, q)=n P(i, q)=p L(I, q)=h(i, J ⁿ ...(11) Calculate.

The recurrence formula calculations for equations (7) and (8) described above are performed for each block by converting the BL frame portion of the input pattern into blocks, as shown in FIG. 2 (explanatory diagram).

Finally, the recognition result of the input pattern is determined by the following procedure as a determination process.

Initial condition q^=argmin [T(I, q)]…(12) qn ―→ F q=q^, m=1...(13) Recognized word n^=N(m, q) Word starting point l^=L(m, q) State transition q^=P(m, q) …(14) seek.

If l^ > 0, repeat equation (14) with q = q^ and m = l^. If l^=0, end.

The recurrence formula (7) of the method described above is calculated along the path shown in FIG. 3a. For this reason, the correspondence between the input pattern and the standard pattern on the time axis is limited to expansion/contraction from 1/2 to 2 times. This is called "tilt restriction" and eliminates locally unnatural time axis correspondences. By using this slope restriction, it is possible to increase the recognition rate.

However, when using recurrence formula (7), the work memories G ₁ (p, n, j), H ₁ (p, n, j), G ₂
(p, n, j), H ₂ (p, n, j) are required.
Furthermore, data transfer (corresponding to equations (6) and (10)) between the calculation unit and the work memory is required. That is, when using the recurrence formula (7), there is a drawback that the work memory becomes large and data transfer increases.

On the other hand, it is possible to use a recurrence formula without slope restrictions in the blockwise DP matching method. In this case, the calculation is performed along the path shown in FIG. 3b. The calculation procedure for this DP matching is described above.
It is performed in the same way as equations (4) to (14), but using equation (15) instead of equation (4) as the initial condition, T(0, 0) = 0 T(i, q) = ∞, i≠0, q≠0 G(p, n, j)=∞ …(15) Using equation (16) instead of equation (6) as the boundary condition with block b-1, g(i _s −1 , j)=G(p, n, j) h(i _s −1, j)=H(p, n, j) …(16) As the recurrence equation, use equation (17) instead of equation (7). Use, g(i, j) = d(i, j) + ming(i-1, j) g(i-1, i-1) g(i-1, j-2) …(17) (10) Use equation (18) instead of Eq.

G(p, n, j) = g(i _e , j) H(p, n, j) = h(i _e , j) ...(18) The recurrence formula (17) without slope restriction explained above is The method used can reduce the work memory by half compared to the method using recurrence formula (7) with slope control.
Furthermore, data transfer can be reduced to 1/2.

However, since there is no slope limit, the correspondence between the time axes of the input pattern and the standard pattern is allowed to become unnatural, causing erroneous recognition.

[Problem that the invention seeks to solve]

The above-mentioned conventional continuous speech recognition apparatus has the disadvantage that when using a recurrence formula with slope restriction, the work memory is large and data transfer is frequent. On the other hand, when using a recurrence formula without slope restrictions, there is a drawback that there are many erroneous recognitions.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a continuous speech recognition device which eliminates the above-mentioned drawbacks, has a small work memory, requires little data transfer, and has a high recognition rate.

[Means for solving problems]

The continuous speech recognition device of the present invention includes an input pattern memory section that stores input patterns, a standard pattern memory section that stores standard patterns, and a rule (which means that state transition p→q occurs when word n is input). an automaton storage unit that stores a state transition table Δ and a final state group F, which are a group of p, q, n);
In the state transition specified by state p and word n, the input pattern at each point (i, j) in block b having the width of the input pattern BL frame within the area defined by input pattern time i and standard pattern time j. A distance calculation unit that calculates the distance d(i, j) between the feature vector and the feature vector of the standard pattern, and a distance calculation unit that calculates the distance d(i, j) between the feature vector and the feature vector of the standard pattern;
A boundary DP calculation unit that calculates the DP matching calculation using the first recurrence formula, and a DP inside the block b.
A main body DP calculation unit that calculates a matching calculation using a second recurrence formula, and a determination unit that determines a combination of words that yields the minimum cumulative distance calculated by the boundary DP calculation unit and the main body DP calculation unit. There is.

[Effect]

Next, the operation of the present invention will be explained with reference to FIG.

FIG. 4 is a diagram for explaining the principle of the present invention.

The calculation of DP matching according to the principles of the present invention proceeds by using a recurrence formula without a slope restriction at the boundary of a block, and using a recurrence formula with a slope restriction inside the block.

The initial conditions are T(0, 0)=0, T(i, q)=∞, i≠0, q=≠ G(p, n, j)=∞...(19), and from block b=1, I/ (22) to (25) based on the boundary conditions of equations (20) and (21) sequentially up to BL.
The recurrence formula of (p, q, n)n -→ Δ is the pair (p, q, n)
n). In other words, the boundary conditions with state p are i=i _s , ..., i _e g(i-1, 0)=T(i-1, p) h(i-1, 0)=i-1 ...(20 ) However, i _s = (b-1)・BL+1, i _e =b・BL, and the boundary conditions with block b-1 are j=1,
..., J ⁿ for standard pattern time j g(i _s -1, j) = G(p, n, j) h (i _s -1, j) = H(p, n, j) ...(21) The first recurrence formula g(i _s , j) = d(i _s , j) + ming(i
_s −1, j) g(i _s −1, j−1) g(i _s −1, j−2) …(22) h(h _s , j)=h(i^, j^) …( 23) However, (i^, j^) is (i, j) that gives the minimum on the right side of equation (22).

Then, the second recurrence formula g(i, j) = d(i, j) + mind(i-1, j) + calculates the block body.
g(i-2, j-1) g(i-1, j-1) g(i-1, j-2) …(24) h(i, j)=h(i^, j^)… (25) However, (i^, j^) is (i, j) that gives the minimum on the right side of equation (24).

Calculate from i=i _s +1 to i _e .

The boundary value of the calculation result is stored for calculation of the next block.

G (p, n, j) = g (i _e , j) H (p, n, j) = H (i _e , j) ...(26) The calculations of equations (21) and (25) are standard patterns. Time j=J ⁿ
If T( _i , q)>g(i, J ⁿ then T(i, q ₎ =g(i, J ⁿ ) N(i, q)=n P(i, q)=p L(i, q)=h(i, J ⁿ ...(27) It is obtained using the same procedure as the method, ie, equations (12), (13), and (14).

〔Example〕

Next, regarding the present invention, FIGS. 1, 5, - 1
This will be explained in detail using FIG.

FIG. 1 is a block diagram showing an embodiment of the continuous speech recognition device of the present invention, and FIGS. 5, 6, 7, and 8 each show a partially detailed configuration of the embodiment of FIG. 1. FIG. 9 is a time chart showing the time relationship of the operations in FIG. 1, and FIGS. 10a to 10d are flow charts showing the flow of the operations in FIG. 1.

The standard pattern memory unit 130 stores a standard pattern B ⁿ of word n included in the word set Σ, and the automaton storage unit 230 stores state transition rules (p, q, n) and designation information for the final state F. remembered.

When an unknown input voice is input from the microphone 100, the input section 110 performs frequency analysis, converts it into a vector _a1 representing the characteristic, and sequentially sends it to the input pattern memory section 120. In addition, the input section 110
is provided with a function of determining a voice section by detecting the voice level, and generates a voice section signal S that is "1" during the voice section and "0" otherwise. The control unit 240 controls the voice section signal S
Initialization pulse SET ₁ is generated at the rising edge of . This results in block 10 in Figure 10a.
The initialization corresponding to G memory 150 and T memory 2
00.

When the above initialization is completed, the input pattern block signal b is counted as 1, 2, etc. in synchronization with the input of the input feature a _i for the subsequent BL frames. In this input pattern block b, the word designation signal n from the control section 240 changes from 1 to N. each word n
The state table in the automaton storage section 230 is referred to, and a state designation signal q defined by the word n and state p is also output.

Next, the operation within one cycle of word n and state p will be explained. Through this one cycle, calculations shown in the shaded areas in FIG. 2 are executed. That is, under the boundary conditions of equations (20) and (21), (22) and (25)
Calculate the formula. First, a signal from the control unit 240
By SET ₂ , block 11,1 in Figure 10a
2 is set in the g memory 330 and h memory 340 of the DP matching work memory. Subsequently, the standard pattern time signal j from the control section 240 changes from 1 to J ⁿ . At each standard pattern time j, the input pattern time signal i ₁ is
From i _s (i _s = (b-1)・BL+1), i _e (i _e =b・BL)
changes up to.

Block 13 of FIG. 10b in the i _s frame.
The calculation shown in is performed by the boundary DP calculation unit 310.

FIG. 5 is a block diagram showing an example of a boundary DP calculating section. First, the i _s frame of the input pattern and the nth
J-frames of the standard pattern of words are read out,
The inter-vector distance d(i, j) shown in equation (9) above is determined by the distance calculation unit 300.

On the other hand, the boundary DP calculation unit 310 calculates the minimum value of the three dissimilarities in parallel with the distance calculation. That is, from the g memory 330, g(i _s −1, j), g
(i _s -1, j-1, g (i _s -1, j-2) and h (i _s -1, j), h (i _s -1, j-
1), h(i _s -1, j-2) are read out and stored in registers G1, G2, G3 and H1, H2, H3, respectively. The comparison circuit 312 detects the minimum value from the three registers G1, G2, and G3, and selects the register Hn^ corresponding to the register Gn^ (where n^ is 1, 2, or 3) from which the minimum value was obtained. It emits a gate signal n^ to The contents of the register Hn^ selected by the gate signal n^ are stored in the h memory 340h (i _s ,
j). Further, the minimum value g (i _s −1, j^) outputted from the comparison circuit 312 is added to the distance d (i _s , j) obtained by the distance calculating section 300 by the adder 311, and g memory 330g
(i _s , j).

Then, at each input point time i (i changes from i _s +1 to i _e ), block 1 of FIG.
The calculation shown in 4 is performed by the main body DP calculation section 320.

FIG. 6 is a block diagram showing an example of the main body DP calculation section. First, the i-frame of the input pattern is read out, and the distance calculation unit 300 calculates the distance between the vectors shown in equation (9) to obtain d(i, j).

On the other hand, the main body DP calculation unit 320 calculates the minimum value on the right side of the second recurrence formula of equation (24) in parallel with the distance calculation. That is, g(i-2, i-1) read from the g memory 330 and the delay circuit 32
The distance d(i-1,j) delayed by one time in step 4 is added by adder 323 and stored in register G1. Also, from the g memory 333, g(i-1, j
-1) and g(i-1, j-2) are read out and stored in registers G2 and G3, respectively. moreover,
From the h memory 340, h(i-2, j-1), h(i
-1, j-1) and h(i-1, j-2) are read out and stored in registers H1, H2, and H3, respectively.

The comparison circuit 322 has three registers G1, G2,
The minimum value is detected from G3, and a gate signal n^ is generated to select the register Hn^ corresponding to the register Gn^ (n^ is any one of 1, 2, or 3) from which the minimum value was obtained.
The contents of the register Hn^ selected by the gate signal n^ are written to h(i, j) of the h memory 340. Further, the minimum value output from the comparison circuit 322 is the distance d calculated by the distance calculation section 300.
(i, j) by the adder 321, and g
It is written to g(i,j) in memory 330. By changing the input pattern time i from i _s +1 to _ie , the processing for the standard pattern time j is completed. Further, after the standard pattern time j reaches the terminal J ⁿ , the signal is changed as shown in block 15 of FIG.
According to SET3, the result of recurrence formula (24) g(i _e , i),
h(i _e , j) in table memory G(p, n, j),
Store in H(p, n, j). Next, control section 2
According to the signal i2 (varying from i _s to i _e ) issued by 40, the comparison shown in block 16 of FIG. 10c is carried out. That is, as shown in FIG. 7, the table memory 200
From T(i, q) and g(i, q) from g memory 330.
J ⁿ ) is read out and compared by the comparator circuit 170. If T(I, q)>g(i, J ⁿ ), a wp signal is generated, and g(i, J ^N ), n, p, h( i, J ⁿ ) are table memories T(i, q), N(i, q), P(i, q),
written to L(i,q).

With the above operations, the processing of state p, word n, and input pattern block b is completed. Further, the state designation signal p is changed from p ₁ to π, thereby completing the processing for the word designation signal n. Furthermore, when the word designation signal n changes from 1 to N, the processing for input pattern block b ends. After the input pattern block signal becomes I/BL,
The determination process shown in equations (12), (13), and (14) is started.

The determination unit 220 is configured as shown in FIG.
First, as the processing of block 17 in FIG. 10d,
The states q included in the final state set F are sequentially specified from the automaton storage unit 230 according to the signal q3, and T(I, q) is read from the T memory 200,
The comparison circuit 221, minimum value register 222, and state register 223 are used to obtain the state q^ that provides the minimum T(I, q). Next, as the process of block 18 in FIG. 10d, the determination control unit 227 determines that q=q^,
Assuming m=I, address signals m3 and q are transferred to N memory 1.
90, send to L memory 210, P memory 180 N
Read out (m, q), L (m, q), and P (m, q).
These L(m, q) and P(m, q) become the next time m and state q, and N(m, q) is output as the recognition result. By repeating this process until l^ becomes zero, recognition results can be obtained sequentially.

Although the configuration of the present invention has been described above based on examples, these descriptions do not limit the scope of the present invention.

This example is explained based on the blockwise DP matching method as described in the specification of Japanese Patent Application No. 58-239303, but multiple frames of the input pattern are collectively made into blocks and DP matching is executed. If it is a method, it is possible to implement the principle of the present invention, that is, the block boundary part and the block body part, using different recurrence formulas. That is, the Blockwise DP method having the modified DP matching calculation section described in the specification of Japanese Patent Application No. 59-067116 The block described in the specification of Japanese Patent Application No. 59-068015 is tilted diagonally with respect to the standard pattern axis. The principle of the present invention, the block boundary portion, can also be applied to the diagonal block DP method and the variable diagonal block DP method in which the block width is changed according to the standard pattern length described in the specification of Japanese Patent Application No. 59-267830. The first recurrence formula can be used, and the block body portion can use the second recurrence formula.

Furthermore, although equation (22) is used as the first recurrence equation for determining the block boundary portion, any recurrence equation may be used as long as it uses only input pattern times i and i-1. For example, g(i, j) = d(i, j) + ming(i-1, j) g(i-1, j-1) ...(28) or g(i, j) = d( i, j) + min g(i-1, j) g(i-1, j-1) g(i-1, j-2) g(i-1, j-3) ...(29) may also be used.

Also, formula (24) is used as the second recurrence formula for calculating the block body, for example, g(i, j) = mind(i, j) + d(i-1, j) + g(i-
2, j-1) d(i, j)+g(i-1, j-1) d(i, j)+d(i, j-1)/2+g(i-1, j-2)...(30 ) g(i, j) = d(i, j) + mind(i-1, j) + d(i-
2, j) + g(i-3, j-1) (i-1, j) + g(i-2, j-1) g(i-1, j-1) g(i-1, j-2 ) g(i-1, j-3) ...(31) etc. can obviously be used.

〔Effect of the invention〕

As explained above, the present invention makes it possible to reduce the work memories G and H by using only the input pattern times i and i-1 for the boundary part recurrence formula in the calculation of each block of DP matching. It is also possible to reduce the amount of data transferred. Furthermore, the recurrence formula for the main body of each block takes a form with tilt control, which has the effect of reducing the rate of misrecognition.

[Brief explanation of drawings]

FIG. 1 is a block diagram showing an embodiment of the continuous speech recognition device of the present invention, FIGS. 2, 3, and 4 are diagrams for explaining the principle of the present invention, and FIGS.
7 and 8 are partial block diagrams showing a part of the detailed configuration of the embodiment shown in FIG. 1, respectively. FIG. 9 is a time chart showing the time relationship of the operations in FIG. 1, and FIGS. d is a flowchart showing a series of operations in FIG. 100...Microphone, 110...Input section, 12
0...Input pattern memory section, 130...Standard pattern memory section, 150...G memory, 160...H memory,
170... Comparison circuit, 180... P memory, 190...
N memory, 200...T memory, 210...L memory, 220...judgment unit, 230...automaton storage unit, 240...control unit, 300...distance calculation unit, 31
0... Boundary DP calculation section, 320... Main body DP calculation section, 3
30...g memory, 340...h memory.

Claims (1)

[Claims] 1. In a continuous speech recognition device that recognizes speech in which a string of words specified by a finite state automaton is continuously uttered by performing DP matching with a standard pattern, there is an input pattern memory section that stores input patterns, and a standard memory section that stores the standard patterns. A pattern memory unit, an automaton storage unit that stores a state transition table Δ, which is a group of rules (p, q, n) that means that state transition p → q occurs when word n is input, and a final state group F. Then, in the state transition specified by the state p and word n, the input pattern within the area defined by the input pattern time i and the standard pattern time j.
Each point (i,
a distance calculation unit that calculates the distance d(i, j) between the feature vector of the input pattern of j) and the feature vector of the standard pattern;
a boundary DP calculation unit that calculates DP matching calculation for the boundary portion with block b using the first recurrence formula;
A main body DP calculation unit that calculates using a second recurrence formula different from the recurrence formula, and a body DP calculation unit that calculates the boundary DP calculation unit and the main body DP.
A continuous speech recognition device comprising: a determination unit that determines a combination of words that yields the minimum cumulative distance determined by the calculation unit.