JPS6140120B2 - - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION The present invention relates to a speech recognition device that is used to automatically recognize speech uttered by a person word by word, and to express the recognition results in, for example, printed text. The present invention relates to a real-time continuous speech recognition device that can perform continuous recognition over time.

As will be discussed later, conventional speech recognition devices cannot recognize continuous speech in real time, can only utter at most 10 words at a time, and usually only 4 or 5 words, and the calculation result Because the amount of calculation required to output the words is enormous, the device becomes complex and large-scale, which increases the calculation time and limits the number of vocabulary that can be recognized.

In view of the above, the present invention has no limit to the number of words that can be spoken at once, minimizes the amount of calculation required to output recognition results, and has an extremely convenient configuration, which ultimately reduces the size of the device. The main purpose of this invention is to provide a real-time continuous speech recognition device that can handle an increased number of vocabularies.

First, the schematic configuration of this type of speech recognition device is shown in FIG. 1 and will be explained.

The audio input that enters the audio input section A, such as a microphone, becomes a digital signal via an analog-to-digital converter B, and is converted into a digital signal by an analysis section C, such as a band pass filter or correlator (actually, the above converter B is also used). (generally, it contains the following information), is analyzed, and becomes a characteristic pattern. Generally, analysis is performed on approximately
This analysis is performed on digital signals in intervals of about 20 msec, but this analysis is usually performed with shifts of about 10 msec. Further, the number of bands of the band pass filter or the order of the correlator is usually about 10 to 20. Therefore, the output from analysis section C is
This is a time series of 10- to 20-dimensional vectors every 10 msec.

When the word to be recognized is not registered in the device, the output of the analysis section C is stored in the standard pattern storage section via the changeover switch S. On the other hand, if the vocabulary to be recognized is registered and stored in advance in the standard pattern storage section, the analysis output is passed through the switch S to the word recognition section E for recognition.

Here, the output from the input speech analysis section C is expressed by the following equation: { _(t,x) :1xL} (1). Here, t=1, 2, . . . , and the interval t is the interval at which the analysis is performed, that is, for example, 10 msec. In addition, x is, for example, the analysis unit C
If is a band pass filter, it represents the number of each band, and L is usually 10 to 10 from the above.
It is 20. At this time, the above _(t,x) indicates the power or magnitude of band number x at time t.

Next, one of the words generally registered in the standard pattern storage section D, the word with word name i, is expressed as {Z _(j,x) :1jT _i ,1xL} (2). This is called a standard pattern of word name i, and T _i indicates the pattern length of this standard pattern.

By the way, conventionally, when uttering one word or several consecutive words, the beginning and end of the word are specified, and the input pattern between the specified points of time is targeted for recognition. This recognition is completed by calculating the distance between the standard pattern group and the previous input pattern and determining the word name for the one with the smallest value.
Dynamic Programming or DP (Dynamic Programming) is considered to be the most effective method. In other words, in this distance calculation, as it is well known, the input pattern usually has a different speaking speed compared to the standard pattern, so the input pattern is expanded or contracted to fit at the most corresponding place. However, this is done using the dynamic programming method described above. This is commonly referred to as time normalization.

Under these circumstances, the disadvantage of the conventional method is that the distance calculation begins at the end of the utterance, and the time progression for distance calculation and the time progression of the target pattern are separate progressions. It is in. For this reason, real-time processing that is essentially necessary in speech recognition processing, that is, processing using a recognition method in which recognition is completed by the time the next time input is received, has not been possible. Therefore, up until now, what has been called ``real-time processing'' has only meant ``a short period of time after'' the utterance has finished. Not only that, but even if the results were to be produced ``shortly after'' the utterance, the calculations involved were enormous, leading to the drawbacks mentioned above.

In view of these deficiencies, the inventors have made efforts to obtain special knowledge as a premise for establishing the present invention.

In other words, the above defect is caused by the input pattern ((1)
This problem can be overcome if the distance from the word name i in accordance with the meaning of time normalization can be easily calculated at a certain time t of the formula (formula) by taking into account past inputs before time t. can. Now, suppose that such a distance is found at each time, and this is
Let Ai(t) be the value of λ, and then i ^* (t): ^min _i A _i(t) = A _i ^* _(t) (t)λ (3 ⁾ t), at time t, i ^*
It is possible to automatically determine that the word name (t) is recognized. i ^* in equation (3)
If (t) does not exist at time t, it can be assumed that no word has been uttered at that time (in other words, it is in the middle of being uttered) or that no word is uttered at that time. i
^* Once (t) is determined, the word name corresponding to it has already been known, so it can be displayed on the output section F in FIG.

The present invention is based on the knowledge that it is sufficient to obtain A _i(t) having the role described above, and the word recognition unit E is configured to automatically calculate this A _i(t). It is. As will be understood from the detailed description below, the method according to the configuration of the present invention can be referred to as continuous DP, which means that the previously described DP (dynamic programming) considers the past at each time. Because it is completed by
This makes it possible to recognize word names at each time, and in this sense, it is possible to recognize consecutive words.

An embodiment of the construction of the word recognition section of the device of the present invention is shown in FIG. 2, which is roughly divided into six circuit systems 1 to 6. Although the configuration shown in the second diagram relates to one word name i, this i will be omitted below for simplicity.

Circuit 1 is a partial distance calculation circuit, where T Calculate. This is, so to speak, a partial distance for calculating the entire distance, or more precisely, a partial distance of the analyzed audio input at a certain time to each point of the standard pattern. In this circuit 1, T units for calculating equation (4) are arranged in parallel, and this T corresponds to the standard pattern T _i according to equation (2) described above. Of course, Z given to circuit 1 is the first
It is determined by the output from the standard pattern storage section D in the figure.

Therefore, the output of circuit 1 is T Q _(t,j) , which is calculated every time the input { _(t,x) :1xL} comes in, and these are independently calculated. It is now possible to calculate.

The outputs from these T circuits 1 then enter a partial distance optimal integrator circuit 2. This circuit system 2 and the associated circuit system 4 (described later) have T _(t,j) corresponding to each of Q _(t,j) , j=1,
2 _{, .} For this calculation,
In this case, a quantity serving as an integral value of past optimal partial distances up to two units (stored by circuit system 3, which will be described later) and the output of circuit system 1 are involved.

This circuit is determined by the following simple calculation formulas (5) and (6).

The meanings of these expressions are as follows. formula
In order to understand the meaning of (5), it is necessary to understand the meaning of circuit 3 at the same time.

The circuit 3 stores the amount that is the past optimal integral value at a certain time, in this case two units of time before time t. This is determined together with the output Q _(t,j) of circuit 1.

The amount that becomes the optimal integral value at time t is determined by selecting three of the optimal integral values one or two units ago, and multiplying the output of each circuit by the multiplier 9 using the adder 8. By the addition operation, candidates for quantities that are the optimum integral value at three times t are created, and the comparator 10 selects the smallest one among these three, forming an integral determining circuit 7. This is equation (5)
This is the meaning of

The reason why the smallest of the three is selected is that the optimal distance is determined as the one that is the smallest possible distance from the standard pattern. This allows the speaker to speak at a different speed than when creating the standard pattern.

Now, this is P _(t,j) of circuit system 2, j=1,
2, ..., T, but how can the optimal distance be found just by doing this? Now, the number of quantities that will be the optimal integral value determined at time t is T, that is, P _{(t,j )} , j=1, 2, ..., T. It is clear that these are newly determined as those at time t, taking into consideration the quantities that are the past optimal integral values. This is because circuit system 3 is created by delaying P _(t,j) , j1, 2, . . . , T created in circuit system 2 using delay circuit G or the like.

From the above, to put it in precise terms, the circuit system 2 optimally integrates the partial distances to each point of the standard pattern calculated by the partial distance calculation circuit system 1, and The circuit system 3 obtains the optimal integral quantity corresponding to the optimal integral quantity, and the circuit system 3 obtains the past optimal integral quantity obtained by the above-mentioned partial distance optimal integral circuit system 2, which is necessary for calculating the quantity that becomes the optimal integral quantity. It can be said that it memorizes the quantity.

Now, let's consider the meaning of P _(t,T) shown at the right end of Figure 2. This clearly means that S={P _(t-2,T-1) , P _(t-1,T-1) , P _(t-1,T-2) } and Q _(t,T) It will be understood that it is determined from this. Therefore, any one of S, for example P
Let's take _(t-1,T-1) . This is what was at T-1 of circuit system 2 at time t' _(=t-1) . Now, return to this t′ _(=t-1) , and circuit system 2
Considering the situation, clearly this P _(t ′ _,T-1) also becomes S′={P _(t ′ _-2,T-2) , P _(t ′ _-1,T-2) , P _{( t} ′ _-1,T-3) } and Q _(t ′ _,T-1) . Therefore, any one of this S′, for example, P _(t ′ _-2,T-
If we take ₂₎ and also consider time t'-2, we get
This P _(t ′ _-2,T-2) is S″={P _(t ′ _-4,T-3) , P _(t ′ _-3,T-3) , P _(t ′ _{-3,T- 4)} } and Q _(t ′ _-2,T-2) .

If we repeat this operation until P(., 1) appears, we will find the one at the left end of the circuit system 2. And in the above
At the nodes S' to S'', S'' to S, S to S〓..., the optimal distance is the past distance and Q.
Since it is created with (·, j) in mind, it can be understood that overall, P _{(t, T)} takes the optimal distance from the standard pattern in equation (2).

That is, The distance P at time t that takes into account the past inputs is
It can be seen that it is created as _(t,T) . Also, P
_(t,j) , j=1, 2, ..., T are determined using only those in circuit system 1 and circuit system 3 at that time, so to speak, they are always self-procured at each time. It turns out that this is sufficient.

From the above, it should be understood that the meaning of P _(t,T) clearly represents the optimal distance.

Now, it can be seen that P _(t,T) is determined by equation (7), but in the middle of the calculation, Q _(t,j) is given a weight Kj (in this case, it is 2 to 3 from equation (5)). In addition, generally the length T of the standard pattern is different for different words, so P _(t,T
) , that is, it is meaningless when comparing P _(t,Tl) and P _(t,Tn) (l≠m), and at the same time, P (t,Tl) and P (t,Tn) (l≠m)
There is no meaning in comparing _(t1,T) and P _(t2,T) (t ₁ ≠t ₂ ). This is because at times t ₁ and t ₂ , P _(t1,T) and P
This is because the sum of weights Kj that make up _(t2,T) is different.

From these facts, Kj required to make P _(t,T)
If we find the sum of and divide P _(t,T) by this, we get
Both of the above two problems are solved. C _(t,j) in equation (6) is determined for this purpose. formula
It can be seen from (5) and equation (6) that C _(t,j) represents the sum of the weights Kj required until P _(t,j) is created.
That is, the sum of the weights Kj is created by circuit system 4 in the figure, and circuit system 4 is, in summary, the past optimal integral quantity corresponding to each point of the standard pattern. and the value obtained by multiplying the current partial distance to each point of the standard pattern by a predetermined weight, and calculate the amount that corresponds to each point of the standard pattern at that time and is also the optimal integral amount described above. Optimum weight for calculating the sum of the weights required to obtain the optimal integral amount when determining the optimal integral amount corresponding to each point of the standard pattern from among the candidates. This can be said to be a sum calculation circuit, in which the past optimal weight sum calculated by the circuit system 4 (stored in the circuit system 5) is used.

Thus, if we divide P _(t,T) by C _(t,T) corresponding to P _(t,T) , we get P _(t,T) /C _(t,T) =A _(t) (8), and this equation (8) clearly shows that it is a value that is sufficiently meaningful to compare between different words and different times. Incidentally, this calculation is performed in circuit system 6, but to express this accurately, the amount that is the optimal integral amount at a point equal to the pattern length of the standard pattern is It can be said to be a circuit that outputs the value divided by the sum of the weights as the distance from the standard pattern.

If we consider equations (5) and (6) as a practical problem, how to determine the value of circuit system 3 at t = 1 (i.e., when the device starts operating)? There is. This is P _(0,j) , P _(-1,j) , j=
1, 2, ..., T, ^C (0, j), ^C (-1, j), j
= 1, 2, ..., T is determined, P _{(0, j)} = P _{(-1, j)} = M, 〓 (9) j = 1, 2, ..., T C _{(0 ,j)} =C _(-1,j) =O, j=1,2,...,T, and if the value M is set to be a sufficiently large value compared to the value λ mentioned earlier, these initial conditions A involved in
Since the value of _(t) is sufficiently larger than λ, it can be assumed that there is no relationship in recognition.

As is clear from the above, the configuration shown in the second diagram corresponds to one standard pattern,
Now, considering N standard patterns, it is better to have N in the configuration shown in the second figure. This is because if there are N, they are P _{( t,j
)} , C _(t,j) can be calculated, and N pieces of A _(t) are created at the same time. However, even if there is only one configuration shown in the second figure, N pieces of A _(t) can be made in series, but it will take N times as much time.

However, in any case, in principle, when considering N standard patterns, N A _(t) will be created.

Although the purpose of the present invention has been achieved in this way, let us now refer to these as A _i(t) , i=1, 2, ..., N, and explain how words can be determined from these. put.

Now, suppose _tF is the time when the word with word name i is finished being uttered, whether it is uttered singly or as one of the words to be uttered continuously. However, this t _F is considered for the purpose of this explanation, and does not need to be determined in advance in any sense in recognition. Then, when you finish uttering the word with word name i,
A _i(tF) is smaller than A _k(tF) , k≠i, so A _i(t) during word utterance, t<t _F , and A after utterance.
_i(t) , t>t _FAlso , A _i(t) at the time when no voice is uttered
₎ is clearly smaller than

Therefore, although the value of λ has been determined in advance, based on the above equation (3), i ^* _(t) : ^min _i A _i(t) = A _i ^* _(t) (t) λ (3) (If there is no i ^* (t) that satisfies equation (3), then ^{i * (} t) = φ (empty)), i ^* (t) shows the recognition results at each time. If i ^* (t) = φ is not considered, i ^* (t) recognizes one of the N words, and at the same time also indicates the recognition time. In this sense, it can be said that the device of the present invention enables continuous word recognition.

However, i ^* (t) shows a recognition result that is not empty only at a certain time, and it does not mean that everything before and after that is φ (this is due to the ambiguity at the end of the utterance). will last for several hours. Therefore, i ^* _(t) = i ^* (t+1) =...i ^* (t+H)
In practice, recognition will be determined when at least H identical words are recognized continuously.

Compared to the configuration shown in FIG. 2, FIG. 3 uses not only Q _(t,j) but also Q _(t-1,j) , Q _(t-1,j) , Q _(t,j-1) , that is, the partial distance one time ago is also stored and used in the memory circuit 7, and although there are differences in software, the basic configuration is the same as shown in the second figure. Since the functions of the circuit system are satisfied, only circuit diagrams in which the same components are given the same reference numerals will be shown.

As detailed above, according to the present invention, although the number of vocabulary is limited, it is possible to continuously recognize an infinite number of words at once, and the recognition section is conveniently configured and has the same structure. It has the great effect of being able to be constructed from uniform elements, and is extremely useful in applications such as voice typewriters. Furthermore, as is clear from the configurations shown in the second and third figures, calculations and identification can be performed in synchronization with the master clock, and the amount of calculation for each clock can be reduced using equations (5), (6), and (3). It is possible to calculate the result in real time, that is, within the normal input interval (usually about 10 msec as mentioned above). can be produced. Incidentally, the amount of calculation has been reduced to about 1/5000 compared to the most excellent conventional method.

[Brief explanation of the drawing]

FIG. 1 is a schematic configuration diagram of a speech recognition device, FIG. 2 is a schematic configuration diagram of an embodiment of the word recognition section of the device of the present invention, and FIG. 3 is a schematic diagram of a second embodiment.
It is. In the figure, A is a voice input section, C is a voice input analysis section,
D is a standard pattern storage unit, E is a word recognition unit, 1 is a partial distance calculation circuit, 2 is a partial distance optimal integration circuit,
3 is a storage circuit for the past optimal integral amount; 4 is a circuit for calculating the sum of optimal weights; 5 is a circuit for storing the sum of past optimal weights; 6 is a distance output circuit from the standard pattern;
7 is an integral determining circuit, 8 is an adder, 9 is a multiplier, 1
0 is a comparator, and 11 is a storage circuit for a partial distance one time ago.

Claims (1)

[Claims] 1. A speech input section, an analysis section for the speech input, and a standard pattern storage section, which calculates the distance between the analyzed speech input and the standard pattern, and calculates the word name of the input speech. In a speech recognition device having a word recognition unit for identifying a word and an output unit for outputting the recognition result, the word recognition unit at least recognizes the analyzed speech input at a certain time to each point of a standard pattern. a circuit that calculates each partial distance; a partial distance optimal integration circuit that optimally integrates the partial distances to each point of the standard pattern to obtain an optimal integral amount corresponding to each point of the standard pattern; A circuit for storing the past optimal integral quantity obtained by the partial distance optimal integral circuit, which is used to calculate the quantity, and a circuit that stores the past optimal integral quantity corresponding to each point of the standard pattern. and a value obtained by multiplying the partial distance at the certain time to each point of the standard pattern by a predetermined weight, and calculate the optimal integral amount at the certain time corresponding to each point of the standard pattern. means for determining candidates; and an optimal weight sum calculation circuit for calculating the sum of weights required to determine the optimal integral amount when determining the optimal integral amount corresponding to each point of the standard pattern from among the candidates; , the circuit that stores the sum of the past optimal weights required to calculate the sum of the optimal weights at a certain time, and the optimal integral amount from the integrating circuit at a point equal to the pattern length of the standard pattern. a circuit that outputs a value obtained by dividing the sum of the optimal weights from the weight integration circuit at a point equal to the pattern length of the standard pattern as a distance from the standard pattern; Time continuous speech recognition device.