JPS62220996A - Voice recognition method and apparatus - Google Patents

Abstract

(57) [Summary] This bulletin contains application data before electronic filing, so abstract data is not recorded.

Description

[Detailed description of the invention] The invention will be described in the following I+18N order.

A. Industrial field of application B. Prior art C0 Problems to be solved by the invention E. Embodiment E-1. Voice recognition system E-1A. Main points (Figures 1 and 2) g-IB
, auditory model and its implementation (Fig. 4) F-IC, detailed matching (Figs. 3, 11) E-ID, basic fast matching (Fig. 12) E-IK, another fast matching ( v
, 15, 16) E-IF, Matching based on the first J label (Section 161) E-IG, Tree structure of a single note and high-speed matching (Figure 17) E-1) 1. Language model E-IJ, stack decoder (Figures 21 and 22)
Figure) E-1 shows the phonetic basic format structure 4 (Figure 3) E-IL
, Construction of basic phonemic forms (Figure 26) E-2, Selection of probable words from a vocabulary meeting by Pauling (Second Edition)
Figures 5 to 29) F0 Effects of the Invention 06 Table A, Industrial Field of Application This invention relates generally to speech recognition technology, and in particular to technology for forming a list of likely words selected from yellow words. It is related to.

B. Prior Art In the probabilistic processing method for speech recognition, a speech waveform is first converted into a label or string of phonemes by a speech processing device. The labels, each representing a type of sound, are typically selected from an alphabet of about 200 different labels. The generation of such labels is described in various papers, such as those listed below.

IETiEE [Proceeding of
thergaa), vol. 64, pp. 532-556 (1
Continuous speech recognition (con
tiny 5peeeh Recogniz
ionby 5 statistical method
The paper entitled ``Da)''.

There has been discussion about Markov model sound machines (also called stochastic finite state machines), which employ labels to perform speech recognition.

Markov models typically have an MI number of states and transitions between those states. Furthermore, one usually uses a Markov model with: (a) the probability of each transition occurring;

It consists of assigning certainty values for the individual probabilities of forming each label at various transitions. Furthermore, Markov
The model is based on the IEF on pattern analysis and mechanical intelligence.
F41i records (T EE3'E' T(an・5aet
Sonon Pattern Analysis
and Machine Intelligence
), Volume PAMI-5, Thickness 2. March 1983, L.
R, Barr (l1ahl), F.

Jel+nek, and R, L, -f
- A Maximum Likelihood Method for Continuous Speech Recognition by Mereer
oodApnroaeh to Continuo
5pesehReco(nition)” and other papers that refer to it.

In speech recognition, when a label string is provided by a speech processing device, a matching process is performed to determine which word in a search is most likely.

As set forth in commonly assigned U.S. Patent Application No. 672,974, filed November 19, 1984,
Phonetic matching consists of: (a) characterizing each word of the word zero by a sequence of Markov model sound machines, and characterizing each sequence of the sound machine representing an immediate word resulting in a sequence of labels generated by the rt++ speech processing unit; This is done by determining the likelihood of each word in the sound machine.
It is called.

As described in the above-mentioned US patent application, a word base form can be composed of phonetic sound machines. In this example, each sound machine preferably corresponds to an ff-voice sound pressure and has 7 states and 13 transitions.

Alternatively, each word base form may be formed as a sequence of phonemic sound machines. A phonemic sound machine is a simpler Markov model than a phonetic sound machine, and preferably consists of two states. Then, between the first state and the state of building 2, there is an empty field Wild(null t(ansit
ion ), and the first state and the second state v! !
There are also non-empty transitions in between such that a label can be generated from an alphabet of labels. The first state results in a self-loop in which a label can be formed. Then, during the training phase, statistics for each phonetic sound machine are determined. That is, for each phoneme, the probability of each transition and the probability that each label will be generated at each non-empty transition are calculated from the known pronunciation. , word base forms are formed by concatenating phonemic sound machines.

For systems that employ phonemic and phonetic elementary forms, the most likely word (or sequence of words) that corresponds to the sequence of labels produced by the speech processing device in response to unknown speech input is used. ) is the ultimate goal. One method for achieving this goal is described in the above-mentioned U.S. patent application. In particular, in that method:
The number of words is first reduced to a list of likely candidate words Vr,''!, taking the total number of words in the word buds, and then this list of words is considered in a more detailed matching or language modeling procedure. , then preferably the most likely word is selected; in reducing the number of candidate words, an approximation is applied to the sound machine according to the method taught by the above-mentioned US patent application; This provides fast processing without requiring excessive computation. When achieving word reduction, an approximate phonetic match is performed by operations based on a matching grid. This approximate phonetic match has been found to be effective and sufficient to determine which words should be subjected to more detailed matching and language model-based processing.

C1 Problems to be Solved by the Invention The purpose of the invention is to determine which words in common words have a relatively high likelihood in response to a sequence of phonetic labels generated by a speech processing device. The object of the present invention is to provide a fast and computationally simple method for and an apparatus for carrying out the method.

Means for Solving the D0 Problem The present invention teaches a non-conventional method for reducing the number of words to be examined in detailed matching. That is, the invention relates to a polling method in which a table is provided in which each label in the alphabet gives one vote for each word of the word cloud. reflects the likelihood that a word in has generated a given label.The value of the vote is calculated from the probability of the label output and the transition probability statistics obtained during the training session. .

According to one embodiment of the invention, the target word is selected when the label string is generated by audio processing. From the voting table, each label in the column is recognized and the vote for each label corresponding to the target word is determined. and,
All votes for the label for the target word are accumulated and combined to give a likelihood score. By repeating this process for each word in the word cloud, a probability score for each word can be obtained. A list of likely candidate words can be obtained from the likelihood scores.

? In the A2 embodiment, a second table is also formed containing the penalty each label has for each word in the word cloud.

The penalty assigned to a given label represents the likelihood of a word not producing that given label. Second
In the embodiment, both label candidates and penalties are considered in calculating the likelihood score for a given word based on the label sequence.

To account for length, the likelihood score is preferably scaled based on the label's aK being considered in k to calculate the word's likelihood score.

Furthermore, when the end point of a word along the generated label is not determined, the present invention provides a method to
The invention further provides that the likelihood score is to be calculated at successive time intervals.° For the target augmentation word, the likelihood score is preferably calculated for all other words in the word f. It specifies that the score with the highest likelihood be given compared to

According to the present invention, an instant word dictionary is provided in which each word is generated by a sequence of at least one probabilistic finite state sound machine, and a speech processing device generates a speech label in response to speech input. We will teach you how to select likely words from songs. The method is as follows: (a) Each label in the alphabet takes a single INK vote of forming a first table representing the first table. Further, the method preferably includes: (b) a penalty is assigned to each label for each word of the block, and the penalty assigned to a given label of a given word is (c) for a given label column, forming a table representing the likelihood of a given label not generated by the word model; determining the likelihood of a particular word including combining the votes with the penalties of all labels not in the column for that particular word.

Furthermore, the method preferably includes the above (
It also has a step ff of repeating steps a), (b) and (c).

If desired, the above method can be applied to the above US IM Demo No. 67.
It can be used in combination with No. 2974.

E. Example E.-1. Speech Recognition System E.-IA. Overview of Configuration FIG. 1 shows a schematic block diagram of a speech recognition system 1000. The system 1000 includes a stack decoder 1002, a speech processing unit 1004 connected to the stack decoder, an array processor 1006 used in performing fast approximate speech-to-voice matching, and a detailed speech matching. The array processor 1008, the language model 1010, and the work
station 1012.

In this system, the speech processing device 1004 is designed to convert a speech waveform input into a sequence of labels or phonemes that classify the corresponding sound type in a broad sense. This model is based on a unique model called the ears of K, which is described in Japanese Patent Application No. 1982-211229 filed by the present applicant.

Audio processing device f! Labels or phonemes from 11004 enter stack decoder 1002. Logically, stack decoder 1002 can be implemented by the block elements shown in FIG. That is, stack decoder 1002 has interfaces 1022, 102
4, 1026 and 1028, the audio processing device 10
04, has a search block 1020 in communication with a fast match processor 1006, a detailed match processor 1008, and a language model 1010, as well as with a work station 1012.

In operation, phonemes from speech processing unit 1004 are passed to fast matching processor 100 by search block 1020.
Leads to 6. This high-speed matching process will be explained later, but the above-mentioned US patent application file A729 related to the applicant
It is also mentioned in issue 74. Simply stated, the purpose of this match is to find the most likely 4 for a given column of labels.
The purpose is to decide on five words.

Fast matching is designed to test for word augmentation in a word association and reduce the error of candidate words for a given input label sequence. This fast matching is based on stochastic finite state machines, also called Markov models.

Once the number of candidate words has been reduced through fast matching, the stack decoder 1002 communicates with a language model 1010, which preferably detects any triglyphs (tri-g(am)) present. Based on the contextual
) determine certainty.

Preferably, based on the calculation results of the language model, words that have a reasonable probability of being spoken words after detailed matching are considered in the candidates for fast matching. This detailed matching procedure is described in U.S. Patent Application No. 67, cited above.
No. 2974.

The detailed matching procedure is carried out by an Una Markov model sound machine, such as the machine shown in FIG.

After that detailed matching, the language model is preferably invoked again to determine the likelihood of the 4j words. Stack decoder 1002 of the present invention that utilizes information obtained by fast and detailed matching and application of language models
is designed to determine the most likely path-spanning sequence of words corresponding to the generated label sequence.

Two conventional techniques for finding the most likely string of rabbit candy are Viterbi decoding and stack decoding. These techniques are described in I., R., Barr, F., supra.

Zielinecz and R.L. Mercer. In particular, Viterbi decoding is described in Chapter 5, and Kishi-stack decoding is described in Chapter 6.

In single stack decoding techniques, paths of different lengths are listed in the augmented stack depending on their certainty. Single stack decoding must take into account the fact that the certainty depends on the length of the part or path, ie normalization must be employed in general.

Guiterubi's technique, on the other hand, requires no such standardization and is generally practical for small tasks.

Another technique is to examine each possible combination of words as a collapsible word string and determine which combination has the highest probability of producing a generated label string with fewer than K words! Decoding can also be done per system. However, the amount of computation required for this technique makes it impractical for large language systems.

Stack decoder 1002 essentially serves to control other block circuits, but does not perform many operations. Additionally, the stack decoder 1002 is preferably an "l/1rtual Machine/System
mProduct IntroductionRe
It includes a 4341 processor running under the TBM VM1570 operating system as described in publications such as Lease 3 (19Fl3). The array processor that performs the phase coverage calculations is implemented by a commercially available floating point system (FPS) 190L.

A novel technique with multiple scooking and a unique decision strategy to determine the best word string or path, R
, Barr, F., Jelinek, and RoL, Mercer, and will be discussed later.

E-IB, Auditory Model and Its Implementation@4 In FIG. 4, a particular embodiment of an audio processing device 1100 (labeled 1004 in FIG. 1) is illustrated. In this figure, the audio wave input (e.g., normal conversation) is analog-digital (
A/D) is input to the converter 11a2, and the A/D
Converter 1102 samples its input at a given rate. The typical sampling rate is
One sample every 50 microseconds. A time window generator 11o4 is provided to shape the ends of the digital signal from the A/D converter 1102. The output of the time window generator 1104 is a fast Fourier transform circuit (FF
T) 1106, the FFT 1106 provides a frequency spectrum for each time window.

The output of FFT 1106 is then labeled 7172. .

is processed to generate y. To generate the label, feature:y4R block 1108. cluster block 1110, prototype block 1112,
Four circuit blocks, label creation block 1114, work together. In the generation of labels, the protocol is determined as a point (or vector) in space based on the selected % signature or speech force, which then gives the protocol a corresponding point (or vector) in space that can be compared. is characterized by the same selected characteristics.

In particular, when determining the protocol, cluster block 1
110 classifies the set of points as distinct clusters. Methods for determining clusters are based on probability distributions, such as Gaussian distributions, applied to speech. A prototype for each cluster related to the centroid or other characteristics of the cluster is generated by prototype block 1112. The generated protocol and audio input, characterized by the same selected features, are then
Input to label creation block 1114.

Label creation block 1114 performs a comparison procedure that assigns one label to a particular audio input.

Selection of appropriate features is an important factor in obtaining labels representing speech (speech) wave input. The audio processing device described herein is an improved and cured special W song selection block 110.
It has 8. According to this speech processing device, an auditory model is obtained, which is applied to the speech processing device of the speech recognition system.

To explain the M-party model, reference is made to FIG.

FIG. 5 is a diagram showing a part of the inside of the human ear.

The internal hair cells 1200 are fluid-holding channels 1
204 is shown with an end 1202 projecting into it. Upstream of the inner hair cells are the outer hair cells 120
6, these also have passages 1204VC projecting ends 12
08. Nerves for sending information to the brain are connected to the inner hair cells 1200 and the outer hair cells 1206. In particular, neurons undergo electrochemical changes that result in electrical impulses being sent via nerves to the brain for processing. This electrochemical change is
It is stimulated by mechanical movement of the "basement membrane" 210.

It is conventionally known that the base [1210] acts as a frequency analyzer for audio wave input, and locations along the basilar membrane 1210 respond to individual critical frequency bands. In turn, different portions of basilar membrane 1210 respond to corresponding frequency bands, which affects the perceived loudness of sound wave input. That is, even if two sounds of similar intensity occupy the same frequency band, the two sounds are perceived to be louder if they are in different critical frequency bands.

It has been found that there are approximately 22 critical frequency bands determined by the basilar membrane 1210.

In order to match the frequency response of the basilar membrane 1210, the audio processing device 1100 of the present embodiment has a 1. Preferably, the audio wave input is separated into some or all of the critical frequency bands, and the signal components are individually examined for each separated critical frequency band. This function appropriately filters the signal from FFT 1106 (FIG. 4) so that during feature selection block 1108,
This is achieved by providing a separate signal for each critical frequency band being tested.

The individual inputs are also blocked within a time window (preferably 25.6 milliseconds) by a time window generator 1104. Therefore, feature selection block element 1108
preferably includes 22 signals, each representing the sound intensity in a given frequency band corresponding to one time frame.

Filtering is preferably performed by a conventional critical band filter 1300, shown in diagram P6. The individual signals are then given equal loudness.
Processed by converter 1502. This converter 1302 accounts for changes in perceived loudness as a function of frequency. In this regard, note that the sound of PFl at a given dR level at one frequency may differ in audible loudness from a second sound at a different frequency and the same dR level. I want to be Converter 1302 may be based on experimental data to convert the signals in each frequency band such that the various frequency bands are measured on the same magnitude scale. For example, converter 1302 is preferably adapted from the 1933 Fletcher and M.
(Unson), and by making some changes to it, we can make a correspondence between the bounce of the sound and the isoline curve of the loudness of the sound. The modified results of these studies are
It is shown in the fPZ diagram. That is, (100
As can be seen from the isoline curve passing through the point 0Hz, 40dR), the sound of 1000)Tz, 40dB is 100Hz, 6
It is comparable to a 0 dB sound at a level of 1 loudness.

Converter 1302 preferably adjusts the sound volume according to the curve of FIG. 7 in order to provide equal sensitivity regardless of frequency.

As can be further seen from the differential diagram, not only is there a dependence on frequency, but sound intensity and sound volume do not correspond. That is, changes in sound intensity or amplitude are not necessarily reflected by similar changes in audible loudness. For example, at 100 Hz, if the sound intensity changes by 10 dB at 110 dB, and if the sound intensity changes by 10 dB at 20 dB, the change in the audible sound volume is much larger in the former. This difference is a function of magnitude scaling, which compresses magnitude in a predetermined manner.
Processed by block element 1304.

Preferably, size scaling block element 1304
The intensity P is compressed to its cube root P1/3 by replacing it with sane, which is the measured value of the loudness of the sound produced by the phono (ohon).

Figure 8 is a diagram showing empirically obtained Thorn vs. Phone data. By employing this method, the above model is almost accurate for a large field of view of conversational speech. Note that 1 sone is defined as a sound level of 1 KT (z, 40 dB).

Referring again to FIG. 6, there is illustrated a time-varying response block element 1306 that processes isolineally, thickness-scaled signals corresponding to each critical frequency band. Specifically, in each time frame, the neural firing rate (neu(al)
firinfC(ate)f is determined. According to this speech processing device, the neural firing rate is expressed as follows.

f = (So + DL) n (1)
In this equation, n is the amount of neural transmission signal (neurot(ansmitter)), So is the spontaneous firing constant related to neural firing independent of acoustic wave input, L is the S1 constant of magnitude, and D
is the displacement constant. That is, (So)n corresponds to the spontaneous neural firing "IgK" that occurs regardless of whether there is a sound wave input, and DLn corresponds to the firing rate due to the sound wave input.

Importantly, the value of n is characterized by this audio processing device as time-varying based on the following relation: dn/dt=Ao-(So+Sh+I)L)n (2
) In this equation, Ao is the replenishment constant and Sh is the decay constant of the spontaneous neutral transmission 1S. The novel relationship shown in Equation (2) is that neural transmission signals are being formed at a certain rate and are characterized by (a) collapse (ShXn), (h) spontaneous firing, and (c) acoustic waves. This takes into consideration the fact that it is lost due to neural firing due to input. These modeled actual positions are the fifth
As shown in the figure.

Equation (2) also shows that this audio processing arrangement is non-linear in the sense that the next 1F and the next firing rate of the neural transmission signal depend multiplicatively on the current conditions of the neural transmission signal ridge at least. It also reflects the fact that In other words, the amount of neural transmission signals at time (t+Δt)K is the sum of the neural transmission signals at time t, K(dn/dt)·Δt, and when expressed in the formula, n ( t+Δt)=n(t)+(dn/dt)
φΔt(3) Equations (1), (2), and (3) indicate that the above-mentioned auditory system is adaptive over time, and that the signals on the auditory nerve are nonlinear with respect to the acoustic wave force. do. This paper describes a time-varying signal analyzer that takes advantage of this fact. In this respect, this audio processing device is designed to better match the apparent temporal changes in the nervous system.
A first model that embodies nonlinear signal processing in a speech recognition system is presented.

In order to reduce the number of unknowns in equations (1) and (2),
This audio processing device uses the following equation (4) to which a constant sound volume LK is applied.

So+Sh+DL=1/T
(4) In this equation, T is the time after the audio wave input is generated.
It is a measurement of the time it takes for the auditory response to drop to 67% of its maximum value. T is a function of loudness, and according to this audio processing device, is a value obtained from a graph showing the decline in response to various loudness levels. That is, when a sound of a certain loudness is generated, it generates a first high level response, after which the response declines to a steady state level with a time constant T.

If there is no audio wave input, T=To (about 50 milliseconds)
It is. In the case of Tj, m a x, T:=T*ax (
(about 30 milliseconds). By setting 8o=1, when L=O, I/(So+Sh) is 5 centiseconds. Also, L is Lmax, and Lmax=
20 non, equation (5) becomes as follows.

So+Sn+D(20)=1/30 (5)
According to the above data and formula, p, So and Sh are calculated by formula (6) and (
7) is determined as follows.

So=DLmax/(R+(DLmaxToR) −1
) (6) S h = 1 / T o - S o
(7) f s l L=Lm
ax where R=near 7 here surface-(8) where fs l +tdn/dt=o represents the firing rate at a given sound level.

R is the only variable left in the audio processor. Therefore, to change the performance of the audio processing device, only R can be changed. That is, R is usually a single parameter that can be adjusted to change performance, which means minimizing steady-state effects versus transition effects. Minimizing steady-state effects means that frequency response differences, speaker differences, background noise, and distortions that affect the steady-state sugar portion of speech rather than the transition portion of speech are generally the same. This is desirable because it produces a non-constant output pattern for the audio input. The value of R is
Preferably, it is set by a button to optimize the error rate of the entire speech recognition system. The appropriate value thus found for K is R=15. Then, So =0.
0888.5h=r), 1111, and D is 0.00
It becomes 666.

Referring to Fig. 9, a flowchart of this audio processing device is shown. In the hole 9 diagram, preferably 20KH
The digitized audio within a time frame of 25.6 ms, sampled at
The output from the Hanning window 1320 is passed through the sky 1320, and the output from the Hanning window 1320 is preferably input to a 7-lier transformer (T) F
'T) 1322. The thus transformed output is filtered by element 1324 to provide an output with an intensity density corresponding to each of at least one complete frequency band (preferably all critical frequency bands or at least 20 critical frequency bands). . This intensity density is then converted to loudness levels by a logarithmic conversion step 1326. This is easily accomplished based on the graph of FIG. The subsequent processing includes threshold updating processing at step 1360, and is shown in FIG.

In FIG. 10, the sensory threshold Tf and the auditory threshold Th are determined in step 1340 for each filtered frequency band m, with Tf=120 dB%Th=Od
The initial power is set to R. After that, the audio counter and all frame registers.

The histogram register is reset at step 1342.

Each histogram includes bins (bsn) representing the number of samples or counts between which the intensity or similar measurements within a given frequency band lie around individual WJs. The current histogram preferably represents, for each given frequency band, the number of centiseconds during which the loudness is in any of a plurality of loudness ranges, e.g., a third frequency band. In this case, the strength is 10 d
There may be a 20 centisecond interval between B and 20 dB.

Similarly, in the %200th frequency band, 50dR
and 60 dB between 1 and 1 out of 1000 centiseconds
There may be 50 centiseconds. Percentiles are obtained from the total number of samples (or centiseconds) and the counts contained in the bins.

In step 1344, the frames from the filter outputs of the individual frequency bands are checked and the appropriate histogram bins are incremented, one for each filter, in step 1346. The total number of bins with amplitudes greater than 55 dB are summed for each filter (ie, frequency band) in step 1548 to determine the number of filters that represent the presence of speech. Then, select the minimum number of sounds (for example, 2
If there are no filters (6 out of 0), then the next frame is checked in step 1344.

If, in step 1350, there are a sufficient number of filters representing speech, then in step 1352, a speech counter is incremented. , voice color/ri, step 1354
is incremented in step 1352 until 10 seconds of audio occur, then new values of T and Th are determined for each filter in step 1356.

The new values of T, and Th are determined for a given filter as follows. For T, the 55th from the top of 1000 bins (i.e., the 96.5th percentile of audio)
The dB value of the bin holding samples of is defined to be RIN□. Next, kT, is set as T,=BINH+40dR. For Th, the dB value of the bin holding the 0.01th percentile sample from the lowest bin is B
Defined to be INL. That is, RrNL is the bin that is the largest number of samples in the histogram excluding the number of samples classified as speech. Then, T is defined as Th = B I NL - 30 dB.

In FIG. 9, the audio amplitude is determined in step 13 based on the thresholds updated in steps 1330 and 1332.
32, it is converted and scaled. The method for doing this is as described above. Another way to obtain and scale the samples is to take the filter amplitude "a" (after the bins have been incremented) and convert to dB based on:

aan=20Jng (a)-10(9) Each filter amplitude is then scaled to a range between KO and 120 to give equal magnitude according to the following equation:

a @q””120 (a dB-Th)/ (Tt
Th) a@qj can be changed from the magnitude level (Pon) to the approximate magnitude at Thorn (K by correlating the signal of IKH, with 1 at 40 dB) using the following formula: 4- To be done.

L''=(a @Q'-30)/4 (11) Then, the size at the Thorn is approximated as follows.

L (approximation>=1o(t)/20 (12) The size of the sown is then taken as input in step 1
Equations (1) and (2) IC-7'- are obtained in step 1534, and from this K, the output firing rate f corresponding to each frequency band is determined (step 1535). When there are 22 frequency bands, a 22-dimensional vector characterizes the audio wave input by sequential time frames. but,
-For resolute, the customary mel (met: unit of height of i)
Twenty frequency bands are examined using a filter bank scaled by .

Before processing the next time frame (step 1336), the next state of n is determined in step 1337 based on equation (3).

Audio processing mentioned above 4! The arrangement is improved in applications where the filing rate f and l)n of the neutral transmission signal have a large DC pedestal. That is, the equation shown below for reducing the pedestal height is obtained, where the dynamic range of the f and n equation terms is important.

First, in the steady state, when there is no audio wave input (L=0), equation (2) is solved as follows for the internal state pn' in the steady state.

n' = A / (So + Sh)
(13) The internal state of the scene n (t ) of the neural transmission signal is expressed as follows by the constant part n' and the time-varying part nII.

n(t)=n'+n''(t)
(14) By combining Equation (1) and Equation (14), the following equation for firing rate MF is obtained.

f (t) = (So+DL) (n'+6" (
t)) In this 2, the term 5oXn' is a constant, and all other terms are the time-varying part of n, and T
)L.

Since the subsequent processing concerns only the squared difference between the output vectors, the constant term can be ignored. Therefore, formula (13)
Using
t), then f” (t)=(So
+T)L)n” (t)+DLA/(So+Sh)
(16) Considering equation (3), the next state is n(t+Δ1)=・n'(t+Δt)+n'' (t+
Δt) (17)=n' (t+Δt)+n” (t)+
(Ao−(So+Sh+11L)n(t))Δt
(18)=n”(t+Δt)+n”
(t)-(So+5h)n'(t)-r)Ln'rt)
Δt+Ao△t−(S o +S h +D L
)nII (t)Δt (
19) In equation (19), if all constant terms are ignored, then K
From this, the following formula is obtained.

n lJ (t+Δt)=n” (t) (1-8hΔ
t)−r”(t)Δt (20) Equation (15
) and (20) now constitute the output equation and state update equation, respectively, that are applied to each filter every 10 ms time window. The result of applying these equations is a 20-element vector every 10 m'),
Each component of the vector is a mel-scaled filter.
Corresponds to the firing rate of the individual frequency bands in the bank.

Regarding the example shown earlier, f% dn/dt,
and n(t+1) are respectively the spatial expression of the firing rate f and the following form 1i11n(t+Δt)
stipulates.

It should be noted that the values contributing to various equation terms (e.g. t.

= 5 centiseconds, tL, Tl = 3 centiseconds, Ao = 1
, R=1.5. T, mar==20) may be set to other values, then Sol Sh and D have the preferred derived values of 0.088B and 0.1111, respectively.
Note that the values are different from 1 and 0°Do 666.

This audio model was implemented by the present inventor on floating point system FPS190L hardware using the PI,/I programming language, but various other software or hardware could be used.

g-jC, Detailed Verification Figure 3 shows a sample detailed verification sound machine 2000. Each detailed matching sound machine has (a) a plurality of states st, (b) a plurality of transitions tr(SjlSi) and (
Note that the transitions include those with different state quantities and those that return from a certain state to itself, and each transition is associated with a probability); The actual label probability and K corresponding to each two pels
Therefore, it is a stochastic finite state machine that cannot be characterized.

In FIG. 3, the detailed matching sound machine 2000 has seven states S1 to S7 and 13 states tr1 to tr15.
is given. As can be seen from FIG. 5, the sound machine 2000 has three transitions tr11 indicated by broken lines,
It has tr12 and tr15. In each of these three transitions, the sound is 1 without producing a label.
changes from one state to another; such a transition is therefore called a zero transition. On the other hand, labels can be generated along the transitions from tr1 to tr10. Special Ktr1~tr
For each of the 10 transitions, one or more labels are
They can have individual probabilities of being generated. Preferably, for each transition there is a probability associated with the label that can be generated in the system. That is, if there are 200 labels that can be selectively generated by the audio channel, then each transition (that is not a zero transition) has 200 ``actual label probabilities'' associated with it. and each probability corresponds to the probability that the corresponding label is generated by the sound at a particular transition.In Figure 3, the actual label probability for transition tr1 is p[t') (t is It is an integer from 1 to 200 and is displayed by a symbol indicating the label number.For example, in the case of label 1, the detailed verification sound machine 200
There is a probability P[1] that 0 generates label 1 at transition tr1. The actual various label probabilities are stored in labels and their corresponding transitions.

Now, corresponding to the given sound, the label column 7112y3.
. . is given as detailed m verification sound machine 2000Vc, the verification procedure is executed. The procedure associated with the detailed verification tone machine will be explained with reference to FIG.

FIG. 11 is a grid diagram of the sound machine of FIG.

Similar to the sound machine of FIG. 3, this grid diagram shows zero transitions from state S1 to state nS7, and transitions from state S to state S2 and from state S1 to state ns4. , and transitions between different states are also shown. This grid diagram is also horizontally scaled in time. In FIG. 11, the starting time probabilities q 1 and ql represent, for a sound, the probability that the sound has a starting time at time 1=1 and t 2 =t 1, respectively. Each start time t□ and tlVc
, various transitions are shown q. Note that the time interval between successive start times is preferably equal to the length of the time interval of the label.

If the detailed match sound machine 2000 is employed to determine how close a given note is to the input string label, the distribution at the end of that note is determined, which determines the match value for that note. used to make decisions. The concept of being based on the distribution of end times is common to all embodiments of the sound machine described here in connection with the matching procedure.

In forming the end point distribution for performing detailed matching, the detailed matching sound machine 2000 performs rigorous and complex calculations.

With reference to Figure 11, let us first consider the calculations required to have start and end times at time t ''Rt □. Therefore, the following probability is applied.

Pr (87,t”t□)=Q□T (1→7)+
Pr (S2, tF t□) T (2→7) + Pr (S 3, t=to)
T(3→7) In this formula, Pr is the probability of being in the state shown in parentheses, and T is the probability of transition of the state number shown in parentheses in the direction of the arrow. Equation (21) displays the individual probabilities corresponding to the three conditions such that the end time occurs at t=to. moreover,
It can be seen that the end time at t=to is limited to the occurrence of [, case, state %S7 in this example.

Next, noting the end time t = t I K, Akira can see that calculations must be performed for all states except state S1. State S1 is started at the end of the previous note. Note that for convenience of explanation, only calculations related to state S4 are shown.

In the case of state S4, the calculation formula is K as follows.

Pr(S4, t=t1)==Pr(S1,t=t
□)T(1-+4)Pr(yl !1→4)tenPr(S
a, 1=10) T (4→4) Pr (y 114→4) In other words, equation (22) shows that the probability that the sound machine is in state S4 at time 1=11 is the sum of the following two terms. It depends. That is, one of the terms is (a) t=to
De-state! 1iilWith a probability in S1, from state S to state S
4 multiplied by the probability that a given label y1 is generated when there is a transition from state S1 to state S4, and the other term is (b) time t = t □
The probability that the shape P4S4 is the shape! It is the probability of transitioning from PS4 to itself multiplied by the probability that a given label Y1 will be generated when there is a transition from state S4 to itself.

Similarly, for other states n (except for state pS1), calculations are performed to find the probability that the sound is in a particular state at time t:t1. In general, when determining the probability of being in a particular state at a given time, the detailed Teruim's hand vc is defined as (a) each previous state that has a transition leading to that particular state, and its Recognize the individual probabilities of each previous state; (b) for each previous state, K the labels that must be generated at the transition between each previous state and the current state to match the label sequence; and (c) combining the probability of each previous state with the individual value representing the label probability to find the probability of the particular state for the corresponding transition. The overall probability of being in a particular state is determined from the probability of that state over all transitions that lead to that state. Note that the calculation for state 87 is that the sound is state 1F! Note that when ending at s7 we include terms for three zero transitions that allow the note to start and end at time t = t1.

Analogous to the probability calculations for times 1=1 and t =t I K, probability calculations for further sequences of finishing times are preferably performed to form finishing time distributions. The value of the ending distribution corresponding to a given single 9fk indicates how well that given single note matches the incoming carapel. How well does a word match a sequence of input labels? When a decision is made, the plurality of sounds representing the word are processed. Each note then generates an ending distribution of probability values. The match value for that Kishi sound can be obtained by adding up the ending probabilities and then taking the logarithm of the sum. The searched start time distribution of the next single note can be obtained by, for example, dividing the multi-value by the total value and scaling each value so that the sum of the scaled values becomes 1 and standardizing the end time distribution. You can get it.

Note that there are at least two methods for determining the number of phones to be tested for a given word or word string. First, the depth first method
) In the basic form? In G, moving subtotals are calculated sequentially for each note (-). Then, if the subtotal is 1<
If it is found to be less than or equal to the corresponding scheduled threshold, the calculation ends. Alternatively, breadth-first method (breadth
In the first method), calculations are performed on the similar phonetic positions fffK of each word. Sf
fc, that is, the calculation for the second phonetic sound of each word is performed as a calculation following the first phonetic sound of each word, and so on. In breadth-first construction, the same number of single sounds ′ in different words
The calculated values along the lines are compared at the same relative pitch along those notes. In either of these methods, the single M (single or plural) for which the sum of matching values is maximum is the object to be determined.

Detailed matching is performed in APAL (Array Processor Assembly Language). APAL is a subsidiary of Floating Point System Co., Ltd. (Floatin (Po1
nt System, Inc.) 190L-specific assembler.

The detailed matching includes the actual probability of each label (i.e., the probability that a given phone produces a given label y in a given transition), the transition probability for each sound machine, and the It should be appreciated that considerable memory is required to store the probability that a given note will be in a given state at a given time after the start time. FPS (Floating Point System) mentioned above
190L corresponds to an end time, a match value based on the sum (preferably the logarithm of the sum of the end point probabilities), a start time based on a previously generated end point probability, and consecutive single sounds in the word shu. be set up to perform various calculations, such as matching scores for words based on matching values.

Furthermore, the detailed matching preferably takes into account tail probabilities in the matching procedure. Terminal probability is a robust measure of the certainty of a label, independent of words. In a very simple example,
A given terminal probability corresponds to the probability of a label following another label. This likelihood is easily determined, for example, from a sequence of labels generated by several sample voices.

Therefore, detailed matching requires sufficient storage capacity to accommodate the basic form, the statistics for the Markov model, and the terminal probabilities. For example, for a vocabulary of 5000 words, each word consisting of approximately 10 sounds, the basic format requires 5000×10 memories. Also, if there are 70 different phones (with a Markov model attached to each phone), 200 different labels, and 10 transitions with each label having a probability of being generated, then the statistics are: 70x10x200 positions are required. but,
The sound machine is preferably divided into three parts, namely a beginning part, a middle part and an end part, accompanied by corresponding statistics (preferably - includes a continuous part VC 5 self-loops) . Therefore, the required amount of Kii is 70
×3×200. Regarding the terminal probability, 20
A storage location of 0X200 is required. In this array,
Integer to 50 and floating point storage to 82 give satisfactory operation.

Note that detailed matching can be performed by using phonemic sounds rather than phonetic sounds.

E-ID, more than basic high-speed matching, detailed matching is computationally expensive;
A basic fast match and another fast match are performed that reduce the amount of computation required at the expense of some accuracy. This fast matching is preferably used in combination with detailed matching,
That is, fast matching lists probable candidate words from word t, and detailed matching is performed on at most the listed candidate words.

Fast approximate voice matching techniques are the subject of the aforementioned commonly assigned US Pat. No. Hf'R672,974. In a fast approximate phonetic matching technique, each sound machine is preferably unilinearized by replacing the actual probability corresponding to each label at all transitions in a given sound machine with a specific replacement value. Ru. A particular replacement value is preferably a match that is achieved by detailed matching when the replacement value does not replace the actual label probability when the replacement value is used. The song is chosen to be an overestimation of its value. One way to ensure this condition is by choosing each replacement value such that any probability corresponding to a given label in a given sound machine is no greater than the replacement value. By replacing the actual label probabilities in the sound machine with corresponding replacement values, the amount of computation required in determining the match score for a given word is significantly reduced. Furthermore, the replacement value is preferably an overestimation so that the matching score obtained is no less than the score that would have been determined without the replacement.

In a particular embodiment, such as performing phonetic matching in a linguistic decoder with a Markov model, each of its sound machines is designated by a separate line to include (h) to (cl).

(q) Transition path between n number of states and the state W.

(bl Transition tr(SjlS
i). Each of them represents the probability of transition to state Sj given the current state %St;
and old may be in the same state or in different states.

(c) Actual label probability P(Yk　l　i−+j　).

Each label probability P(Ykll→j) represents the probability that label yk is generated by a given sound machine on a given transition from one state to its next state, where k
is a subscript to identify the label. Each sound machine has (d) each Yk? r single specific value P
' (Yk ) and (e) each actual output probability P(Y
kli→j) by a single specific value p' (Yk) assigned to the corresponding Yk.

Preferably, the replacement value is at least equal in magnitude to the actual value of the corresponding Yk label at any transition of the particular sound machine. A fast matching procedure is employed to determine a list of 10 comparable word candidates corresponding to the input label and selected from the etymology as the most probable word. A language model is preferably applied to these word candidates and detailed matching is performed.

In this way, if the number of words to be considered through detailed matching is reduced to about 1 word, the calculation cost will be significantly reduced and the accuracy will not be reduced.

Basic fast matching is performed by substituting a single value for the actual label probability of a given label at all transitions such that a given label can be generated in a given sound machine. Simplify.

That is, regardless of the transition in a given sound machine whose label has a probability of occurrence, the probability is replaced by a single specific value. This value is an overestimation value that is at least as large as the maximum probability of a label occurring in a given sound machine transition.

By setting the replacement value of label probability to be the maximum value of the actual label probability for a given label in a given sound machine, the match value generated using basic fast matching is at least as detailed as It is as large as the matching value obtained by employing matching. Thus, basic fast matching typically overestimates the matching value of each button so that more words are widely selected as candidates. That is, words that are considered candidates based on detailed matching also pass this basic fast matching-based criterion.

Referring to rod 121'21, the sound machine for basic fast matching is shown. Labels (also referred to as symbols and phonemes) are input into the basic fast matching sound machine along with the starting time distribution. This starting time distribution and label string input are similar to those input to the detailed matching sound machine described above. It should be appreciated that the onset time distribution may in some cases not be a distribution over multiple times, but may represent, for example, the exact time following a period of silence at the onset of a sound. However, if the audio is continuous, it is used to determine the starting point distribution (which will be explained in more detail below). , 1f
Machine 3000 generates an end time distribution and matching values corresponding to particular sounds from the end time distribution. The matching value of a certain word is determined as the sum total of the matching values of phonemes (at least the h sound at the beginning of the f sound of the word).

Referring to FIG. 13, the basic fast matching operation is illustrated diagrammatically. This basic fast matching operation only concerns the starting time distribution, the number or length of labels produced by the phone, and the replacement value "Yk" associated with each label Yk. Then, for a given tone machine Kt= The basic fast matching replaces transition probabilities with length distribution probabilities by compromising all actual label probabilities for a given label in the This eliminates the need to have an actual label probability (which may be different for each transition) and a probability of being in a given state at a given time.

In this regard, the length distribution is determined from a detailed matching model. In particular, for each length in the length distribution, a detailed matching procedure checks each state individually and, corresponding to each state, (a) given a particular label length; and (b) determining the various transition paths in which the currently examined state may occur, with no involvement in the output along the transition. The probabilities corresponding to all transition paths of a given length leading to each given transition path are summed, and then the probabilities corresponding to all states of that given given length in that distribution are summed. The sum is added.

The above procedure is repeated for each length. According to the preferred form of the matching procedure, these operations are performed using trellis diagrams known in the field of Markov models.
This is done with reference to diag(am).

That is, for a two-transfer path that has many branches along the lattice structure, the calculation for each common branch needs to be done only once, and the calculated value is added to each route containing the common branches. reactor.

The button 213 includes two limitations as an example. Assume that the length of the label generated by the # sound in ff1′1 is one of 1o1, 1.12, and 0.1, 2.3 with a probability of 16, respectively. In addition, the starting time is also limited, which allows each person to have a probability of 03% Ql
, Q2 and q3 are only possible. With these limitations, the following equation determines the end time distribution of the target note.

'o=qo'.

'1''Q110+qO1101 '2'Q2 'O+Q1'I n2+Qo '2'J
"2'3="510+q211 "3"Ql 12
"2"3+q013pI P2 T13 '4"4311°4 +Q212 p5 p4+Q11
3T12113 o4 '5''Q312941)5+Q215 Dr, p4
T15'6=qs 13p4 ''5 p Considering these equations, it can be seen that Φ3 includes terms corresponding to each of the four starting times.

The term 1 represents the probability that the augmentation starts at time t '' t 3 and produces a zero label length (i.e., the shore sounds start and end at the same time). Pit 2
The term represents the probability that a single note starts at time t=t゛2, the label length is 1, and label 3 is generated by that note.The term 3 represents the probability that a note starts at time t=t starts at t1 and has label length 2 (i.e. label 2
and 6) represents the probability that labels 2 and 3 are generated by a single note. Similarly, the term for Tate 4 indicates that the single note is at time t=t
o, the label length is 3, and represents the probability that the three labels 1, 2, and 3 are generated by a single phone.

Comparing the total 'IIL# required for basic high-speed verification with the amount of calculation required for detailed verification, it can be seen that the former is relatively simpler than the latter. In this regard, the Yk value remains the same for each representation of all equations, as does the label length probability. Furthermore, given the length and start time limitations, later calculations of the end time become easier. For example, in Φ6, the note must start at time t=t3, and the three labels 4, 5 and 6 must be generated by the note schedule of that ending time to be applied. When generating the target Okion matching value,
The end point probabilities along the determined end point distribution button are summed. If desired, the logarithm of the summation is taken to give the following equation:

Matching value = l ogi (Φ +...+Φ6) As mentioned before, the matching score value for a certain J1 word is
It can be easily determined by summing the matching values for consecutive single sounds in a particular word.

Now, in explaining the generation of the starting time distribution, reference is made to Figure 14. In fP14 (a) lc, the single @'rHa1 is repeated and divided into elemental single sounds. FIG. 14(b) shows a label sequence over time. In FIG. 14<e) a first starting point distribution is shown. This onset time distribution is obtained from the end time distribution of the previous most recent phone (in the previous 4S word, which may contain a silence, i.e., a word without a sound). Figure 14(c). Based on the label input and the start time distribution, the start time distribution ΦDT( of the single note T)H is generated. ) is determined by multiplying the period exceeding the threshold (A) by 79 i/l. (·A) is determined separately for each end point distribution.

Preferably, (A) is a function of the total value of the end time distribution of the target single note. Thus, the period between times a and b represents the time during which the starting point distribution for the single note U)I is set (see FIG. 14(e)).

The period between times C° and d in Fig. 14 I's) corresponds to the period in which the end time distribution of the single note DH exceeds the threshold fX- and the start time distribution of the next tsubo tone is set. . The value of the start time distribution p can be obtained, for example, by normalizing the end time distribution by dividing each introduction time by the sum of end times exceeding the threshold (A).

This basic high-speed verification sound machine 73000 is F'loi
tlng Po1nt System+s 190L
It was executed using the APAL program in the program. Other hardware and software may also be used to develop a particular form of matching procedure in accordance with the above teachings.

E-IE, separate fast matching Basic fast matching can be done alone or with detailed matching or '
*The word model is combined to significantly reduce the amount of calculation required. To further reduce the required total fi amount, the method described here can be further refined by defining a uniform label length ax distribution between two lengths, the minimum length and the maximum length in L. , which further simplifies detailed matching. In basic fast matching, the probability of a phone generating a label of a given length, fj, 1°, 11.12, etc., typically has different values. However, according to this alternative fast match, the probabilities for each length of the label are replaced by a single uniform value.

Preferably, the minimum length is equal to the minimum length that has a non-zero probability in the original length distribution, but other lengths may be used if desired.

The choice of maximum length is more arbitrary than the choice of minimum length, but
It is important that the probability of a length being less than the minimum length and greater than the maximum length is set to zero. length'm
By setting the ratio to exist only between the maximum and minimum lengths, a uniform pseudo-distribution is given. In one method, the uniform probability can be set as the average probability over the pseudo-distribution. Alternatively, the uniform probability is the probability of the length being replaced by the uniform value above! It may be set as a catfish large value.

The effect of characterizing all label length probabilities as equal is easily seen with reference to the equations shown above for the basic fast match termination time distribution. especially,
Length probability can be expressed as a constant term.

By setting L, T11n to zero and replacing all length probabilities with a single constant value, the termination point distribution is characterized as follows.

θ1=Φm/1=Qm+em−IPm where “1” is a single uniform displacement value, and the value of Pm is preferably the displacement of a given label produced during a given note at time m. corresponds to a value.

For the above equation for θm, the matching value is defined as follows.

Verification value "IQglo (θ0+01+...+0m)+
l o tz 1 (+ (1)
Comparing the basic fast match and this alternative fast match:
It can be seen that the number of additions and multiplications required is reduced by more than K by adopting this alternative fast match. L
It was found that for rn, n=o, a basic fast match requires 40 JPRs and 20 additions in that length probabilities must be taken into account. Using this alternative fast match, θ□ is determined recursively, requiring only one multiplication and one addition for each successive θ□.

Figures 15 and 16 are provided to illustrate how this alternative fast match simplifies calculations.

In FIG. 15(a), the sound machine embodiment 3100 corresponds to a minimum length Ln11n=o. The maximum length at this time is defined as infinite so that the length distribution can be characterized as uniform. Figure 15(b)V
In C, a grid diagram generated from the sound machine 3100 is shown. Here, assuming that the starting time after qn is outside the starting time distribution, one addition and one multiplication are required to determine all successive θ□ for m<n. It is. Also, to find the end time, 1
Only one multiplication is required, and addition W is not necessary.

Figure 16 shows the case of "mfn='. Figure 16 (a
) shows a special example of a sound machine 3200 for this purpose, and FIG. 16(b) shows a corresponding grid diagram.

Since Lrn1n=4, the lattice diagram of Figure 16 (rb) is 1.
%V%W and has zero probability along the path marked with 2. θ4 and θ. Note that for those ending times extending between However, for end times greater than n+4,
One time! P calculation only, no addition required. This example is F
'Implemented using APAL code on PS 190L.

It should be noted that states may be added to the embodiment of FIG. 15 or 16 as desired.
In order to further normalize the initial J-label based matching basic fast matching and the fast matching above, the idea is to consider only the first 5 labels of the string input to the sound machine in the matching. be done. 1/100th
Assuming that labels are generated by the audio processing equipment of the audio channel at a rate of one per second, the corresponding value for J is 100. In short, a label corresponding to a second or so of audio is given to check for a match between the note and the label coming into the sound machine. By limiting the number of labels that are inspected, two advantages are realized. First, the decoding delay is reduced. Second
Additionally, interludes in comparing short word scores with long word scores are substantially avoided. Of course, the length of J may be varied as desired.

The effect of limiting the number of labels inspected is
If you refer to the grid diagram in Figure (b), you can see it from K. The score of &t1 fast matching without the above improvement is the sum of the probabilities of θ□ along the bottom row of this figure. That is,
The probability of being in state S4 at each time 1 n starting at 1 = 10 (for Lmin = 0) or 1 = 14 (for L, = 4) is taken as θ□, and all θ□ is totaled to Lrn1
If n=4, there is no probability of being in the state P4S4 at any time before t4. However, using the above improvement, θ
The sum of □ ends at time J. In FIG. 16 (bl, time J corresponds to time tn+2.

Inspect J label at or above J time interval? To end,
This results in two probability sums when calculating matching scores: 1K, as mentioned above, there is a row computation along the bottom row of the lattice diagram, but only up to time J-1. time J
State 5411 at each time up to -1? : Certain probabilities are summed to obtain a row score value. 2nd VC, state S at time J. There is a score for the row corresponding to the Japanese button with a probability that there is a single note in each state of −84. That is, the row score is: This matching value for a phone is obtained by summing the row score and the column score and then taking the logarithm of that sum. To maintain fast matching for the next phone, the values along the bottom row, preferably including time J, are used to obtain the onset time distribution of the next phone.

After calculating the matching value for each successive ψ sound, the sum a1 for all the sounds is the sum of the matching scores for all the gongs, as mentioned before.

When we examine how the end point distribution is generated using the basic 2C high-speed matching described above and the other Hatakehaya Eima, we find that the calculated values of the column scores easily match the high-speed matching calculated values. I can take it. In order to better apply the improved technique of limiting the number of labels examined to high-speed matching, the matching technique described herein requires replacing the scores in one column with the scores in another row. That is, the score of another row is calculated for the single note in state s4 (Figure 16 (b)) between time J and J+K (K is the maximum number of eggs in the sound machine). Therefore, if the sound machine has 10 states, this refinement would add 10 termination times along the lower rows of the grid diagram on which the probabilities are calculated. Then, all the probabilities along the bottom row up to time J+K and the probability at time J+KK are added to obtain the match value for a given phone. Also, as before, consecutive single-phone matches 164 are summed to obtain matching scores for busy and added words.

This example was implemented using APAL code on an FPS 190L. However, it can also be implemented using other code and hardware.

E-1G, Kishion's tree structure and high-speed matching Using basic fast matching or another fast matching Calculate 1 round of sound machine matching with or without limiting the %maximum label from K The computational time required to do so is significantly reduced. Furthermore, even in cases where detailed matching is performed on the words obtained - fast matching - the computational time savings are large enough.

Once the match values for the phone are calculated, they are compared along the branches of the tree structure 4100, as shown in Figure 17, to determine which path for the phone is most likely. In FIG. 17, the sum of matching values for the single notes corresponding to DH and DHl (connected from 4102 to branch 4104) is greater than the sum of matching values for the single notes that branch from the single note MX.
It should be a ridiculously high value corresponding to the spoken word "the". In this regard, the single note combination value of the first MX note is calculated only once and then used for each base form extending from that MX note (branches 4104 and 41
06). Furthermore, if the total score value calculated for the first branch column is found to be significantly lower than the total score value of the other branch columns or much lower than the threshold , all base forms extending from its first column are removed from the candidate word at the same time. For example, when it turns out that MX is not a likely route, from branch 41o8 to branch 4
The basic forms associated up to 118 are rejected at the same time.

By using this fast matching and tree structure, an ordered list of 'IBM' words is obtained with an extremely economical total of 19 words.

Regarding memory requirements, note that the tree structure of the sounds, the statistics of the sounds, and the terminal probabilities are what should be memorized. For the tree structure, there are 25,000 arcs and 4 data words characterizing each arc. The first data word is the index to the next arc or note. The data word of rod 2 is the branch VC
Represents the number of the next note. The third data word represents at which node of the trI diagram the arc is located. Also, the data word @4 represents the current single note. Therefore, for a tree structure, 2500
0x4 storage locations are required. In fast matching, there are 100 different phonemes and 200 black phonemes. Then, if a phoneme has a single broken tree generated somewhere in a phone, 100×200 storage locations for statistical probabilities are required. For a change, Gosuke of terminal probability,
200x200 storage locations are required. For this fast lookup, 100 integer and 60 floating point stores are sufficient.

E-1) r. Language model As mentioned earlier, we use a language model that stores information related to Peng language in context, such as triplet characters, to estimate the probability of correct word usage. You can also use K to increase it.

The language model 1010 (Kan 11 Kan) has unique properties. In particular, a converted trigraph system is used. According to this method, a sample text is examined to determine the likelihood of Ml[okero, single, Φ words, ordered pairs of words, and l1kl ordered three-speed words. . Then, a list is formed of the most probable 11 strokes of Sansei Tsubogo and the most probable pair of J144.

Furthermore, the probability that the double word is not in the list of the three-speed part word list and the probability that the paired 4S word is not in the list of paired words are formed, respectively.

According to this language model, a certain Reibe's simple F? [When two words follow, a determination is made as to whether this current word and the following two words are present in the three-speed single KN') strike. And, if so, the memorized weight assigned to that three-speed candy is shown.

Also, if the current situation is 4. % and the two words that follow it are three-speed I! P)% If it does not exist in the IJ strike, then the current 1. A determination is made as to whether Ft and the next word are present in the paired word list. Then, if so, the W rate at which the third speed word is not present in the binary word list is multiplied by the probability of that pair, and the product is assigned to the word at hand. Or, if the current word and the next (and the next) word are not on the double word list and the paired word list, then the probability that the double word is not on the three-speed kimo list only for the current word, Paired words are multiplied by the probability that they are not (paired words). This f* is then assigned to the word in question.

Referring to FIG. 18, there is shown a flowchart showing details of the sound machine used for voice matching. In step 5002, the word containing word (approximately 50
00 pieces) are defined. Next, 1, each word is pronounced by a bank of sound machines (step 50r14). Although the sound machine is described as, for example, an aural sound machine, it may alternatively include an aural sound. The generation of words by a sequence of phonetic or phonemic sound machines will be explained below.

In step 5006, the basic forms of words are arranged in the tree structure. The statistics for each sound machine in each word basic form are F, Jelinek, V
Continuous speech 159 (cont
Inious Speech Recognition
by StatisticalMethod
It is determined by Eleven Gen., based on the well-known forward backward algorithm published in the paper "S)".

In step 5009, the values to be substituted for the actual parameter values or statistics used during detailed matching are determined. For example, a value to be substituted for the actual label output probability is determined. In step 5010, the determined value is determined such that the sound in each word basic form has an approximate substitution value.
Replaced by the actual memorized probability. All approximations related to basic fast matching are performed during step 5010.

A determination is then made as to whether to enhance the power of phonetic matching (step 5011), and if not, the determined value can be used for basic approximate matching. is set, and other evaluation values associated with other approximations are not set (step 5012). Also,
If another improved approximate match is desired, then step 5 (118) is performed. A uniform string length determination is then made (step 5018) and further improvement is desired (step 5,020''). A decision is then made as to whether a further improvement in approximate matching is desired, then phonetic matching is limited to the first five labels in the generated string (step 502'').The calculated parameter values are then set in step ``5012'', regardless of whether an improved approximation match has been selected, at which point each sound machine in each word base form performs a fast approximation. This means that K has been trained by the desired approximation that makes matching possible.

E-1J, Stack Decoder A preferred stack decoder for use in the speech recognition system of FIG.
Bahl, F.
), and R,L,-r-Mereer. Therefore, this preferred stack decoder will be explained below.

FIG. 19 and FIG. 20 show a plurality of sequential labels Y Y generated at sequential label intervals or label positions.
···It is shown.

Whistle 201'QK also illustrates several generated word paths: path A, path B, and path C. In the context of diagram 19, path A is entry 1 to be
Path B corresponds to the entry "tw.

Corresponding to the word path at hand, there is a label (or equivalently, a label interval) that has the highest probability that the word path has ended. , such a label is called a "boundary label".

For a word string W< representing a string of words, the most likely end time that appears in the label string as a "boundary label" between two words is the 18M Technical Disclosure Pultin.
T') iselosureBulletln)
volume 2 3, number
er 4. September 1980, L, R, Pearl, F.
"Fast Aeoustle MatchC" by Jielinecz, R.L.
and the method described in the document entitled 'Imputation)'. Simply put,
This document covers two similar matters, namely (a)
(b) how much of the label string Y is caused by a word (yet a word sequence); and (b) at which label interval the partial sentence corresponding to the part of the label string ends. Discuss ways to get involved.

For any word path, there is a "likelihood value" associated with each label or label interval in the label string, from the first label to the boundary label. Taken together, all likelihood values for a given word path represent the "likelihood vector" for that given word path. Therefore, for each word path there is a corresponding likelihood vector.

The likelihood value Lt is shown in Figure 20.

The “likelihood envelope” At between labels it, which corresponds to the collection of word paths W1, W2, ...WS, is defined mathematically as follows: A = max(L (Wl ),...L (WS
)) That is, for each label interval, the likelihood hull path contains the highest likelihood value associated with any word path in the collection. In FIG. 220, the likelihood envelope 1p1040 is illustrated.

A word path is 1 complete if it corresponds to a complete sentence.
It is thought that. The complete path is preferably identified by the speaker's input, for example by pressing a button when the speaker reaches the end of the sentence. This input is
Synchronized with label spacing and marks end of sentence. A complete word path cannot be extended by adding words.

A "partial" word path corresponds to an incomplete sentence and can be extended.

Partial pathways are classified as "live" or "dead".

That is, if a word path has already been extended, the word path is "dead", and if it has not been extended yet, it is "alive". Using this classification, a path that has already been lengthened to form a word path lengthened to one or more lengths is not considered again for lengthening at a later time.

Each word path can also be marked as ``good'' or 1 bad in relation to the likelihood envelope.

That is, if a word path has a likelihood value that is within Δ of the maximum likelihood envelope for a label corresponding to a boundary label, then that word path is good. Otherwise, the word path is 1 bad. Note that preferably, but not necessarily, Δ is a constant value such that each value of the maximum likelihood envelope reduced to act as a threshold level.

There are 11 stack elements for each label interval.

Each live word path is assigned a stack element corresponding to the label interval that corresponds to the threshold level of that path. A stack element can have 0.1 or more path entries, and the entries are listed in order of likelihood value.

Next, the stepping performed by the stack decoder 1002 in the diagram will be described.

Forming the likelihood hull path and determining which word path is better is correlated with the steps shown in the sample 70-Char-IC of FIG.

In the flowchart of FIG. 22, zero (null)
The path first stacks 941 (0) at step 5050.
K is input. and, if there are complete paths determined in advance, a stack (complete) containing those complete paths.
! Two children are given (step 5052). stack(
Each complete path in the (complete) element has an associated likelihood vector. The likelihood vector of the complete path with the highest likelihood for the boundary label initially determines the maximum likelihood envelope. If there is no complete path in the stacked (complete) elements, the maximum likelihood envelope is −ω
It is initialized as . Furthermore, if a complete path is identified and fc, the maximum likelihood envelope can still be initialized as 1.

Steps 5054 and 505 for envelope initialization
It is indicated by 61'1m.

After the maximum likelihood envelope is initialized, it is lowered by a predetermined value Δ, and a Δ-good region is formed above the lowered likelihood value, and the likelihood is A Δ-defective region is formed below the value.

It is believed that the larger the value of Δ, the more word paths can be extended. 10 K to calculate L.
If 10 is used, a value of 2.0 for Δ gives satisfactory results. This value of l is preferably, but not necessarily, uniform along the length of the label interval.

If a word path has a likelihood at a boundary label that is in the Δ-good region, then the word path is marked as "2 good." Otherwise, the word path is marked as 1 bad.

ttg, as shown in Figure 22, the loop for updating the likelihood bond and marking word paths as good (extendable) or bad starts by finding the longest unmarked word path K. To be done (Stella 7'5058)
. Then, if two or more unmarked word paths exist in the stack corresponding to the length of the word path of that length, the word path with the greatest likelihood for its boundary label is selected. Ru. If an S-word path is found, it is marked as "good" if the likelihood at its boundary label is within the Δ-good region, otherwise it is marked as "1 bad" (step 5060).

If the word path is marked as ``1 bad,'' another unmarked live path is found and marked (step 5062). If the word path is marked as ``good,'' then the 1 good path is found and marked (step 5062). The likelihood hull line is updated to include the likelihood value of the path marked ``. That is, for each label interval, the current likelihood value in the rhyme likelihood envelope and (b)
An updated likelihood value is determined as a likelihood value greater than one between the likelihood values associated with word paths marked as "good." This is indicated by steps 5064 and 5066. After the envelope path is updated, the longest and best unmarked live word path is found again (step 5058).

The loop is then repeated until there are no unmarked word paths left. Then, once all unmarked words have been found, the shortest word path marked "good" is selected. If there is more than one path with the shortest length marked as good, the path with the highest likelihood at its boundary label is selected (step 5070). The selected shortest path is then extended. That is, preferably, performing the fast matching, language model, detailed matching, and language model procedures determines at least one likely descendant word from K as described above. Each likely successive word forms an extended word path. In particular, an extended word path is formed by adding a key word that most likely follows the end of the shortest shortened word path.

That'J! After an extended word path has been formed to the triangulated shortest word path, the selected word path is removed from the stack in which it was an entry, and each extended word path is It is placed in the appropriate stack for that purpose. In particular, the extended word path results in an entry placed in the stack corresponding to the boundary label of the extended word path (step 5072).

Regarding step 5072, the operation of selecting and extending nine paths is described with reference to FIG.

After a path is found in step 5070, a procedure as described below is performed, which extends the word path or paths based on a suitable approximate match.

That is, in step 6000 (Figure 21), audio processing %ff1002 (Figure 1) generates a label string as described above. A column of labels is provided as input to enable execution of step 6002. In step 6002, to obtain an ordered list of candidate words according to the teachings presented above,
One of the basic approximate matching or improved approximate matching procedures is performed. The language model (as previously described) is then applied in step 6004. The words remaining after the language model is applied are input with the labels generated in the detailed matching processor, which performs step 6006. The detailed matching yields a list of remaining word candidates, which is preferably multiplied by the language model K in step 6008.

The likely words thus determined by the approximate matching, detailed matching, and language model are used to extend the path found in step 5070 of FIG. 21. Step 600 B (Figure ffi22) Each of the determined probable words is divided into a plurality of extended 4! The word path is individually appended to the found word path so that the found word path can be formed.

Referring again to Figure 22, after the extended path has been formed and the stack has been re-formed, the process is repeated nine times by returning to step 5052.

Each iteration thus consists of selecting the shortest best word path and extending it. A word path marked as ``1 bad'' in the first processing will be marked as 1 bad in subsequent iterations.
It may be a good 2. The characterization of a live word path as 1 good or 1 bad is thus done independently for each iteration. In practice, the likelihood envelope does not change significantly from one iteration to the next, so the calculation to determine whether the word path is 1 good or 1 bad. will be effectively implemented. Furthermore, no standardization is required.

When a complete sentence is identified, preferably step 507
4 is executed. That is, when there are no unmarked star word paths remaining and no good word paths to extend, decoding ends. The complete word path with the highest likelihood at each boundary label is
It is considered to be the most likely univocal sequence for the string.

Note that in the case of continuous speech in which the end of a sentence is not identified, the path extension is continuous or continues for a predetermined number of words depending on the preference of the system user.

E-IK, Construction of Phonetic Basic FormsThere are speech-based Markov model sound machines that can be used in forming basic forms. That is, each sound machine uses the International Phonetic Alphabet (International Phonetic Alphabet).
ional PhoneticAlphabet ).

For a given word, there is a sequence of phonetic sounds, each corresponding to an individual sound machine. Each sound machine is W'P
It contains states of i and multiple transitions between those states, some of which can generate a phoneme output and some (called zero transitions) cannot. As mentioned above, the statistics associated with each sound machine include (a) the probability that a given transition will occur, and (b) the likelihood that a given transition will produce a particular phoneme. Preferably,
In each non-zero transition, each phoneme is associated with some probability. Table listed at the end of the detailed description of the invention■
The phoneme alphabet shown in preferably includes 200
There are several phonemes. The sound machine used to form the acoustic sound machine is shown in FIG. A row of such sound machines is provided for each word. Statistics or probabilities are entered into the sound machine during the training period when known words are sounded. Then, the transition & rate and phoneme probabilities in various phonetic sound machines are determined by focusing on the phoneme strings generated when a known phonetic sound is pronounced at least once, and using the well-known forward-backward ( forward
-backward) is determined during the training period by applying the algorithm.

An example of the statistics of one single sound identified as T (short sound T) is
As shown in the table listed at the end of the detailed description of the invention. As an approximation, the transitions tr1, tr of the sound machine in Figure @3
The label output probability distributions of Tr2 and tr13 are expressed by a single distribution. Also, transitions tr3, tr4, tr5, and tr9 are expressed by a single distribution, and transitions tr6,
tr7 and trlo are also represented by a single distribution.

This can be seen in Table 2 by assigning arcs (or transitions) to the individual columns 4.5 or 6.

Table 2 shows the probability of each transition and the label (i.e. phoneme)
It shows the probability of being generated at the leading edge, middle, and trailing edge of the sound t)T(.In the case of this nTT sound, for example,
The probability of transitioning from state S1 to state S2 is 11.072
It is calculated to be 43. From state P4s1 to state S4
The transition probability to is 0.92757 (in this case, these are the only two possible transitions from the initial state, so the sum of their probabilities is 1).

Regarding the label output probability, the DT (sound is the phoneme AE13 at the end of its single note, i.e., the 6th line of Table 2 (see Table 1)
The probability of generating is 0.091. Also in Table 2 are counts associated with each node (state). This node count represents the number of times the sound is in the corresponding state during the training period. Statistics such as Table 2 are found for each sound machine.

Arranging phonetic sound machines into word basic forms is typically performed by a phonetician and is generally not performed automatically.

E-IL, construction of basic phonemic forms FIG. 23 shows a specific example of phonemic sounds.

This phonemic sound has two states and three transitions. In this figure, the zero transition is shown as a dashed line, which depicts the path from state 1 to state 2 where no label is formed. The self-loop transition in state 1 allows any number of labels to be generated from it. Non-zero transitions between state 1 and state 2 are allowed to form a label. The probabilities associated with each transition and each label in the transition are determined in a manner similar to that described in relation to the basic form of the phonetic type in the step #.

The basic phonemic word form is constructed by combining is-like sounds. One method for doing this is described in commonly assigned US patent application Ser. No. 697,174, filed February 1, 1985. Preferably, the phonemic word base form is generated by multiple pronunciations of the corresponding word.

This concerns the applicant. It is described in Japanese Patent No. 738933, filed on May 29, 1985. Briefly, one method for generating basic forms from a large number of pronunciations consists of the following steps.

(PL) Convert multiple pronunciations of a word fragment into individual phoneme strings.

(b) - Define the set tl)tle+Markov model sound machine.

(c) Determine the best single note machine P1 to generate the IJ song.

cd) The above-mentioned number of phonemes) In order to generate an IJ song, the best 2 consisting of the form P1P2 or P 2 P 1
Decide on the front shoulder type.

Arrange the best 2-front shoulder form above for each phoneme string.

(f) Divide each phoneme string into a left part and a right part. At this time, the left part corresponds to the sound machine of Pit 1 of the two-front shoulder type, and the right part corresponds to the sound machine of Pit 2 of the two-front shoulder type.

6r) Identify each left part as a left substring and each right part as a right substring) IJ song, respectively.

can) process the set of left substrings in the same way as the set of phoneme strings corresponding to multiple pronunciations. The process includes preventing the substrings from further separating when the single front shoulder type has a higher probability than the best two front shoulder type.

(j) Process the set of right substrings in the same way as the set of phoneme strings corresponding to multiple pronunciations. The process includes preventing the substrings from further separating when the single front shoulder type has a higher probability than the best two front shoulder type.

藺 Combines unseparated single tones in an order that corresponds to the order of the phoneme substrings to which they correspond.

The number of these model elements is typically approximately equal to the number of phonemes obtained corresponding to the pronunciation of the word.

The basic formal model then utters one known pronunciation,
It is trained (or incorporates statistics) by inputting it into an audio processor that generates a string of labels in response to the vocal button.

Then, based on the known pronunciation and the generated label, the familiar forward-back
The word model statistics are obtained according to the kward) algorithm.

FIG. 24 shows a grid corresponding to phonemic sounds. This lattice consists of the first
This is considerably simpler than the grid in Figure 1.

E-2, Selection of Probable Words from Word t by Pauling Referring to FIG. 25, a 70-chart of one embodiment of the present invention is illustrated. This flowchart 80001
In C, the word weight of the word is first determined in step 8002. These words correspond to either a business communication word weight or a technical word weight, depending on the user. At this time, there are 511DO or more words in the word, but the number of words can be changed.

Each key is represented by a row of Markov model sound machines according to the teachings of the E-IK or E-IL chapters above. That is, each word has a # of sequential phonetic sound machines.
It can appear as a built-up basic form or a structured basic form of a sequential phonetic sound machine.

Next time? :, step 8006 calculates one vote for each label of each word.

Step 8006 of calculating votes is *25.26.27
．． 28 and 29FJQ.

tjt26 is a diagram showing the distribution of audio labels of a given sound machine P. The counts shown in this figure are obtained from statistics generated during the open cavity period. That is, recall that during the separate line period, a known pronunciation corresponding to a known phonetic sequence is uttered and a label string is generated in response. thus,
The number of times each label is generated when a known sound is pronounced is obtained during the separate line period. Incidentally, a distribution map as shown in Fig. 26 is generated for each single note.

From the separate line data, not only the information contained in FIG. 26 can be obtained, but also the expected value of the label for a given note. That is, when a known pronunciation corresponding to a given language is uttered, the number of labels generated is recorded. The number of labels corresponding to a given phone is recorded each time a known pronunciation is uttered. From this information, the most likely or expected number corresponding to a given note is then determined. The tIC27 diagram is a diagram showing the expected number of labels for each single note.

If these sounds are phonemically corresponding, the average number of expected labels for each phone should typically be about 1. For phonetic sounds, the number of labels can range over a very wide range.

Extracting information from the training data involves continuous speech recognition using statistical methods as described above.
Soe@ch R@eo(nitionby
The forward-backward method is described in detail in the paper titled
This can be achieved by using a ward-backward algorithm. Briefly, the forward-backward algorithm (a) looks forward from the initial state of the Markov model to state i and determines the statistics leading to the state quantity on the 'forward path'; (b) the Markov model From the final state of q(l+1)
Look backwards at Toyode and see “state (l) on backward route #”.
+1) to the final state, consisting of determining the probability of each transition between state 1 and state (1+1') in a single note. From state i to state (l+
The transition probability to 1) and the label output for it are:
It is combined with other statistics in determining the probability of a particular transition occurring given a label string.

As shown in M28154 with respect to Word 1 and Word 2, each word is known to be a predetermined string of single sounds. Given the sound sequence for each unit @ and the information described in connection with Figures 25 and 26, we can determine how many times a given label is most likely to occur with a particular word WK. can be determined. For #, words like word 1, the number of times label 1 is expected is the count of 2 pel 1 for sound P1, the count of label 1 for sound P3, and the label 1 for sound P6Vc. This is the sum of the calculated values, etc. Similarly, the number of times label 2 is expected for word IK is the sum of the count value of label 2 for *p, the count value of label 2 for sound P3, etc. The expected count value for each label of word 1 is 2
It is calculated by performing the above steps for each of 00 labels. The count value for a particular label represents the number of occurrences of the particular label divided by the total number of labels generated during the training period, or represents the number of occurrences of the label during the separate line period itself.

In FIG. 29, the expected count value for each label in a specific @ (for example, word 1) is shown.

For a given word, from the expected label counts as shown in FIG. 29, the "votes" for each label of that word are calculated. The vote for label L' for a given word V represents the likelihood that word W generates label L'. This vote corresponds to the logarithm of the probability that word W generates label L'. Preferably, the vote value is calculated using the following formula:

Votes ==log10 (?r (L' 1viF) 1-
The values of these votes are stored in a table as shown in FIG. , for each word 1 to W, each label is 2
It has the value of the vote expressed by V with double subscript. This first subscript corresponds to the label, and the second subscript is the single F
Compatible with FFtK. Therefore, for example, v12 is the value of 10 votes for the label regarding word 2.

Referring again to No. 25, a process is illustrated that includes step 8008 of generating a label in response to an unknown speech input, and selecting a likely courtesy word from common pronunciation using a polling button. This process is performed using audio processing @e 100
4 (Fig. 1).

For the word at hand: the label generated in the 301st vl table is looked up (look u+11)
be done. Then, the vote value of each generated label is searched for that word. This vote value is then accumulated to give a total vote value for that word (step ao1o). For example, if the labels 1.3 and 5 were generated, the vote values v11, v81 and "51 would be computed and combined. If the vote values are the logarithm of the probabilities, then they are are summed to give the overall vote value for word 1. A similar procedure is performed for each word in word 9, and from this button labels 1, 3 and 5 "vote" for each medium salary. It turns out.

According to one embodiment of the invention, the vote value accumulated for each Chinese word serves as the likelihood score value for that word. The n words (where n is a predetermined integer) with the highest accumulated vote value are then determined as candidate words, which are later processed in the detailed matching and language model described above. It turns out.

In another embodiment, a "penalty" is calculated along with the vote value. That is, for each word, a penalty is calculated and assigned (step 8012). This penalty represents the likelihood that the label at hand is not generated by a given 凯 word.

There are various ways to calculate the penalty.

One method for calculating the 4S@ penalty imposed by the phonemic base form involves assuming that each phonetic sound generates only one label. For a given label and a particular phonemic sound, the penalty for that given label corresponds to the logarithm of the probability that a different label is produced by that particular phonemic sound. Therefore, the penalty for label 1 of sound P2 is label 2
Any label up to label 200 corresponds to the logarithm of the probability that one label occurs.

Note that the assumption that there is one label output for each phonemic sound, although not accurate, has been found to be satisfactory for calculating penalties. Once the penalty for the label connected to each sound is determined in this way, the penalty for a word forming a sequence of known sounds is easily determined.

The single label penalty for each word is shown in FIG. Each penalty is identified as a PgN with two subscripts, the first subscript representing the label and the second subscript representing the word.

Returning again to Figure Pole 25, the generated labels in step 8008 are examined to see which labels in the label alphabet have not been generated. In this way, the penalty for not generating each label is calculated for each label. To obtain the overall penalty for a given word, the penalty for each label not generated for a given word is determined and all such penalties are accumulated (step 8014). If each penalty corresponds to the logarithm of the "zero" probability, then the penalties for a given word are summed over all labels, just as for votes. This procedure i! is repeated for each word of the payoff, so that each word has an overall vote value and an overall penalty given the string of labels generated.

Once the overall vote value and overall penalty are obtained for each word in the feed, a likelihood score value is determined by combining the two values (step 8016). still,
If desired, the overall vote value may be weighted more heavily than the overall penalty, and vice versa.

Additionally, the likelihood score value for each 4j word is preferably scaled based on the number of labels voting (step 8018). In particular, the overall vote value and the overall penalty (
(both of which represent the sum of the logarithms of the probabilities) are added together, and then the final sum value is divided by the number of vocal 2-pels that occurred and were taken into account when calculating the vote value and penalty. .

As a result, the likelihood score value is scaled.

Yet another aspect of the invention relates to determining which labels of a string to consider in voting and penalty (or polling) operations. If the end of a word is distinguished and its corresponding label is known, preferably all labels generated between the known start time and the known end time are considered. However, when it is known that the end time is not known (step 8
020)% The present invention provides the following method. That is, a standard end time is defined, and repeat likelihood score values are calculated for each successive time interval after the standard end time (step 8022).

For example, after 500 milliseconds, a (scaled) likelihood score value is calculated for each word at 50 millisecond intervals, and this operation continues until 1000 milliseconds are reached from the start time of the word's pronunciation. In this example, each word will have 10 (scaled) likelihood score values.

A technique is then applied to perform /%'f< as to which of the 10 likelihood score values to assign to a given 4S square. In particular, for a sequence of likelihood score values obtained for a given word, the highest likelihood score value is selected relative to the likelihood score values of the words obtained at the same time interval. (
Step 8024).

This highest likelihood score value is subtracted from all other likelihood score values for that same time interval.

The word with the highest likelihood score value for a given time interval will then be set to zero, and the likelihood score values of other less likely words will have negative values. Then, a likelihood score value with a small negative value (close to zero) for a given word is assigned to that word as a relative likelihood score value with a high value.

When a likelihood ai point value is assigned to each car F@, the n words to which the brightest likelihood IW score value is assigned are selected as candidate words obtained by polling ( Step 8026).

In one embodiment of the invention, the n words obtained by polling are given as a reduced number of words list, 1\'! of this list. The words are subjected to the detailed matching and language model processing described above. Obtained by boring. This reduced list of numbers serves in this embodiment as an alternative to the phonetic fast matching shown above. In this regard, phonetic fast matching provides a wedge-shaped lattice structure into which word base forms are input as successive sounds, in which words with the same initial sound are found in the tree-shaped lattice structure 'VC'. It is observed to follow a common branch along. However, for a vocabulary of 2000 words, the polling method of the present invention was found to have a processing speed two to three times faster than a high-speed match with a dendritic lattice structure.

However, in contrast, it is also possible to use a combination of phonetic high-speed verification and polling.

In other words, F! From the generated Markov model and the generated labels, step 80
Approximate fast matching is performed at 28 in parallel with polling. Then, one list is given by phonetic matching and the other list is given by +1 polling. In conventional practice, the entries on one list are
Used when increasing the other list. but,
In approaches that desire to further increase the number of best word candidates by p 4 >, only words that appear in both lists are retained for further processing. step 80
The interaction of these two techniques in 30 depends on the # of the system and the goals on γ in total. In yet another embodiment, a grid-based phonetic fast match may be applied sequentially to the polling list.

Apparatus for performing polling is shown in Figure 32 of the housing. In this figure, element 8102 stores word models trained as described above.

Then, from the statistics applied to the word model, the vote generation PF 8104 calculates the vote value of the label for each car MK, and stores the vote value in the vote table storage device 8106.

Similar to knead, penalty generation prefix 810B calculates the penalty for each label of each word in the idiom and enters the value into penalty table storage 8110.

The 31-word likelihood score calculation device 8112 receives labels generated by the audio processing device 8114 in response to unknown audio input. Then, for the word selected by the word selection element 8116, the word likelihood score meter W, 舊*F 1112 calculates the votes of each label generated for the selected word and the number of labels generated. Combine it with the tC penalty. , the apparatus 8112 also includes means for scaling the likelihood score values, as described above. The likelihood score calculation device may, although not necessarily, include means for repeating the calculation of the likelihood score at successive time intervals))41 after the reference time.

Likelihood score value meter W, device 8112 is Kishi Katari strike equipment device 8
120 with a single F) score value, and word list device 8120
arranges words according to their assigned likelihood score values.

In a practical example of combining a single FFM I7 list obtained by polling with a list obtained by approximate fast matching, a list comparator v8122 is provided.

This device 8122 receives as input a polling list from a wordlist device and a polling list from a phonetic fast match (as described in some embodiments above).

Several features are incorporated to reduce the amount of memory and computation required. Finally, the vote value and penalty can be formatted as integers ranging from 0 to 255. Second, the actual penalty value can be replaced by an approximate penalty calculated corresponding to the vote value from the formula Penalty=a*(Vote value)+h. In this equation, ^ and b are constants, and their values are obtained by least squares regression. Third, labels can be grouped into phonetic classes such that each class contains at least one label. and,
The assignment of labels to classes is determined by hierarchical <att product of the labels so as to maximize the information between the resulting phonetic classes and words.

Note that in accordance with the present invention, periods of silence are detected (by known methods) and ignored.

The present invention also provides PL/
Although it was implemented using I, it can also be implemented using other systems and programming languages.

Furthermore, the present invention can be modified in various ways within the scope of its technical idea.

For example, the end time of a word may be determined by summing the number of expected labels for each sound in the word base form. Additionally, the vote value and penalty may be calculated only for 3f4 interpreted labels, such as the odd numbered labels or the first m labels of the generated label string. however,
It is preferable to consider the vote value and penalty for each label between the beginning and end of a word.

Furthermore, the present invention contemplates the use of various voting formulas other than adding logarithms of probabilities. In the present invention, each label casts a vote for each word in the vocabulary, and this vote
It is widely applied to the actual polling situation and polling method to obtain a short list of candidate words, especially when the word is different from word to word.

Effects of the F8 Invention As described above, according to the present invention, the polling method is used to create a list of words that should be checked in detail in speech recognition, so that such a word list can be This can be achieved with less processing time.

Hongi seepage (V Q coC Rorohehehehehehehe α heroo W W-r r ff P?+Fv+?
sentry sentry (a) sentry
Dimension Dimension Dimension Dimension Beak Rorororororororororororororororororororororolololololololololololololololololololololololololololololololololololololololololololololololoro
&lJ+KukuI+I&h-I-KZ>A11. Q & HHpp

[Brief explanation of drawings]

Diagram 1 of the furnace is a block diagram of the main components of the sound system, Figure 2 is a block diagram showing the block diagram of Figure 1 in more detail, Figure 3 is a diagram of the acoustic sound machine, and Figure 4 is a block diagram of the audio processing device, t4r, 5 is a cross-sectional view showing the inside of the human ear, FIG. The diagram of the curve, Figure 8 shows the correspondence between the tone and the phono, and Figure 9 shows the
The processing flowchart of the audio processing device, FIG. 10, is! 9
Detailed flowchart for updating the threshold value in Figure 1. Figure 11 is a diagram showing the detailed matching grid. Figure Fl 2 is a block diagram of the high speed matching sound machine. #13
Figure 14 shows the mutual relationship between a single note, a label string, and the start and end times. Figure 15 shows the minimum length 0 note machine and its start time distribution. Figure 16 is a diagram showing a minimum length 4 sound machine and its time chart, Figure 17 is a diagram showing a sound god J-shaped structure, and Figure 18 is a diagram showing a sound machine with a minimum length of 4 and its time chart. Figure 19 is a diagram representing the sequential steps of stack decoding; Figure 20 is a flowchart diagram representing the steps performed in the training Kawano machine; Figure 21 is a flowchart of the procedure for extending the found route. Figure 22 is a flowchart of the operation of the stack decoder. Figure 23 is the *elementary sound machine. Figure 24 is a diagram showing the phonemic sound grid, Figure 2S,
Figure 1 and Figure 25.2 are diagrams showing the polling method of the present invention, Figure 25 is a diagram showing the combination of Figures 25.1 and Figure 25.2, and Figure 26 is the count value of the label. Diagram showing distribution, No. 27
Figure 28 shows the number of times each single phone generates each label during the training period, Figure 28 shows the sequence of sounds that make up a word, and Figure 29 shows what is expected for a certain word for each label. Figure 30, which shows the counted value, is a diagram showing the label and vote value for each word, Figure 31.
FIG. 32 is a diagram showing labels and penalty values for each word. FIG. 32 is a block diagram of a polling device of the present invention. Applicant International Business Machines Z's Co., Ltd. → Representative Patent Attorney Hitoshi Yamamoto
Akira (1 other person) 1 Tsukasa liquid mosquito person's thickness''! Sueen Tae Goon Marele 12th figure L (10111213), Q (qOQI q2Q
3) Shiwoshi Takato w Gakuensha Figure 13 Figure 14 Figure 1, Uzeru Stroke 1 Luk 1 Tsuge and 'O 3 Harusho's Hagigoken 1 series (0)''-2-'-a- 11---1---1111---4-THE. (b) 3 Cat α5Sy Teruji 11 11 1 Tai ε Ya 1〈 Paro Figure 19 “To BE ORNOT To BE” & Sukkuu i6 Step, Up Figure 23 Phonemic fJ Sound Mamarin 1 t 2 t3 Phonemic lattice Figure 24 - N ~ Tsuka I Σ Figure 26 Graph's expected fixed number Figure 27 Figure 28 Figure 29 Value of the ticket Figure 30

Claims (2)

[Claims] (1) A method of speech recognition comprising measuring the likelihood of a word from a vocabulary in response to speech input, the method comprising: (a) selected from an alphabet of multiple labels in response to speech input; generate a string of labels, each representing a type of sound; (b) compute the label vote values representing the likelihood that each label is generated when a given word is pronounced; c) accumulating the vote values of at least a plurality of labels among the labels generated in the string for the target word, such that the accumulated vote value is the target word; A speech recognition method that expresses the likelihood of (2) for selecting probable words from a vocabulary, each word being represented by a sequence of at least one probabilistic finite state sound machine, and a speech processing device generating a speech label in response to speech input; In a speech recognition device: (a) means for forming a first table such that each label of the alphabet gives, for each word of the vocabulary, a vote value representing the likelihood that that word generates that label; (b) For each word in each label of the alphabet,
(c) means for forming a second table in which the labels are assigned a penalty representing the likelihood of not being generated according to the model of the word; determining the likelihood of a particular word for a given string of labels, including means for combining the vote value of a label with the penalty of all labels in the string corresponding to that particular word; and a voice recognition device.