JPS59126598A - Voice recognition method and apparatus - Google Patents

Abstract

(57) [Summary] This bulletin contains application data before electronic filing, so abstract data is not recorded.

Description

DETAILED DESCRIPTION OF THE INVENTION The present invention relates to a speech recognition method and apparatus, and specifically:
A method and apparatus for recognizing one or more keywords in a continuous audio signal in real time.

related.

Various speech recognition systems are conventionally known which recognize isolated utterances by comparing a suitably processed unknown isolated audible (audio) signal with one or more child-prepared known keyword signals. More have been suggested.

The term "keyword" is used herein to mean a combination of some V phonemes and sounds, such as part of a syllable, word, phrase, etc.

Although many systems have had limited success, one system in particular has been successfully used commercially to recognize isolated keywords. Operating substantially in accordance with the method described in U.S. Pat. This method has yielded good results, providing a method for recognizing one of a limited range of keywords, provided that the unknown audible signal occurs for a sufficient period of time. It relies on the presumption that it is limited and contains only the utterance of a single keyword.

In continuous audible signals, such as continuous speech speech, where keywords' boundaries are not previously known or marked, keywords are used to segment the incoming audible data, i.e., between phonemes, syllables, words, sentences, etc. In order to determine the boundaries of linguistic units prior to the start of the keyword recognition process,
Various methods have been devised. However, these conventional continuous speech systems have had limited success, in part because no satisfactory segmentation method has been found. Additionally, there are other considerable problems. For example, - comprehensively, limited vocabulary cannot be recognized with a low false alarm rate; recognition accuracy is very sensitive to differences in voice characteristics of different speakers; For example, the system is very sensitive to distortions of the audio signal being analyzed, such as those commonly occurring in audio signals transmitted in common telephone communication equipment.

U.S. Patent Nos. 4.227.176 and 4,241,32
No. 9 and No. 4,227,177, respectively, describe commercially acceptable and effective techniques for successfully recognizing keywords in continuous speech in real time. The general methods described in these patents are now commercially available and have been shown to provide virtually high fidelity and low error rates in shore-independent situations, both experimentally and in practical trials. I understand. however,
Even these techniques, which are at the current state of the art, and the concepts for which they were developed, have drawbacks in both false alarm rates and speaker independent performance.

It is therefore a primary object of the present invention to provide a speech recognition method and apparatus having improved effectiveness in recognizing keywords in continuous unmarked audible signals. It is another object of the present invention to provide a method that is relatively insensitive to phase and amplitude distortions of the unknown audible input signal data, relatively insensitive to variations in the articulation rate of the unknown audible input signal; It is intended to provide a method and apparatus that responds equally well to speakers and thus different speech characteristics, has reliability Σ1 and an improved lower false alarm rate, and operates in real time.

The present invention relates to a speech analysis system for recognizing at least one predetermined keyword in an audible signal. Each keyword is characterized by a pattern template with at least one target pattern, each target pattern representing a short-term power spectrum. It has a dwell time period and a minimum dwell time period.

The method of the invention is characterized by the step of forming from an audible human input signal a series of frame patterns representative of the audible signal at a repetitive frame rate. Each frame pattern is associated with a frame time. Thereafter, for each frame pattern 7 a numerical measure of the frame similarity for the selected one of the target patterns is generated. Preferably.

A numerical measure is generated representing the similarity of each frame turn for each target pattern.

At each frame time, for each keyword, a numerical word score is accumulated using the numerical measurements described above, representing the likelihood that the keyword will finish in the frame time in which it currently exists. This accumulation step includes accumulating numerical measurements for each successive series of repeatedly formed frame patterns, starting with a numerical measurement of the similarity of the current frame pattern to the target pattern occurring at the end of the keyword. Thereafter, at least a preliminary keyword recognition determination is generated whenever the numerical value for the keyword thus determined exceeds a predetermined recognition level.

In another aspect, the invention relates to a speech recognition apparatus having means for forming a series of frame patterns, each associated with a frame time, representing an audible signal from an audible signal at a first repetitive frame rate. Furthermore, means are provided for generating, for each frame pattern, a numerical measure of the respective similarity of the frame pattern with a selected one of the target patterns. Preferably, similar to the method described above, numerical measurements are generated for each frame pattern for each target pattern.

The storage element sums, for each frame time and each keyword, a numerical word score representing the likelihood that the keyword will finish in the frame time in which it then exists. This sum is determined for each frame time and each keyword. This storage element stores, for each keyword, a numerical measure of the breaking of a consecutive series of repeatedly formed frame patterns, starting with a numerical measure of the similarity between the current frame pattern and the last target pattern of the keyword. , including a device for accumulating.

The apparatus further features means for generating at least a preliminary keyword recognition signal whenever the accumulated value for the keyword exceeds a predetermined criterion.

Other objects, features and advantages of the invention will become apparent from the following description of preferred embodiments of the invention, taken in conjunction with the accompanying drawings.

Note that in the drawings, corresponding elements are given corresponding reference numerals.

In one of the particular preferred embodiments described herein, speech recognition includes specially configured analog and digital processing of an incoming audible data signal, generally audio (speech). The system includes electronic equipment and a general purpose digital computer programmed in accordance with the present invention to perform certain other data conversion steps and numerical evaluations. The division of tasks between the hardware and software parts of the system was done in order to obtain a system that can perform voice recognition in real time at a low cost.

However, some portions of the tasks being performed in the hardware of this particular system could be adequately performed in software, and some portions of the tasks being performed with the software programming of this particular example would be
Other embodiments could be implemented with special purpose circuitry. In this latter regard, hardware and software embodiments of the device, if available, will be described.

As mentioned above, in accordance with one aspect of the present invention, an apparatus is provided for recognizing keywords in a continuous audio signal even when the signal is distorted by, for example, telephone lines. Thus, particularly in FIG. 1, the voice input signal designated at 10 may be considered a voice signal generated by a carbon transmitter and receiver over a telephone line spanning any distance and any number of switches. can. Therefore, a representative example of the present invention is the recognition of keywords in audible data supplied via an unknown source (speaker independent system) and received via a telephone system. On the other hand, a human input signal is any audible data signal, such as an audio human input signal, derived from a wireless communication link, such as a commercial broadcast station, or a private communication link.

As can be seen from the above description, one method and apparatus of the present invention relates to the recognition of audio signals that include a series of sounds, phonemes, or other recognizable symbols. In this specification, "keyword", "series of target patterns",
Although reference is made to either "template pattern" or "keyword template," these four terms are generic and considered equivalent. This is a convenient way to represent a recognizable set of audible sounds, or a substitute thereof, that the method and apparatus can detect. These terms should be interpreted broadly and generally to encompass everything from a single phoneme, syllable, or sound to a series of words (in the grammatical sense) as well as a single word. .

The analog-digital (A/D) converter 13 is
It receives the incoming analog data on line 10 and converts the signal amplitude of that data to digital form. The exemplary ψ converter converts the input signal data into 12
It is converted to binary representation of bits, but the conversion is 8000 bits.
It happens at a rate of times/second. In other embodiments, other sampling rates may be employed. For example, a speed of 16 strings 2 can be used if a high quality signal is available. The converter 13 supplies an autocorrelator 17 via its output 15.

Autocorrelator 17 processes the digital input signal to generate short term autocorrelation functions 100 times in 51 seconds and provides seven outputs via line 19 as shown. Each autocorrelation function has 32 values or channels, and each value is calculated into a 30-bit solution.

The autocorrelator will be described in more detail below in connection with FIG.

The autocorrelation function on line 19 is Fourier transformed by a Fourier transform device 21 to obtain the corresponding short-term processed power spectrum via line 23. The spectrum is
It is generated with the same number of repetitions as the autocorrelation function, ie, at a rate of 100 times/second, and each short-term power spectrum has 31 numerical periods with each 16-bit solution.

As will be appreciated, each period of spectrum 31 represents a single park within a frequency band. The Fourier transform device may also include a Hamming or similar window function to reduce unwanted adjacent band responses.

In the first illustrative embodiment, the 7-layer transform as well as subsequent processing steps are performed using peripheral array processors to speed up the operations required iteratively according to the method. The process is carried out under the control of a general purpose digital computer programmed in the following manner. The specific computer employed will be provided by Digital Corporation, located in Massachusetts.
It is model FDP-11 manufactured by Equipment Corporation. The particular array processor employed is described in U.S. Patent No. 4.2, assigned to the assignee of this application.
28.498. The program described below in connection with FIG. 3 is set up largely based on the capabilities and characteristics of these available digital processing units.

The short-term processed power spectrum is equalized for frequency response as indicated at 25. However,
This equalization is performed as a function of the peak amplitude occurring within each frequency band or channel, as will be shown in detail below. The spectrum of the frequency response on #26 is equalized.

generated at a rate of 100/sec, and each spectrum is
It has 31 numerical periods evaluated with an accuracy of 16 pits. To facilitate the final evaluation of the incoming audio data, line 2
The windowed spectrum equalized frequency response on 6 undergoes an amplitude transformation as indicated at 35.

This imposes a non-linear amplitude transformation on the incoming spectrum. This transformation will be described in detail later, but at this point it is worth mentioning that it improves the accuracy with which the unknown incoming audio signal can be matched with the reference yellow keyword. In the illustrated embodiment, this transformation is performed on all of the frequency response-equalized but windowed spectra at some point before comparing the spectra to the target pattern representing the reference word keyword.

The amplitude transformed and equalized short term vector on line 38 is then compared to the keyword target pattern at 40. The keyword target pattern indicated by 42 is conversion.

Represents the keyword of the reference plane element in statistical terms with which the equalized spectra can be compared. In this way, candidate words are selected according to the stringency of the comparison, and in the illustrated embodiment, this selection process eliminates overall unsuitable pattern sequences and minimizes the possibility of missing keywords. designed to. A recognition determination is provided via line 44.

Referring to FIG. 1A, the speech recognition system of the present invention employs a controller 45, which may include, for example, F
It may be a general purpose digital computer such as a DP-11 or a hardware controller specifically incorporated into the system.

In the illustrated embodiment, controller 45 receives preprocessed audio data from preprocessor 46 . The preprocessor will be described in detail in conjunction with FIG. Preprocessor 46 receives an audible human analog signal on line 47 and provides processed data to the control preprocessor on interface line 48.

Generally, the operating speed of the control processor is not fast enough to process incoming data in real time if it is a general purpose processor. As a result, it is advantageous to employ various special purpose hardware to effectively increase the processing speed of element 45. In particular, U.S. Pat.
, 228,498, provides significantly increased array processing power by exploiting pipeline effects. In addition, the fifth
．． As detailed in connection with Figures 6.7 and 8, the likelihood function processor 48b can be used in conjunction with a vector processor to further increase the operating speed of the device by a factor of ten.

In a preferred embodiment of the invention, control processor 45
Although is a digital computer, in other specific embodiments described in connection with FIGS. 9 and 10, a significant portion of the processing power is implemented external to the control processor in serial processor 49. The structure of this processor will be described in more detail below in connection with FIGS. 9 and 10. Thus, the apparatus for implementing speech recognition illustrated herein is unique in its speed and ability to be implemented in hardware, software or advantageous combinations of hardware and software. It is something.

Next, the preprocessor will be explained.

In the apparatus illustrated in FIG. 2, the autocorrelation machine with its inherent averaging effect is applied to an analog-to-digital converter 13 acting on incoming analog audio data, typically an audio signal, supplied via line 10. This is performed on a digital data string generated by converter 13
generates a digital human input signal on line 15. Digital processing functions as well as analog-to-digital conversion are timed under clock oscillator 510 control. The clock oscillator generates a basic timing signal of 256,000 pulses/second, and this signal is fed to a frequency divider 52 to obtain a second timing signal of s,ooo pulses/second. The slow timing signal controls analog-to-digital converter 13 as well as latch register 53. The conoratch register thus holds the 12-bit result of the last conversion until the next conversion is completed.

The autocorrelation product is generated by a digital multiplier 56 which multiplies the number contained in register 53 by the output of a 32 word shift register 58.

Because register 58 operates in a circular mode and is driven by a high speed clock frequency, one rotation of shift register data is performed for each analog-to-digital conversion.

The human power for the shift register 58 is − times cycle size,
- from register 53 once during the roll. Digi,
One power supply to the multiplexer 56 is provided directly from the latch register 53, and the other power is provided via the multiplexer 59 from the current output of the shift register (with one exception described below). Multiplications are performed at high clock frequencies.

In this way, each value resulting from the A/'I) transformation is multiplied by each of the previous 31 transformed values. As will be apparent to those skilled in the art, the signal generated by K is equivalent to multiplying the human signal by a signal delayed by 32 different time increments (one delayed 0). To obtain a zero delay correlation, i.e. to produce a power of the signal, multiplexer 59 multiplies the current value of latch register 53 by itself at the time each possible value is being introduced into the shift register. This time□
The switching function is indicated at 60.

Again, as will be apparent to those skilled in the art, the product of one transform and its 31 predecessors does not fairly represent the energy distribution f, or spectrum, for a given sampling interval.

Therefore, the apparatus of FIG. 2 averages these multiple sets of products.

The accumulation step for averaging is connected to an adder 65 and
Provided by 32 word shift registers 63 forming a set of 32 accumulators. That is, each word is added to the corresponding increment from the gauge multiplexer, and then
Can be recycled. This loop passes through a gate 67 which is controlled by a divide-by-N 69 driven by a low frequency 4-way signal. Divider 69 divides the low frequency clock by a factor that determines the number of cap-to-cap autocorrelation functions that are accumulated and thus averaged before shift register 63 is read.

In the illustrated implementation, 80 samples are accumulated before being read. In other words, N for N divider 69 is equal to 80. After the 80 transform samples have been correlated and accumulated, divider-by-one 69 triggers computer interrupt circuit 71 via line 72. At this point, the contents of shift register 63 are serially read into computer memory via appropriate interface circuitry 73.

The 32 consecutive words in the register are presented to the computer via interface 73 in terms of numbers. As is obvious to those familiar with the technology, this data transfer from the peripheral unit, i.e. the autocorrelator, to the computer is common.

This may be accomplished by direct memory access methods.

It will be seen that based on the 80 samples being averaged with an initial sampling rate of 8000, 100 averaged autocorrelation functions are provided to the computer every second.

While the contents of the shift register are being read from the computer, gate 67 is closed so that each word of the shift register is reset to zero, allowing the accumulation process to resume.

Expressed mathematically, the operation of the device shown in FIG. 2 can be described as follows.

Assuming that the analog-to-digital converter generates a time sequence S (t), where t=o, To, 2'
l'. .

--1-T, is the sampling interval (1/8000 seconds in the example embodiment), and the example digital correlation circuit of FIG. 2, ignoring start-up ambiguities, calculates the autocorrelation function It can be thought of as a thing.

a(j, t)=Σ S(t+kTo)S(t+(k,)
)To) filk=0 where j=0. 1. 2', ---31, 280 To.

160'fo　-, 80n'l'o, -m-.

These autocorrelation functions correspond to the correlation outputs on line 19 of FIG.

Referring to FIG. 3, the digital correlation diagram operates to continuously transmit a series of data blocks to the computer at a rate of one correlation function every 10 milliseconds.

This is indicated at 77 in FIG. Each data block represents an autocorrelation function induced into a corresponding subdivision time interval. As mentioned above, the exemplary autocorrelation function is provided to the computer at a rate of 100 32 word functions per second. This analysis interval is referred to below as a "frame".

In a first illustrative embodiment, processing of the autocorrelation function data is performed on a suitably programmed dedicated digital computer. A flowchart containing the functions provided by the computer program is shown in FIG. However, various of the steps may be performed by hardware (described below) rather than software, and some of the functions performed by the apparatus of FIG. 2 may correspond to the flowchart of FIG. It should be pointed out that this can also be accomplished in software by making modifications.

The digital correlator of FIG. 2 performs a time-averaging operation of the instantaneously generated autocorrelation function, but the average autocorrelation function read out to the computer does not interfere with the processing and evaluation of the j1st order of the samples. Contains some kind of anomalous discontinuity or non-uniformity. Therefore, each block of data, i.e. each autocorrelation function a(j, t,), is
First, it is smoothed with respect to time. This is indicated at 78 in the flowchart of FIG. A preferred smoothing method is one in which the smoothed autocorrelation output a, ('j, t) is given by the following formula.

as(j, t)=Ooa(j, t)+C1a
(j, t-T)+O,a(J, t-2T) (2
) where a(j, t) is the unsmoothed human autocorrelation function defined in equation (1), a5(j, t) is the smoothed autocorrelation output, j represents the delay time, and t is Representing real time, T represents the time interval @(frames) between successively generated autocorrelation functions, which in the preferred embodiment is equal to 0.01 seconds. Weighting is the function C6+ OH,
Cg is preferably 1/4.1 in the illustrated embodiment
/2.1/4 is preferably chosen, but other values may also be chosen. For example, a smoothing function that approximates a Gaussian impulse response with a cutoff frequency of 20 Hz could be implemented in computer software. However, experiments have shown that satisfactory results can be obtained with the easy-to-implement smoothing function exemplified by Equation (2).As mentioned above, the smoothing function is Applied separately.

Although the following analysis involves various operations on the short-term Fourier power spectrum of the audio signal, for simplicity and speed of processing, the transformation of the autocorrelation function into the frequency domain is described in an illustrative example. is implemented using 8-bit arithmetic.

At the high end of the bandpass around 3 KHz, the spectral power density is reduced to a level insufficient for resolution in 8-bit quantities. Therefore, the frequency response of the system is 6
Slanted at a rising rate of db/octave. This is indicated by 79. This high frequency enhancement is accomplished by taking the second derivative of the autocorrelation function with respect to that variable, the time delay.

The differential operation is as shown in the following equation.

b(j, t)=-a(j+1, t)+2a(j,
t) -a(j-1, t) (3) To find the differential value with respect to j = 0, since the autocorrelation function is symmetric with respect to 0, a (-j, t) = a (10 J, t ).

Furthermore, since there is no data for (32), it is assumed that the differential value at j-31 is the same as the differential value when J=30.

As shown in the flowchart of FIG. 3, the next step in the analysis procedure after high frequency enhancement is to calculate the signal power in the current frame interval by finding the peak absolute value of the autocorrelation.

The estimated value of power P(t) is as follows.

P(t)=maxlb(i,t)l (
4) To prepare the autocorrelation function for the 8-pit spectrum analysis, the smoothed autocorrelation function was block standardized (at 80) with respect to P(t) and the top 8 of each standard value
The pit is manually input to the spectral analysis software.

Therefore, the standardized and smoothed autocorrelation function is:

c(j, t)=127b(j, t)/p(t)
(51 A cosine Fourier transform is then applied to each time-smoothed, frequency-weighted, standardized autocorrelation function c(j,t) as indicated at 80 to produce a 31-point power spectrum. .Cosine value Ma)
IJ Lux is given by the following equation. That is, S(iJ)
=126g(i)(cos(2πi/8000)f(j
)).

J=o, 1.2. L-, 31(6) where S(i
, j ) is f(j)11,! at time t. The spectral energy of the band centered at, g (i)
=1/2 (1+cos 2πIA3) is the (Hamming) window function envelope to reduce sidelobes, and f(j)=30+1000(0,0552j+0.43
8) 110.63 out.

j=0. 1. 2. -----, 31
(71 This is the analysis frequency equally spaced on the tonic pitch so-called "mel" curve. As can be seen, this is the frequency of the bandwidth of a typical communication channel of approximately 3000-50oOI-(,!) The main pitch for (
mel scale) corresponds to the frequency axis spacing.

Since the spectral analysis requires adding lags from -31 to +31, only positive values of J are required, assuming that the autocorrelation is symmetric about zero. However, to avoid calculating the lag 00 term twice, the cosine matrine, x, is adjusted as follows.

S(0,j)=126/2=63, for remainder 3
(8) Thus, the calculated power spectrum is given by the following equation.

(9) Here the second result corresponds to the frequency f).

As can also be seen, each point or value within each spectrum represents a corresponding frequency band.

Although this Fourier transform can be performed entirely within conventional computer hardware, the process could be considerably speeded up by utilizing an external hardware multiplexer or Fast Fourier Transform (FF'l') peripheral.

However, this type of modular structure and operation is well known in the art and will not be described in detail here. Advantageously, the hardware fast Fourier transform peripheral incorporates a frequency smoothing function, in which case each spectrum is
In the window weighting, the frequency is smoothed according to the function g(1).

This is performed at 83 of block 85, which corresponds to performing a Fourier transform in hardware.

If there is considerable background noise, an estimate of the background power spectrum can be obtained at this stage by S
' must be subtracted from (j, t). The frame(s) selected to represent noise must not contain any audio signal. The optimal rule for selecting the noise frame spacing will vary depending on the application. If speakers engage in intercommunication with machines controlled, for example, by a voice recognition device, it is advantageous to arbitrarily select a frame, for example in an interval immediately after one machine has finished speaking with its voice response unit. . If there are fewer constraints, the noise frame can be found by selecting the frame with the lowest amplitude of audible human effort during the past 1-2 seconds.

Once the successive smoothed power spectra are received from the fast Fourier transform peripheral 85, determine the peak power spectral envelopes (generally different) for the spectra from the peripheral 85 and fast Fourier transform them accordingly, as described below. Equalization of the communication channel is achieved by changing the output of the conversion device. Each newly generated peak amplitude corresponding to and modified by the incoming windowed power spectrum S'(J,t), where J is assigned to a plurality of frequencies of the spectrum, is It is the result of fast attack, slow decay, and peak detection functions for each spectral channel or band. The windowed power spectrum is normalized with respect to each period of the corresponding peak amplitude spectrum.

This is designated at 87.89.91.

In the illustrated embodiment, the "old" peak amplitude spectrum p(J,t;-'I'), determined before receiving the new windowed spectrum, is replaced by the newly arrived spectrum S'(j , t) in a frequency band to frequency band manner.

Then, the new peak spectrum p(j, t).

Generated according to the rules below. The power amplitude of each band of the "old" peak amplitude spectrum is multiplied by a fixed fraction, e.g. 1023/1024 in this example. This corresponds to the slow decay part of the peak detection function. The incoming spectrum s'( j, t) frequency band J
If the power amplitude of is thicker than the power amplitude of the corresponding frequency band of the decay peak amplitude spectrum, then the decay peak amplitude spectral value for that (or those) frequency bands is equal to the spectral value of the corresponding band of the incoming windowed spectrum. It can be replaced with This corresponds to the fast attack part of the peak detection function.

Mathematically, the peak detection function can be expressed as:

That is, p(j, t)=max p(j, t-T)-(1-
E) ・p(t) −8(j, t).

j=0.1. , -----, 31 M here J
is reassigned to each of the frequency bands, p(j, t) is the resulting peak spectrum, and p(j, t-'I')
is the “old” or preceding peak spectrum, and S
'(j,t) is the newly arrived partially processed power spectrum, PCl) is the power estimate at time t, and r is the decay parameter.

According to Equation H, the peak spectrum typically collapses at a rate of 1-E in the absence of higher values of spectral input. Usually E is equal to "/1024. However,
It may be undesirable to allow the peak spectrum to decay during periods of silence, especially if rapid changes in the communication channel or audio characteristics are not expected. To limit the silent frames, the same method adopted to select the background noise frames is adopted. The amplitude of the past 128 frames (p(t)
) is examined and the minimum value is found. If the amplitude of the current frame is less than four times this minimum value, the current frame is determined to be silent, and for E:
The value "0" is substituted for the value 1/1024.

After the peak spectrum is generated, the resulting peak amplitude spectrum p(j,t) is calculated by averaging each frequency band peak value with the peak value corresponding to the adjacent frequency of the newly generated peak spectrum. Smoothed (89). Thus, the total frequency bandwidth contributing to the average value is approximately equal to the representative frequency spacing between formant frequencies. As will be apparent to those familiar with the art of speech recognition, this interval is on the order of approximately 1000 Hz.

Averaging in this particular manner preserves the useful information in the spectrum, namely the local fluctuations representing the 7-ormant resonance, while suppressing the overall enhancement of the frequency spectrum. In a preferred embodiment, the peak spectrum is smoothed in frequency by a moving average function that cuts seven adjacent frequency bands. The average function is of first order.

At the end of the bus band, p(k,t) becomes 0 for k less than 0 and k greater than 31. The standard envelope hQ) takes into account the number of valid data elements actually added. Thus, h(o) = 7/
4. h(1) = 715, h(2) = 7. /
6.

h(3) = 1, ----- h(28) = 1
, h(29) = 7/6.

h(30) = 775, and h(31) = 7
/4. The obtained smoothed peak amplitude spectrum e(
j, t) is.

Next, it is used to standardize the power spectrum just received and perform one-frequency equalization, which means that the amplitude value of each frequency node of the incoming smoothed spectrum S/(5, t) is
Smoothed peak spectrum e(j.

t) by the corresponding frequency band value. Mathematically, this can be expressed as first order.

5n(j,t)'(8'(j,t)/e(j,t))3
2757 (12) where 5n(f,t) is the peak normalized and smoothed power spectrum and J
is assigned to each frequency node. This step is indicated at 91. Here, we obtain a series of frequency-equalized and standardized short-term noise spectra, which emphasize changes in the frequency content of the incoming speech signal and which are generally characterized by long-term frequency enhancement or distortion. is suppressed. This frequency compensation method is more effective than conventional frequency compensation systems, where the compensation criterion is the average noise level for either the entire signal or each frequency band. It has been found to be very advantageous in the recognition of transmitted audio signals.

Note that although the sequential spectra have been variously processed and equalized, the data representing the incoming audio signal still includes spectra occurring at a rate of 100/sec.

The normalized and frequency equalized spectrum as indicated at 91 undergoes an amplitude transformation as indicated at 93. This is done by performing a non-linear scaling operation on the spectral amplitude values.

Choosing an individual equalized and standardized vector torque such as Sn(j − t ) (from Equation 12), where J indicates the different frequency bands of the spectrum and t stands for real time, the non- Surface line scaled spectrum X(j
, t) are defined by a quadratic linear fractional function.

5n(j, t) -A X(j, t)=128-5(., t) +A j
-0, 1, ---, 30"" where A is the spectrum Sf from 3=θ to 31] (j.

t) and is defined as below.

Here, J indicates the frequency band of the power spectrum. The 31 period of the spectrum is expressed as
1, t ) = 16 Log, A
% This scale function (Equation 13) does not have a flexible threshold and gradual saturation effect on spectral intensities that deviate significantly from the short-term average value A. Mathematically stated, it is approximately linear for intensities near the average, approximately logarithmic for intensities away from the average, and substantially constant for extreme intensity values. In the case of a logarithmic scale, the function X(j,t) is symmetric about 0 and exhibits threshold and saturation behavior that produces a proportion function that stimulates the auditory nerve. In fact, the entire recognition system performs considerably better for this particular non-linear scale function than for either linear or logarithmic scales of spectral amplitude.

In this way, a series of short-term power spectra X(
j, t) (here, 1=0.01°0.02.0.03
, 0.04, --- 1 second, j=O, ---.

30 (corresponding to the frequency band of the generated power spectrum))) are generated. 32 words are prepared for each spectrum, and A (Equation 15), that is, the value of the average value of the spectrum values, is stored as 32 words. This amplitude-converted short-term power spectrum, referred to below as "one frame", is a first-in first-out, in the illustrated embodiment, with a storage capacity for 256 32-word spectra, as indicated at 95. Stored in circular memory. (Thus, in the illustrated embodiment, 2.56 seconds of audio input signal is available for analysis. This storage capacitance can be stored at different real-time times for analysis and evaluation, if necessary. To provide a protean recognition system that can select a spectrum and thus move forward or backward in time as required for analysis.

Thus, the frames for the last 2.56 seconds are stored in circular memory and available when needed. In the illustrated embodiment, during operation, each frame is stored for 2.56 seconds. Thus, the frame that entered circular memory at time t□ is lost from memory 2.56 seconds later when a new frame corresponding to time t + 2.56 seconds is stored. In other words, it is shifted.

The frames passed through the circular memory are compared, preferably in real-time, with known word combinations to determine and identify the keywords in the input turn. Each word conjugation is represented by a template knot that statistically represents a plurality of non-overlapping multi-frame (preferably three-frame) design sets or multiple processed word vectors formed into a Googet pattern. These notuturns may be selected to represent the meaningful acoustic events of the conjugation word, and are then stored at 99.

The spectrum forming the design set pattern is line 1
To handle a sequence of unknown speech inputs over 0, the system described above (FIG. 3) is used to generate words spoken in various situations.

Thus, each plane word has a generally plural series of design turns p(1)□ associated with it.

p(i)Q,-, and each pattern gives one indication for its first keyword in the region of the short-term power spectrum. The collection of design set patterns for each keyword forms a statistical basis for generating target patterns.

In an exemplary embodiment of the invention, design set patterns p(i), 5 can each be thought of as a 96-element array constituting three selected frames arranged in series. The frames forming the pattern should be at least 30 milliseconds apart to avoid unnecessary correlation due to smoothing in time. In other embodiments of the invention, other sampling methods may be implemented to select frames.

However, a preferred method is to select frames - a fixed duration, preferably 30 milliseconds apart, and space the non-overlapping design set patterns for a time interval that defines the keywords. That is, the first
The design set pattern p□ corresponds to the part near the start point of the keyword, the second pattern p2 corresponds to the part after the time, and so on, and the pattern p+
+ I)Q + --- forms a statistical criterion, or word template, for the set of target patterns to which the incoming audio data is matched.

Target pattern t1. tQ --- are each p
(i) consists of statistical data generated from the corresponding p(i)j by assuming that j consists of independent Gaussian variables. This assumption allows likelihood statistics to be generated between the incoming data and the target pattern, which will be described below. Thus, the target turn is
It consists of an array containing, as an entry, the mean standard deviation and area standardization factor for the corresponding design set pattern array entry collection. More accurate likelihood statistics will be explained later.

As is obvious to those skilled in the art, almost every keyword has more than one contextual and/or regional pronunciation, and therefore more than one "spelling" of the design set pattern. are doing. Thus, a glossary word with the patterned spelling p□, p* r--1 described above is in practice generally p(iL,I)(i)
Q----1i=1.2°----, it can be expressed as M. Here each p(i)j is a possible alternative description for the third class of design set patterns, and there are a total of M different spellings for each word.

Therefore, the target pattern t□+ t2 ”−+
t1 represents, in the most general sense, a plurality of alternative statistical spellings for the design set pattern, each of the first blueprint or class. Thus, in the illustrated embodiment, the term "target pattern" is used in a very general sense, such that each target pattern has two or more permissible alternative "statistical spellings." It is possible.

The preliminary processing of the incoming unknown audio signal and the audio signal forming the reference pattern is now complete.

Next, processing of the stored spectrum will be explained.

Referring to FIG. 3, first, the spectrum or frame stored at 95 representing the incoming continuous audible data is converted into a stored target pattern template shown at 99 representing the field element keyword according to the method described below. compared to

For each 10 ms frame, the stored reference pattern and comparison pattern are the current spectral vector S(j, t), the spectrum 3 frames ago S(j
, t-0,03), and the spectrum S 6 frames ago
(j, t O, 06) are formed at 97° by adjoining them to form the following 96-element pattern.

Each multiframe pattern thus formed is described, for example, in U.S. Pat.
No. 7,176 and No. 4,227,177. However, while these transformations are useful in the context of the present invention, they do not form part of the invention and do not explain how the teachings of the above-identified US patents may be adapted to the methods and apparatus taught therein. It will be obvious to engineers in this field whether to do so.

Thus, in example embodiments, the transformation can reduce cross-correlation, reduce dimensionality, and increase separation between target patterns. The multi-frame patterns that make up the equalized spectrum are the transformed pattern (or series of transformed patterns) as the target pattern (or series of target patterns).
is manually fed into a statistical likelihood calculation block, designated 100, which measures the probability of matching .

Next, calculation of statistical likelihood will be explained.

The multi-seven frame pattern x(J,t) formed as described above is supplied as input to the statistical likelihood calculation block. As mentioned above, this processor measures the probability that each successively given multi-frame pattern (representing one song of unknown human voice) matches each of the target patterns of the keyword template at the start of the machine. Provide value. Typically, each datum representing the target pattern has a slightly asymmetric probability density, but is nevertheless typically is statistically well approximated by a Gaussian distribution of . Here, 1 is the sequential designation of the element of the th target pattern.

A simple implementation of this process is to use different values of 1 and k.
It is assumed that the data associated with k are uncorrelated, and therefore the joint probability density for data X belonging to target pattern k is (logarithmically): That is, L (XIK) = p (x, x) = Σ1/21n2
Since the logarithm (var (i, K)) is a monotonic function, this statistic means that the probability of matching any one target pattern of the keyword template is greater than the probability of matching the target pattern of any other word millet. greater or lesser, or alternatively sufficient to determine whether the probability of matching a particular pattern exceeds a predetermined minimum level. Each input multi-frame pattern has its statistical likelihood L(X
IK).

The resulting statistical likelihood L(XIK) is interpreted as the relative likelihood of the occurrence of the target pattern named K at the time t that pattern X occurs.

As is well understood by those skilled in the art, ranking of these likelihood statistics constitutes speech recognition insofar as it can only be performed with a single target pattern. These likelihood bullets can be utilized in different ways in the overall system depending on the final function to be performed.

For probabilistic models, Gaussian distributions can be used (e.g., U.S. Pat. Nos. 4,241,329 and 4,2
27,176 and 4.227.177),
Laplace distribution, i.e. P (x) = (1/Jz s′)exp −(−fi
l X-m1/8') (where m is the statistical mean, S'
is the standard deviation of the variable X) requires less calculation,
For example, it has been found to perform much like the Gaussian distribution in the speaker-independent isolated word recognition method described in US Pat. No. 4,038,503. The degree of similarity L (x l k) between the unknown person's cuckoo turn

Since the accuracy scores L of a series of patterns are combined to form the accuracy score of a spoken word or phrase, the score L(XI k) for each frame is the best (minimum) score of all reference patterns for that frame. It is adjusted by decreasing . That is, L'(Xlk) -minL(Xli) 0
8) Therefore, the best fitting pattern for each frame will have a score of 0. The adjusted scores for the hypothesized set of reference patterns are accumulated for each frame and directly related to the probability that a favorable decision for the indicated sequence is the correct decision. A sequence score can be obtained.

Comparison of the unknown human scum vector pattern to the stored known pattern is accomplished by calculating the following function for the first reference pattern. That is, where sik is equal to 1/s'ik.

In a calculation performed in normal software, the following instructions would be executed to calculate the algebraic function 5lx-ul (Equation 19).

1. Calculate x-u. 2. Test the sign of x-u. 3. If x-u is negative, negate it to form the absolute value. 4. Multiply by i. 5. Add the result to the accumulator. In a typical speech recognition system with a 20-word surface element, approximately 222 different reference patterns would be provided. The number of steps required to solve this, not including indirect operations, is 5 x 96 x 222 = 106560 steps, which must be performed within 10 ms to keep up with the real-time spectral frame rate. Must be.

Therefore, a processor must be able to execute approximately 11 million instructions per second to just determine the likelihood function. Taking into account the requisite speed, U.S. Pat.
Dedicated Priority Function Hardware Module 2 Compatible with the System Vector Processor Disclosed in No. 28,498
00 (FIG. 5) is adopted.

In this dedicated hardware, the five steps mentioned above are performed simultaneously with two sets of independent variables, so in reality, ten instructions are executed in the time it takes to execute one instruction. carried out. Since the basic vector processor operates at a speed of 8 million instructions/second, the effective calculation speed for the likelihood function is 200
If adopted, the number of instructions will be approximately 80 million instructions/second.

Referring to FIG. 5, hardware module 200
employs a combination of hardware pipelining and parallel processing to enable simultaneous execution of ten steps. two identical parts 202.204
are each performing five arithmetic steps on independent input data independent variables, and the two results are combined by an adder 206 connected to their outputs. The cumulative value from the adder 206 is 1 to 96 in equation (19).
and this value is the sum of U.S. Pat.
, 498.

In operation, pipeline processing registers hold intermediate data for the following processing stages.

1. Input independent variable (clock operated register 208゜2
10, 212.214.216.218) 2.1C-
Absolute value of u (clock operated registers 220, 222) 5 Multiplier output (clock operated registers 224, 22
6) Once the input data is held in the clock-operated registers 208-218, the magnitude of χ-U is
, 230. Referring to FIG. 6, subtractor/absolute value circuits 228, 230 respectively calculate M1 and second subtractors 232, 234 (one calculates x - u, the other calculates u - x) and positive results. A multiplexer 236 is provided for selection. registers 208, 21
The input independent variables x and U on lines 238 and 240 originating from 0 are each 8-bit numbers from -128 to +127.

Since the difference output of the 8-bit subtracter may overflow to 9 bits (for example, 127 - (-128) = 2
55), extra circuitry is required and employed to handle arithmetic overflow conditions. This condition is determined by overflow detector 235. Therefore, the input is the sign of "X" (on +li! 235 a),
The sign of ruJ (on II 235b) and the sign of [x − u j (on line 235c).

Referring now to FIG. 7, the overflow detector, in this illustrative embodiment, includes three rickshaw gates 268.2.
70 and an OR gate 272. The truth table of FIG. 8 represents the one overflow condition as a function of input.

The overflow condition is handled by making four selections in multiplexer 2361 (which is the circuit that selects the positive subtractor output). These selections are line -24
It is defined in binary levels above 2 and 244. [242
The upper level represents the sign of X-H.0244 The upper sign represents overflow if it is 1. Thus, the choices are as follows.

11224 Line 224 0 0 Select output of subtracter 232 1 0
The select subtracter 234 multiplexer is controlled in this manner and acts like an 8-pole 4at switch. Shifting operations are performed combinatorially by connecting the subtracted outputs to the appropriate multiplexer outputs.

The shift has the effect of arithmetically dividing by two.

If an overflow occurs during subtraction, the output of the multiplexer will be the output of the subtractor divided by two. It is therefore necessary to recall this condition later in the calculation so that the final result can be multiplied by 2 to recover the correct scale factor. This restoration is done in the multiplexer after the last pipeline processing register.

Therefore, the pipeline processing registers 220°222.
An extra bit is provided at 224.226 and passes through the second multiplexer 248.250. These multiplexers have an overflow bit set (1
IC equal), each 8x8 bit multiplier 2
Shift up the multiplication product of 52.254 by 1 bit,
Multiply by 2. The multiplication operation can be performed on a standard integrated circuit device, such as the TRW part number MPY-8-HJ, which accepts an 8-bit number and outputs the product. Thus, multipliers 252, 254 multiply i and i on each clock pulse. The 100 value resulting in the 1x-ul product is properly timed by the extra data register 256.258).

The output of multiplier 252.254 is output to register 224.22.
6/(score is stored and output to the rest of the circuit via adder 206 via lines 260 and 262.

The same dedicated hardware module A/200 can also be employed to compute two-vector inner products, such as those required in matrix multiplication. This is accomplished with gating circuits 264.266 allowing bypass in subtraction, absolute value circuits 228.230. In this mode of operation, the 1 data rXJ and "i" input buses are applied directly to pipeline processing registers 220, 222 as multiplier inputs.

Next, word level detection processing will be explained.

A ``spelling'' of a keyword according to a preferred embodiment of the present invention is a set of reference pattern names or ``phones'' (target pattern) in a given order and the minimum and maximum associated with each phone in the spelling. This is the dwell time (duration time). Matching an unknown incoming cover turn sequence to a keyword spelling is accomplished by associating each input cover turn with a spelled phone. The degree of "attribute" is measured by the likelihood score of the pattern for a single phone. The overall one-word score for each odd input waste vector frame is calculated for each keyword spelling KM as follows.

Referring to FIG. 4, the current frame (corresponding to circle 402)
Assume that is the end of the keyword.

The word score for each keyword of the plane element is determined as follows. The 10th tribute to word score is
Whichever is better (or smaller) is the likelihood score of the current kataan or the immediately preceding phoneme for the final phoneme of the keyword spelling.

The next seven frames later in time (corresponding to circle 404) are examined next. If the minimum dwell time of the current phoneme has not yet elapsed, the contribution of the current (i.e., immediately preceding) pattern is determined by either (a) the likelihood score for the current phoneme or (b
) is the better of the likelihood scores for the immediately preceding phoneme. This contribution is added to the partial word score. If the minimum dwell time has elapsed, the likelihood scores for the current and immediately preceding phonemes are examined. If the score of the immediately previous phoneme is good, the immediately previous phoneme becomes the current phoneme (pass 4).
06), the score is cumulative to the word score. The minimum and maximum dwell times are then reset. Otherwise, the current phonetic note remains at current binding and its likelihood score is added to the word score. If the maximum dwell time for the current phoneme has elapsed, a penalty is added to the word score and the immediately previous phoneme becomes the current phoneme. The analysis is complete when the immediately preceding phoneme is absent, and the final word score is the contents of the word score accumulator divided by the number of frames from beginning to end (i.e., the average likelihood score per frame for spelling). ).

Detection thresholds for word scores are set to establish a trade-off between detection probability and false alarm probability. 420 if the word score for any spelling is better than the threshold value
(Figure 3) detection is declared. If two or more detections occur within too short a period of time, arbitration logic selects the best of the overlapping detections.

Since the assumed "current" phoneme changes monotonically during word spelling and never regresses to a previous state, the above description of the word detection method can be rewritten as a dynamic programming problem.

Next, one dynamic programming method will be explained.

Referring to FIG. 4A, according to this dynamic programming approach, keyword recognition can be expressed as a problem of finding a suitable path through an abstract state space. Each day (in this figure) represents a possible state, also called a dwell time position or register, through which the process of making a decision can pass.

Vertical break 11. The spaces s2o, 522M each represent hypothetical states that the decision-making process may pass through in determining whether a pattern currently matches or does not match a phoneme.

This space is divided into a mandatory dwell time portion 524 and an optional dwell time portion 526. The required dwell time portion is named by the minimum duration of a particular 1'' phoneme or pattern. The arbitrary dwell time portion represents the maximum additional duration of the pattern. Each day within the optional or required dwell time portion represents one frame time of the series of frames formed and corresponds to an interval of 0.01 seconds from frame to frame. Thus, each day identifies a hypothetical current phonetic position in one keyword spelling7, and also
of the pattern along with the number of frames (in 0,01 seconds) beyond that in that phoneme or target pattern that corresponds to the previous "circle" or position in that phoneme or target pattern that is assumed to have elapsed since the beginning of the current phoneme. Represents the current duration. After a pattern (phoneme) begins and a minimum dwell time period has elapsed, there are several possible paths to proceed to the node or location (circle) 528 of the next target pattern (iHI) noj11. This depends on when the decision is made to move on to the next pattern (phoneme) of spelling. These decision possibilities are represented in this figure by several arrows pointing towards the circle 528\C. The starting point of the next pattern (phoneme) is represented by a circle 528;
This transition to a phoneme) may be made from any node or position during any dwell time of the current pattern (phoneme) or from the last node of the required dwell time period.

U.S. Permit No. 4,241,329, No. 4,227,17
The keyword recognition method described in No. 6 and No. 4,227,177 uses a first node such that the likelihood score for the next pattern (phoneme) K is better than the likelihood score for the current pattern (phoneme) K. Perform the transfer.

That is, the frame matches the next phoneme or pattern better than the current phoneme or pattern. However, the total word score is the average pattern (phoneme) score per frame (ie, per node included in the bus).

The same definition of "total score" that applies to word scores up to the current node can be used to determine when a transfer should occur.

That is, it can be used to determine whether the transition to the next pattern should be made at the first opportunity, eg, corresponding to transfer indicator line 530, or at a later point, eg, corresponding to transfer indicator line 532. Optimally, the bus with the best average score per node is set to the next pattern (phoneme).
It becomes K to choose among them. U.S. Patent Nos. 4,241,329, 4,227,176 and 4,227,177
The standard keyword method for issues is the following pattern (
Since we do not test for potential basses after making the decision to move to a single note, we will therefore have made a nearly optimal decision as measured by the average score/node.

Therefore, the present invention employs an average score/node method for keyword recognition, and detects whenever the average score/node for the "best ending node" K of the last pattern of a keyword exceeds a predetermined threshold. is recorded.

The dynamic programming method requires that in each analysis time frame, a certain starting word just begins (i.e., 1
Requires the likelihood score (that some previous word or other sound just ended). In a closed vocabulary task button, providing this score is a straightforward matter. However, in the keyword task, the reference pattern for all expected sounds is also one
Definitions for all possible words are also not available.

There can be several ways to give a human score.

To illustrate this, it is necessary to further explain certain features of the dynamic programming method. The method is implemented by a defined ordinal array of accumulators (1), A(2), -m-, each storing a score corresponding to a particular sequence of patterns and pattern durations. The content of the first accumulator in the analysis frame at time t is displayed as person (11t). The likelihood score at time t for the reference pattern related to the first accumulator is L
It is displayed as (i, i).

The circular equation for the first frame (or beginning) of the target pattern K non-corresponding accumulators is as follows.

A(1, t): L(i, t,)+A(i-1, t-1
) of the next target pattern A(n,
i) is supplied with the best (minimum) score of the available accumulator for the preceding pattern (ie, the accumulator for which a transfer can be made for the next pattern).

That is, A(n, t) = L(n, t) +mi
n A(i, t-1) i=m, n-1 In this way the optimal duration for the target pattern is found.

As mentioned above, the word score for the detected keyword is the average likelihood score per analysis frame. This is the output score of the last target pattern in the current analysis frame (the score that would have been sent to the accumulator for the next pattern, if there was one) and the continuation of this word when the word started. It is the difference from the input score divided by time. The word duration associated with the accumulated word score as well as the target pattern length can be carried forward and updated from register to register.

The accumulator for the input score corresponding to the first register of the first pattern of keywords is denoted as A(o,t). The simplest input method is to use a direct m ramp function with constant slope a as the input likelihood score for the keyword recognition process.

\ Time Accumulator contents t A (0, t) A (1, t) A (2, t)
---0000 10L(1,1) L(2,1) 2 20 L(1,2)+OL(2,2)+
L(1,1)3 30 L(1,3)+20
L (2, 3) + L (1, 2) 10 At any time,
The number of additions performed in each accumulator is the same for all accumulators, so there are no minuses due to initialization. The effect of C is A(1,t) when C is small.
This can be understood from K by noting that A(2,t) tends to contain better scores than A(2,t), while if C is large then A(2,t) contains better scores.

The results may be biased such that the optimal pattern duration found by this method is too long or too short. Since the ramp function trap propagates through all accumulators, the durations of all patterns in words will be similarly biased.

Instead of accumulating a constant ramp, the contents of the first accumulator are recirculated to add the likelihood score of the current signal with respect to the first reference pattern of the desired keyword instead of one constant. You may. The contents of all remaining votes n units are accurate up to a constant that is subtracted when determining the word score per frame. This method is illustrated in the table below.

Time Accumulator content tA (o, i) aQ, t, )00
Q I L(0,1) L(1,1)2
L(02)4g0.1) L(1,2)+L
(0,1)3 goβ)l-L(C),2))-L(0
,1) L(1,3 pieces L(02)+L(0,1)
t A(2,t) --- 0 1L(2J) 2 L(2,2))-L(1,1) 3 L(2,3)+-L(1,2)+1(0,1 )4
L(2t4)l-L(1,3))-L(02))L(0
, 1) According to this method, the selection of the optimal pattern duration for the second and subsequent patterns of keywords is independent of the duration of the first pattern. on the other hand,
It is impossible to know from the contents of the accumulator how long the first repattern should be. This fact is shown in the above table by the substitution L(2,t)=L(1,t)=L(0,t,)
It is made clear by Vc.

Although three accumulators are provided and a pattern may have a duration of three analysis frames, all three accumulators always contain the same score and there is no unique minimum value to choose from. This problem only affects the judgment (evaluation) of the total word score, and does not affect the classification of subsequent patterns that become statistical data of the Utsugi reference pattern, for example. The presently preferred embodiment uses this method one with an arbitrary constant duration assigned to the first pattern of each keyword.

Next, reference pattern training will be explained.

Sample mean U and variance (
In order to obtain S/, multiple occurrences of each starting word are inserted into the speech recognition system and the entire statistical data of the corresponding preprocessed spectral frame is determined. Critical to the successful operation of the device is the selection of which human body vector frames should correspond to which target or reference patterns.

In the absence of good information, such as the important acoustic phonemes selected by humans for the input word, the time intervals between the start and end points of the spoken word were uniformly spaced apart. Divided into sample intervals. Each of these sub-intervals is associated with a unique reference pattern. One or more three-frame patterns are formed starting in each interval and classified according to the reference pattern associated with that interval. Subsequent examples of the same story word are:
Similarly, it is divided into an equal number of uniformly spaced intervals. The mean values and parances of the three-frame pattern elements extracted from the corresponding sequential intervals are cross-producted on the available gold side of the starting word to form a set of reference patterns for that word. The number of intervals (the number of reference patterns) is approximately 2 per unit of linguistic phoneme included in the opening word.
Or it should be 3.

4. For best results, the starting points of keywords are marked by a procedure that includes human examination of recorded audio waveforms and spectral frames. In order to perform this procedure automatically, it is necessary to speak the words one at a time and demarcate them silently, so that the device accurately finds the boundaries of the words. A reference pattern would be initialized from one such sample of each word spoken in isolation. The total harance is then set to a convenient constant in the reference pattern. The training material may then include occurrences representing the utterances to be recognized and with word boundaries as found by the recognition process.

After the statistical data from a suitable number of training utterances has been accumulated, the found reference pattern is then used in place of the initial reference pattern.

A second pass through the training material is then made. The word is then divided into sparsity-based time intervals by the recognition processor as in FIG. Each three-frame cover turn (or one representative angler cover turn for each reference pattern) is associated with a certain reference pattern by the pattern matching method described above. The mean values and parances are interproducted for 1 second so that they form a final set of reference patterns that are derived in a manner fully compatible with the method used by the recognizer.

The minimum (required) and maximum (required ten optional) dwell times are
Preferably it is determined during the training process. In a preferred embodiment of the invention, the device is trained using several speakers as described above. Furthermore, as described above, the recognition process automatically determines the boundaries of the pattern according to the process described above during the training procedure. The boundaries were thus recorded and identified by the device. Dwell time is stored for each keyword.

At the end of the training process, the dwell times for each pattern are tested and the minimum and maximum dwell times for the pattern are selected. In a preferred embodiment of the invention, a histogram of dwell times is formed and the minimum and maximum dwell times are set at the 25th and 75.100th quantiles, which provides high recognition while maintaining a low false alarm rate. Gives precision. Alternatively, other choices of minimum and maximum dwell times are possible, but there is a trade-off between recognition accuracy and false alarm rate. That is, if minimum and maximum dwell times are selected, higher recognition accuracy is generally obtained at the expense of a higher false alarm rate.

Next, the construction of a device realized using the present speech recognition method will be explained.

As noted above, in the presently preferred embodiment of the invention, the signal and data manipulations beyond those performed by the preprocessor of FIG. 2 are performed by the preprocessor of FIG. Digital Equipment Corporation's PDP that operates in combination with a dedicated vector computer processor -
It was configured to be executed and controlled by a Type 11 combinator.

In addition to using computer programming, the method of the present invention can be implemented using hardware.

Referring to FIG. 9, in one particular example of the hardware of the present invention, likelihood data from a likelihood data generation processor is provided to memory 302 over line 300. The memory 302 has sufficient storage capacity to record the likelihood scores of the input frame patterns for each of the target patterns of the starting keyword being detected. This likelihood score input data is available from the processor over line 300 and is transferred to memory 302 in a predetermined sequence at a high data rate.

This data comes from address output signal 306 on line 30.
stored in memory 302 according to the address output signal via 4. Address counter 306 is on line 300
is incremented by a pulse signal on count line 308 synchronized with the data on line 31
A reset signal via 0 resets it to the initial predetermined address.

The ninth example embodiment is a target pattern shift register memory 312 (a), 312 (b), --
-312(n), and each shift register memory can store likelihood score data for each of the previous 7 frames of the processed audio signal for a particular Targetera F pattern. can. These memories 312 are forward-backward shift registers that can be loaded with data via input lines 314 (in backward shift mode) and via manual input 315 (in forward shift mode). The output of each shift register memory 312 used in this is:
Memory data is available via output line 316 when it has been shifted K in the "forward" direction.

In operation, the memory 312 is shifted closer to the output of line 316 by shifting the contents of the register one data position "forward" with each (forward) clock pulse via line 318. Ru.

Correspondingly, each memory 312 shifts its contents one position backward, ie, closer to input line 315, with each (backward) clock pulse via line 319. In the illustrated embodiment, each memory has two
, 56 seconds of likelihood score data.

The output of each memory 312 via line 316 is connected via a respective gate element 321 to a control comparison circuit 320 . These comparator circuits 320, which will be described in detail in conjunction with FIG.
cumulative, standardized word scores via
and a word score accumulation complete signal via line 324.

In response to the word score accumulation completion signal indicating that the keyword recognition process has been completed according to the method described above, each comparison circuit 3
20 through line 324, the keyword determines whether (a) the normalized word score for that current frame exceeds a predetermined threshold level, and (b) the thread Words that exceed the threshold level are each examined to determine whether they should be considered for further decision processing.

There is one comparison circuit 320 for each keyword in a business. Each comparator circuit 320 thus receives the target signal of its keypad via line 326 as its input.
It has a respective output of memory 312 corresponding to a pattern. As will be described in more detail below, the comparison circuit consisting of a multiple comparison multiplexing element 330 and a duration counting process control element 332 is standardized against the assumption that the keyword ends at the then "current" frame time. and determine the cumulative word score.

The exemplary shift register memory 312 is configured as a recirculating "forward" shift mode or a non-recirculating "reverse" shift mode. In recirculating forward mode, shift register memory 312 receives its inputs over line 315. These inputs are gate element 3
33 and 321 to memory 312.

In the non-circular mode of operation, memory 302 also receives its input on line 314 via gate element 338.

In operation, R is loaded backwards to full capacity through gate device 338, i.e., each memory is loaded with 25
Load the likelihood score of 6 backwards. This manual data is obtained from memory 302 according to a sequential count from address counter 306. Address count C (thus, gate 338 is selectively enabled by an enable signal selectively provided via line 342. The outputs of memory 302 are sequentially controlled by n-decode circuit 344, so that the output of memory 302 is stored in the corresponding memory 312.

Once the first 256 human likelihood scores for each pattern have been loaded into the respective memory 312 (corresponding to 256 reads of the contents of memory 302 in the exemplary embodiment),
Memory 312 is operated in forward recirculation mode, whereby the last input of the shift register (likelihood score corresponding to the 256th frame) is read from memory and passed through the now enabled gates 321 and 333. is the input (via line 315) to the other end of the same shift register memory. Thus, as memory 312 is repeatedly shifted, the likelihood scores for each target pattern for each of the last 256 frames are read out in reverse chronological order and reinserted into the shift register in the same order. 25 through the forward shift line 318.
After a count of six, the shift register returns to its initial data state. However, now the next likelihood score from memory 302 that was loaded during the period of time that register 312 was being shifted is inserted into the register in sequence. This new likelihood score is loaded via the game 33g in response to an inverse 1iJ load pulse on line 319.

The oldest likelihood score in memory 312 is lost.

Shift register 312 will contain the second through 257th likelihood scores for each target pattern. These scores are shifted in the same manner as described above. The shifting and loading process is performed at various frame times so that the likelihood scores are read at the appropriate times for processing as described below.
- With reference to FIG. 10, the output of each group of registers 312 representing a keyword is made available via line 326 to a respective controlled bipolar multiplexer switch 360. The operation of multiplexer 360 is as follows. At the beginning of each frame time, each multiplexer 360 is reset by a reset signal on line 362. In response to a reset signal through line 362, the output twins 364, 366 of multiplexer 360 are connected to fsl power line 1, here lines 326 (a) and 326 (b), respectively.
supplied to At the beginning of the frame time, line 326
The data on (a) represents the human likelihood score during the "current" frame time for the last target pattern of the keyword, and the data on line 326 (b) represents the "current" 7v for the immediately previous target pattern. -Mu p-
Represents the score in the get pattern. The output of multiplexer 360 via lines 364 and 366 is provided to a numerical comparison element, such as an arithmetic element 368. This element 368 provides a "good" input score for line 370, and which of lines 364.366 and 372
to have a good input score.

The good score is added to the contents of adder 374.

(The contents of adder 374 are reset to OK at the beginning of each frame time by a reset signal IC via line 362.) The cumulative likelihood score is then available via line 375 and summed to represent the total cumulative score. Vessel 37
4 is "normalized" by dividing the content of 0 by the number of accumulated likelihood scores N. This division is performed in divider circuit 376. The output on line 378 of divider circuit 376 is Represents the average score/node and is used in determining whether each keyword is a possible detected keyword candidate.

The output of comparator circuit 368 on line 372, along with the minimum and maximum dwell times, is output by multiplexer 360 to the next two input likelihood scores, namely line 326 (b
) and the likelihood scores available through 326(c) (corresponding to immediately before and after (second to last and third to last) the keyword's last target pattern, respectively)
Used to determine whether K should be incremented.

The signal level on line 372 is also used in conjunction with the maximum dwell time signal on line 380 to determine whether to add a penalty to the cumulative word score of 1 adder 374. Therefore, when the maximum dwell time for the target pattern existing at that time has elapsed,
If "good" is the likelihood score on line 364, gate 382 is activated and a penalty count is added to the cumulative score in adder 374.

Program dwell time monitoring element 386, which is essentially a counter, receives minimum and maximum dwell times for various target patterns via lines 388 and 390. When the minimum dwell time exceeds the count in counter 386, a signal level is placed on the maximum dwell time elapsed line 392. When the maximum dwell time for the current target pattern has elapsed, a corresponding signal level \ is placed on line 380 as described above. counter 3
86 is reset when multiplexer 360 is incremented to the next pair of lines by the signal on line 394 (described below). This counter is incremented when recirculating memory 312 is shifted by the cucunto pulse on line 318. Word length counter 896 provides the word length to divider 376 via line 397. counter 396
is reset at the beginning of each frame time by a reset signal K on line 362 (corresponding to the shift register reverse signal on 2-in 319) and recirculating memory 312
is incremented each time is shifted by a pulse on line 318.

According to the invention, the minimum and maximum dwell times are line 3
It controls the increments of multiplexer 360 along with the signal level on 72. For example, if the current target pattern has a "good score" as indicated by the scores on lines 364 and 366, the "next signal" will only be obtained if the maximum dwell time has elapsed. (This is line 318 passing through gate 398 to line 394.
This is done by the upper pulse. ) whereas only the minimum dwell time has elapsed, but the "good signal" is 2 in 3
66, that is, the immediately previous target pattern is "
If so, the pulse on fin 318 passes through gate 400, producing the next signal on line 394:C, and the multiplexer is again incremented to select the next pair of target pattern likelihoods. In all other situations, the multiplexer is not incremented in the illustrated embodiment.

Multiplexer 360 connects the final pair of inputs 326, lines 326 (X-1) and 326 (
If X)K is present, a ``end'' signal is generated on line 324 while the next signal is being received via line y 394. This termination signal has the effect of freezing the output of the divider 376 and communicating that a score for the keyword has been obtained.As mentioned above, the termination signal 1 has the effect of freezing the output of the divider 376 and communicating that a score has been obtained for the keyword.
, the best scores are considered and a decision is made according to the criteria described above. This decision is made once, preferably in real time, before the start of the next frame time, and the whole procedure begins again.

From the foregoing, it will be seen that several objects of the present invention have been achieved and other beneficial results have been obtained.

It will be appreciated that the keyword recognition methods and apparatus described herein may include isolated speech recognition as a special application. Additions, deletions, and other modifications and changes to the described preferred embodiments will be apparent to those skilled in the art and are within the scope of the following claims.

[Brief explanation of the drawing]

FIG. 1 is a flowchart illustrating in general terms the sequence of operations performed in accordance with the invention; FIG. 1A' is a schematic block circuit diagram of a preferred embodiment of the invention; FIG. ! 1 is a schematic block diagram of an electronic device for carrying out specific processing operations in the overall process illustrated in FIG. 1; FIG. 3 is a flow diagram of a digital computer program for carrying out specific procedures in the process of FIG. , FIG. 4 is a diagram of the alignment process of a preferred embodiment of the present invention, FIG.
FIG. 5 is an electrical block diagram of the likelihood function processor of a preferred embodiment of the present invention; FIG. 6 is a diagram of the alignment process according to the dynamic programming method of the present invention; FIG. Subtraction/absolute value circuit! FIG. 7 is an electrical circuit diagram of an overflow detection logic circuit according to a preferred embodiment of the present invention; FIG. 8 is a true value table for the circuit diagram of FIG. 7; FIG. 9 is an electrical circuit diagram of a preferred embodiment of the present invention. FIG. 10 is a block diagram illustrating an example sequential read pattern alignment circuit. FIG. 10 is an electrical circuit diagram of a specific hardware implementation for implementing the duration and comparison control process of the present invention. 13: A/D converter 45; Control processor 46: Preb p setuna 48a Ni-vector processor 48b: Likelihood function processor 49: Sequential decoding processor 51: Clock oscillator 52: M wave number divider 53: Latch 56: Digital multiplier 58 : 32-word circular shift register, register 59: Multiplexer 60: B selection circuit 63: 32-word shift register memory 65: 32-bit adder 67: Gate 71: Computer interrupt circuit 73: Interface No changes to the contents of the drawing) Procedure Masakazu Ura (Method) Kazuo Wakasugi, Commissioner of the Japan Patent Office May 24, 1980 Display of the case Patent Application No. 550 of 1980 Title of the invention Relationship with the voice recognition method and device correction case Patent applicant name: Exxon Corporation Date of notice of agent amendment order: April 26-a-nee, 1982
4. . Ura-masa's subject application, column 11 of the applicant: n, /” ---1 Examination 1.
Power of attorney and its translation (1 copy each) Drawings (1 copy) Contents of amendments to the detailed description of the invention and brief description of the drawings in the specification Engraving of the drawings as attached (no changes to the contents) The description of the specification of the application is as follows: Correct as shown below. t On page 63, line 7, correct the text ``Figure 8'' to ``next''. 2 Add the following table between lines 7 and 8 on page 63. 1 1 1 0 1 1 ・00 1 a 1 1 (overflow) 1
0 0 0 0 1 1 0 0 1 0 1 (overflow) 0
0 1 00 0 0 05 Page 82, line 8, “Figure 9” is corrected to “Figure 8.” 4. On page 83, line 6, "Figure 2" should be corrected to "Figure 8." 5.Page 84, line 14, replace “Figure 10” with “Figure 9”
I am corrected. 6 Page 88, line 11, replace “Figure 10” with “Figure 2”
I am corrected. Z From the last line of page 94 to the first line of page 95, delete the statement ``Figure 8 is a true value table for the circuit diagram of Figure 7''. & On page 95, line 1, Ω that says "Figure 9" is corrected to read "Figure 8." 9 Page 95, line 2 l - Figure 10” should be replaced with “Figure 9”
I am corrected.

Claims (1)

Claims: (1) Each keyword is characterized by a template having at least one target pattern, each target pattern representing at least one short-term power spectrum and each target pattern associated with it. a speech analysis system for recognizing at least one predetermined keyword in an audible signal having a minimum dwell time period and a maximum dwell time period of forming a series of frame patterns, each associated with a frame time, representing, for each frame pattern, a numerical measure of the similarity of each frame pattern to a selected one of the target patterns; and accumulating, for each frame time and each keyword, a numerical word score representing the likelihood that the keyword ended in the frame time, using the numerical measurements, the current frame accumulating, for each keyword, a numerical measure for each successive series of said repeatedly formed frame patterns, starting with a numerical measure of similarity between the pattern and a last target pattern of said keyword; A recognition method comprising an accumulation step and a step of generating at least a preliminary keyword recognition decision whenever a numerical value for a keyword exceeds a predetermined recognition level. (2) the accumulating step includes adding the word score accumulated in each of a series of frame patterns that does not exceed the maximum length time for the then-existing target pattern to the word score of the frame pattern and the then-existing target pattern; summing a numerical measure representing the better of a numerical measure representing the similarity of the frame pattern and a numerical measure representing the similarity of the immediately preceding target pattern; A word score accumulated for each frame pattern occurring over a frame time that exceeds the dwell time is a numerical measure representing the similarity of the frame pattern to the then existing target pattern and the immediately preceding target pattern. summing the better of the numerical measurements representing the similarity of the target patterns; , updating the then-existing target pattern by designating the immediately previous target pattern as a new then-existing target pattern; and updating the dog dwell time for the then-existing target pattern. Whenever you move past the previous target pattern, replace it with the new, then-existing target pattern.
2. The method of claim 1, further comprising the step of indicating as a pattern. (3) maintaining a frame count of the number of pattern frames used in determining the numerical word score for the keyword; and calculating the accumulated numerical word score for the keyword by the pattern used in generating the score; and generating a standardized word score by dividing by the number of frames. 4. The method of claim 3, wherein the second adding step includes adding a penalty value to the accumulated score for the keyword whenever the maximum dwell time of a target pattern element for the keyword is exceeded. Method. (5) Each keyword has at least one target
characterized by a template with patterns, each target pattern representing a small (one short-term power spectrum) and each target pattern having a maximum dwell time period and a maximum dwell time period associated with it. , a speech analysis system for recognizing at least one predetermined keyword in an audible signal, comprising: a series of frame patterns, each associated with a frame time, representing this audible signal from said audible signal at a repeating frame rate; an apparatus for generating, for each frame pattern, a numerical measure of the similarity of each frame pattern with a selected one of said target patterns; Apparatus for accumulating a numerical word score representing the likelihood, for each frame time and for each keyword, using the numerical measure, the numerical similarity between the current frame pattern and the last target of the keyword, the pattern; an accumulating device including a device for accumulating, for each keyword, a numerical measurement value for each of the successive series of said repeatedly formed frame patterns starting with a measurement value; and a recognition level at which the numerical value for the keyword is predetermined. (6) means for generating at least a preliminary keyword recognition decision whenever the accumulating means exceeds a minimum dwell time for the then existing target pattern; The word score accumulated at is added to the better of a numerical measure representing the similarity between the frame pattern and the currently existing Kugetzut pattern and a numerical measure representing the similarity between the frame pattern and the immediately preceding target pattern. a first device for adding a numerical quantity representing the frame pattern and the word score accumulated in each frame pattern occurring at a frame time that exceeds the minimum dwell time of the then present target pattern; a second device for summing the better of a numerical measure representing the similarity of the frame pattern to the immediately preceding target pattern and a numerical measure representing the similarity of the frame pattern to the immediately preceding target pattern; by designating the immediately previous target pattern as the new then-existing target pattern when the numerical measurement for the previous target pattern is better than the numerical measurement for the then-existing target pattern. the trap updates the immediately previous target pattern with the new then-existing target pattern whenever said maximum dwell time for the then-existing target pattern is exceeded;
The device according to claim 5, which is a device for selecting as a pattern. (7) a counter that holds a frame count of the number of pattern frames used in determining said numerical word score for a keyword, and a counter that maintains a frame count of the number of pattern frames used in determining said numerical word score for a keyword; and generating a standardized word score by dividing by the number of pattern frames. (8) The second addition device. 8. The apparatus of claim 7, including means for adding a penalty value to the accumulated score for a keyword whenever a maximum dwell time of a target pattern element for the keyword is exceeded. (9) each keyword is characterized by a template having at least one target pattern, each target pattern representing at least one short-term power spectrum, and each target pattern having at least one required dwell associated therewith; In a speech analysis system for recognizing at least one keyword in an audible signal having a time position and at least one arbitrary dwell time position, a series of frames representative of this audible signal from said audible signal at repeated frame times. forming a pattern; and generating a numerical measure of similarity of each frame pattern for each of the target patterns; and for a second and subsequent required dwell time position of each target pattern. , and for any dwell time position of each target pattern, accumulate the sum of the accumulated score for the previous target pattern dwell time position during the previous frame time and the current value measurement associated with the target pattern. and for the first required dwell time position of the first target pattern of each keyword, the first required dwell time position in the previous frame time.
and the same for the first required dwell time position of each other target pattern. accumulating the sum of the best ending cumulative score for the previous target pattern of the keyword and the current numeric measurement associated with the target pattern; and the possible word ending of the last target pattern of each keyword. The recognition method also includes the step of generating a recognition decision based on the cumulative value of . 10. The method of claim 9, including the step of storing, in association with each dwell time position cumulative score, a word duration count corresponding to the time position length of the keyword associated with the cumulative score at the dwell time position. 11. The method of claim 10, comprising: s) storing, in association with each dwell time position cumulative score, a target pattern duration count corresponding to a positional sequence of dwell time positions in the target pattern. . (b) each keyword is characterized by a template having at least one target pattern, each target pattern representing at least one short-term power spectrum, and each target pattern having at least one required dwell associated therewith; At least +tr=<- and - (in a speech analysis system for recognizing one keyword) in an audible signal, which has a time position and at least one arbitrary dwell time position. an apparatus for forming a series of frame patterns representative of an audible signal; an apparatus for generating a numerical measure of the similarity of each of said frame patterns for each of said target patterns; For the required dwell time position and for each target pattern's optional dwell time position, the accumulated score for the previous target pattern dwell time position during the previous frame time and the current value associated with the target pattern. a first device for accumulating a sum with a measured value;
a second device for accumulating the sum of the score of the zero dwell time position and the current numerical measurement associated with the first target pattern of the keyword, and for the first required dwell time position of each other target pattern; a third device that accumulates the sum of the best ending cumulative score for the previous target pattern of the same keyword and the current numerical measurement associated with the target pattern, and the probability of the last target pattern of each keyword; and a device for generating a recognition decision based on the cumulative value of the end of a certain word. 8) The apparatus of claim 12, comprising a device for storing, in association with each dwell time position cumulative score, a word duration count corresponding to the time position primate of the keyword associated with the cumulative score at the dwell time position. . 14.) a second device for storing, in association with each dwell time position cumulative score, a target no turn duration count corresponding to a positional sequence of dwell time positions in the target pattern. The device described. ) each keyword is modeled by a template with at least one target pattern, each target pattern represents at least one short-term value vector, and each target pattern has at least one associated In a speech analysis system for recognizing at least one keyword in an audible signal, which has a return dwell time position and at least f arbitrary dwell time positions, the incoming audible signal corresponding to the keyword is and making each subinterval correspond to a unique reference pattern. creating a second bus for passing the audible human input signal representing the keyword; determining the interval duration for each sub-interval; repeating said step for a plurality of audible input signals representing the same keyword; and determining criteria associated with each sub-interval. ) generating statistical data describing the duration of each criterion turn; and determining minimum and maximum dwell times for each criterion turn from said collected statistical data. 16. The method of claim 15, wherein the subintervals are initially evenly spaced from the beginning to the end of an audible input keyword.