JPH02298998A - Voice recognition equipment and method thereof - Google Patents

Abstract

PURPOSE: To effectively recognize a speech by allowing a pattern matching unit to analyze a speech signal through the use of both of the output of a neural network unit and a vocabulary identifying output so as to output a signal expressing a word in the speech. CONSTITUTION: A memory 17 includes speech information concerning the vocabulary of recognizable words and the pattern matching unit 16 identifies a boundary between different words and in order to give the first display of the word in the speech, compares stored vocabulary and the speech signal to execute the first analysis of the speech signal. Then this device includes the neural network unit 20 connected with the pattern matching unit 16. This pattern matching unit 16 executes the second analysis of the speech signal utilizing both of the output of the neural network unit 20 and vocabulary identification from first analysis and gives an output signal expressing the word in the speech at least from second analysis.

Description

DETAILED DESCRIPTION OF THE INVENTION (Technical Field) The present invention provides a first analysis of a speech signal for different words.
d) and identify the boundaries between
Words spoken by comparison with words (words 5pok)
en) relates to a speech recognition method of the type which is carried out to give a first representation of the speech recognition method.

BACKGROUND OF THE INVENTION In complex equipment having multiple functions, it is useful to be able to control the equipment by spoken commands. It is also useful where the user's hands are occupied with other tasks, or where the user is disabled and cannot use his or her hands to operate normal mechanical switches and controls.

A problem with speech-controlled devices is that voice recognition is unreliable, especially where the interlocutor's voice is modified by environmental factors such as vibrations. This leads to failure or worse, incorrect operation.

Various techniques are used for speech recognition. One technique involves the use of Markov models, which are useful because boundaries between words in continuous speech can be easily identified.

In noisy environments or where speech is degraded by the nervousness of the interlocutors, Markov model techniques may not provide sufficiently reliable identification of spoken words. recently,
Considerable efforts have been made to improve the performance of such techniques through noise compensation, interpolation, syntax selection and other methods.

An alternative technology that has been proposed for speech recognition is neural networks.
ral nets). These neural network techniques are able to identify individual words with high accuracy even when speech is severely degraded. However, they do not pass for recognition of continuous speech, since they do not recognize word boundaries. This is because it cannot be identified accurately.

DISCLOSURE OF THE INVENTION An object of the present invention is to provide an improved speech recognition device and method.

According to one aspect of the invention, a method of speech recognition of the type defined above is provided, which method uses neural network techniques and word boundaries from a first analysis to give a second representation of spoken words. Performing a second analysis of the speech signal using the identification and providing an output signal representing spoken words from at least the second display.

The first analysis can be performed using a Markov model. Power supply is dynamic time warping template) (dyna
mic time warping template
), and the first analysis can be performed using an asymmetric dynamic time warping algorithm.

Preferably, the first analysis is performed using a plurality of different algorithms, each algorithm determining the confidence that the displayed word is a spoken word.
A signal indicating the word of the power supply memory closest to the speech signal is provided with an indication of .nce), and a comparison is made between the signals provided by the different algorithms. Where the first representation of a spoken word provides a measure of authenticity,
The output signal is provided to respond only to the first indication if the measure of authenticity is greater than a predetermined value.

The second analysis is based on the multilayer berceptron technique (mu) along with the neural network.
lti-1ayer perceptron tech
nique).

The output signal can be used to provide feedback to the interlocutor of the words spoken.

This method uses a noise marking algorithm (
noise marking a1gorithn+
) and performing a syntax restriction on the stored vocabulary according to the syntax of the previously identified words.

The invention also relates to a speech recognition device, comprising a memory containing speech information about the supply of words to be recognized, and a stored vocabulary for identifying boundaries between different words and providing a first representation of spoken words. a pattern matching unit for performing a first analysis of the speech signal comparing speech signals, the apparatus comprising: a pattern matching unit performing a first analysis of the speech signal;
- a neural network unit (20) connected to (16); the pattern matching unit (16) utilizes both the output of the neural network unit (20) and the word boundary identification from the first analysis; The speech recognition device and method according to the invention comprises performing a second analysis of the signal, characterized in that the pattern matching unit (16) provides an output signal representative of the spoken words from at least the second analysis. It will be explained by way of example with reference to the accompanying drawings, in which: FIG.

Embodiment A speech recognition device receives an input speech signal from a microphone 2, indicated generally by the reference symbol 1, and such as that mounted on an oxygen mask on an aircraft pylon+.

An output signal representing the identified word is sent by the device I to the feedback device 3 and the utilization device (utilisat
ion device) 4. The feedback device 3 may be a visual display or an audible device arranged to notify the interlocutor of such words as identified by the device 1. The utilization device 4 may be arranged to control the functions of the aircraft equipment in response to spoken commands recognized by the utilization device from the output signals of the apparatus.

The signal from the microphone 2 is fed to a preamplifier 10 which generates a flat long-average speech spectrum that ensures that all frequency channel outputs occupy a similar dynamic range.
tera+ average 5peech 5pec
tru11) (in which case its characteristics are nominally flat up to 1 kHz). Switch 12 can be set to provide either 3 dB/octave or 6 dB/octave at high frequencies. Preamplifier 10 also includes an antialiasing filter 21 in the form of an 8th order Butterworth low pass filter with a -3 dB cutoff frequency set at 4 kHz.

The output from preamplifier 10 is transmitted via analog-to-digital converter 13 to digital filter bank 14 . The filter bank 14 has 19 channels, implemented as assembly software on a 7MS32010 microprocessor, and
IEE Proc, Volume 127, Part F, Volume 1
Issue, February 1980, J.N. Holmes (J,
rJsR11 channel vocoder (JSRU Channel Vocoder) by N,Ho1a+es)
Based on J. The filter bank 14 has an auditory frequency range of 250-4000) 1z.
It has unequal channel spacing that approximately corresponds to the critical band of erception ). The responses of adjacent channels intersect approximately 3 dB below their peaks. At the center of the channel, the attenuation of neighboring channels is approximately 11 dB.

The signal from filter bank 14 is processed by an integral filter incorporating a noise marking algorithm of the type described by J. S. Briddle et al.
A noise marking unit 15 is supplied.

noise compensating spectral distance measure applied to automatic speech recognition
rum distance measurement) in the Proceedings of the International Conference on Acoustics (Proc, In5t).

Acoust, ), Windmare (Wind*e
re), November 1984. Adaptive noise cancellation technology to reduce periodic noise
oise cancellation technology
e) can be realized, for example, by a unit 15 which can be used to reduce periodic helicopter noise.

The output of the noise marking unit 15 is provided to a pattern matching unit 16 which performs various pattern matching algorithms. The pattern matching unit 16 is connected to the vocabulary memory 17, and the pattern matching unit 16 is connected to the vocabulary memory 17.
is dynamic time warping (DTW: Dyn
(amic Time Warping) template and a Markov model for each word in the vocabulary.

The DTW template uses a single-pass time-aligned averaging technique (
single pass, time-aligned
averaging technique) or embedded training technique (embedded training technique).
can be created using any of the following techniques: This template represents frequency and spectral energy versus time.

A Markov model is derived during training of the device from many utterances of the same word, and spectral and temporal variations are acquired with a statistical model. The Markov model consists of a number of M losing states, each state consisting of a pair of spectral frames and a variance frame. The spectral frame contains 19 values covering the frequency range from 120Hz to 4MHz,
The distributed frame has a state mean period
Contains variance and standard deviation information associated with each spectral vector/feature in the form of duration.

Individual vocalizations during training are in a steady state of speech (Smua
tionary phonet fc 5tates) and their spectral transitions (spectral tran
location) is analyzed to classify it. Model parameters were determined by M.J. Russell (M, J, R
u5sell) and R, Ho
Moore)'s Explicit Modeling of State Occupancy in Hidden Markov Models for Automatic Speech Recognition.
odelling of 5tate occasion
cyin hidden Markov Mod
els for automatic speed
chrecognition) J, Proceedings of the International Conference on Acoustics (Proc'IEBE Int.
Conf, on Acoustics), Speech and SignalP
rocessing), Tampa (Tal19a)
, March 26-29, 1985.
It is evaluated by a regression process using the ``imation algorithm''. Final word model (fina
l word model) is time and intonation (inf
Natural conversational language variability (natu)
ralspoken word variableHy
).

Intermediate between the memory 17 and the pattern matching unit 16 is a syntax unit 18 which performs the usual syntax restrictions on the stored vocabulary with which the speech signals are compared according to the syntax of previously identified words. .

Pattern matching unit 16 is also connected to neural network unit 20. The neural network unit 20 is based on the ``Experiments in 1 solitary digit recognition using a multilayer berceptron'' described by S. M. Peeling and R. H. Moore.
lated digitrecognition us
ing the multi-layer perce
ptron), RSI? E Memorandum No. 4073,
Multilayer persebtron (MLP) like 1987:
Multi-Layer Perceptron).

MLP is characterized by high background noise and low-energy fricative speech (
It has the property of being able to recognize incomplete patterns so as to generate a mask of fricative 5peach). MLP is D. E. Rumelher) (D, E.

``Learning internal representation by error backward propagation'' described by Rumelhart et al.
ernalrepresentation by er
ror back propagation), Cognitive 5science, UC
5D, ICS Report No. 8506, September 1985.

Pattern matching unit 16 uses three different algorithms to select the best match between spoken words and vocabulary words.

One of them is Dee Nis Prydl (J, S.

``Statistical Models and Template Matching: Some Important Relationships Between Two Distinctly Different Techniques of Automatic Speech Recognition'' (Stochast Br1dle)
ic model and template a+a
tching :some important
relations between two
Apparently different technology
uniques for automatic speed
chrecognition), Acoustic Conference Minutes (Pr.
oc, In5t, ofAcoustics), Windmere, November 1984 and Dee Nis Preidl (J, S, Br1d
``Continuous connected word recognition using whole word templates'' by Le) et al.
ord recognition using who
J, Radio and Electronic Engineering
C Engineer), Volume 3, No. 4, April 1983. It is an efficient single pass process that is particularly suited for real-time speech recognition.
ss). This algorithm works efficiently with the noise compensation technique realized by Unit 15.

The second algorithm is the hidden semi-Markov model technique (
03MM: Hidden 5eat Markov
Modeltechnique), where the Markov model contained in the word memory 17 described above is compared with the spoken word signal. The Markov model's additional information about the temporal and intonation variations of spoken words increases recognition performance during pattern matching. In fact, D
T- and HSMM algorithms are integrated with each other. The integrated DTW and 15MM techniques enable identification of boundaries between adjacent words in continuous speech.

The third algorithm uses NLP techniques with neural network 20. The MLP is controlled by the DTW/HSMM algorithm, and the NLP has a variable window viewing the speech buffer (not shown) within the pattern matching unit 16, the size and position of which is determined by the DTW/H5M algorithm.
It is determined by the M algorithm.

In this way, the HSMM algorithm can be used by the MLP to identify word boundaries or endpoints, and spectral-temporal segments or word candidates can be processed by the MLP. Each algorithm uses its description of the speech signal (expl.
anation). A list of several words can be generated by each algorithm with their associated credibility measures. Higher level software within unit 16 compares the independent results achieved by each algorithm and generates an output to feedback device 3 and utilization device 4 based on these results after any weighting.

In this way, the device of the invention allows the use of neural network techniques for the recognition of naturally continuous speech, which was not previously possible. One advantage of the device and method of the invention is that it time and give prompt feedback to the interlocutor. This is particularly important for aircraft applications.

It is appreciated that alternative algorithms can be used; it is only necessary to provide one algorithm capable of identifying word boundaries according to a second algorithm using neural network techniques.

A neural network algorithm need not be used for each word; in some devices it may be arranged so that the Markov algorithm only gives an output as long as its reliability measure is above a certain level; If the conversation is clear or has high background noise, the credibility measure decreases and the device
We consider the neural network algorithm of ``dependenttopinion''.

It will be appreciated that the functions performed by the units described can be performed by programming one or more computers and need not be performed by the discrete units defined above.

The device can be used in many applications, but is particularly suitable for use in high noise environments, such as in the control of machinery and transportation, especially in the control of fixed-wing and rotary-wing aircraft.

[Brief explanation of drawings]

FIG. 1 shows an embodiment of the speech recognition device of the present invention. 1...Speech recognition device 2...Microphone 3...
・Feedback device 4... Utilization device 10... Preamplifier 11・
... Pre-emphasis stage 12 ... Switch 13 ... Analog-to-digital converter 14 ... Digital filter bank 15 ... Noise marking unit 16 ... Pattern matching unit 17 ... Vocabulary memory 18 ...Syntax unit 20...Neural network unit

Claims (1)

Claims: 1. Speech recognition of the type in which a first analysis of the speech signal is carried out so as to identify boundaries between different words and, by comparison with a stored vocabulary, give a first indication of the words spoken. In the method, the method performs a second analysis of the speech signal using neural network techniques and word boundary identification from the first analysis to provide a second representation of the spoken words; and A method comprising the step of providing an output signal representative of spoken words. 2. The method of claim 1, wherein the first analysis is performed using a Markov model. 3. A method according to claim 1 or 2, characterized in that the vocabulary includes dynamic time warping templates. 4. The method of claim 3, wherein the first analysis is performed using an asymmetric dynamic time warping algorithm. 5. A first analysis is performed using a number of different algorithms, each of which generates a signal indicating the word in lexical memory that is closest to the speech signal, along with an indication of confidence that the displayed word is a spoken word. 5. Method according to claim 1, characterized in that a comparison is made between signals provided by different algorithms. 6. characterized in that the first representation of the spoken word provides a measure of reliability, and the output signal is arranged to respond only to the first representation if the measure of reliability is greater than a predetermined value. 6. A method according to any one of claims 1 to 5. 7. A method according to any one of claims 1 to 6, characterized in that the second analysis is performed using a multilayer perceptron technique in conjunction with a neural network. 8. A method according to any one of claims 1 to 7, characterized in that the output signal is used to give feedback to the interlocutor of the words spoken. 9. The method includes the step of performing a noise marking algorithm on the speech signal.
8. The method according to any one of 8. 10. A method according to any one of claims 1 to 9, characterized in that the method comprises the step of performing a syntax restriction on the stored vocabulary according to the syntax of previously identified words. 11. A speech recognition device, comprising a memory containing speech information about a vocabulary of words that can be recognized, and a stored vocabulary and a speech signal for identifying boundaries between different words and providing a first representation of the spoken words. The first of the speech signals to compare
In those including a pattern matching unit for performing analysis, the apparatus includes: a neural network unit (20) connected to a pattern matching unit (16), the pattern matching unit (16) performing a second analysis of the speech signal utilizing both the output and the word boundary identification from the first analysis, and the pattern matching unit (16) providing an output signal representative of spoken words from at least the second analysis; A voice recognition device featuring: