FR2627887A1 - SPEECH RECOGNITION SYSTEM AND METHOD OF FORMING MODELS THAT CAN BE USED IN THIS SYSTEM - Google Patents

Abstract

The present invention relates to apparatus and methods that can be used in connection with speech recognition systems of the type employing models to compare the model with incoming speech. When models have the same signal-to-noise ratio as an unknown speech signal, the recognition efficiency is improved. To carry out the invention, the signal-to-noise ratio of an incoming signal is predicted and the recognition efficiency is therefore optimized by modifying the models before they are used, so that they are as if they were generated. from a speech having the same signal-to-noise ratio as the unknown impending speech. There is provided an apparatus for predicting the signal-to-noise ratio of the impending speech as well as an apparatus which modifies the noise free or low noise models in accordance with said predicted values.

Description

26Z7887

  - The present invention relates, in general, to

  speech recognition and, in particular, a recognition system

  speech using models, in which each of these mo-

  This is produced by the selective addition of noise to increase the probability of speech recognition. It can be seen that the field of speech recognition in general has been widely explored in recent years and that speech recognition systems have been used in many forms. The principle of speech recognition implies that the information obtained in a spoken sound can be directly used to operate a computer or other equipment. For the most part, the prior art considered that a key element for recognizing the information contained in a spoken sound was the distribution of energy with respect to frequency. The frequencies of formants are those where the energy peaks are particularly

  important. The frequencies of formants are the acoustic resonances

  of the oral cavity and are controlled by the tongue, the male

  choir and lips. For a human listener, the determination of

  first two or three formant frequencies is usually enough

  to recognize the sounds of vowels. In this way, known recognition machines included certain means for determining the amplitude or power spectrum of the incoming speech signal. The first stage of speech recognition is called

  pretreatment because it transforms a speech signal into characters

  recognizable parameters that reduce

  bring the data flow to proportions by allowing processing

  is lying. One way to do this, therefore, is to

  monitor the frequency of zero crossing of the signal in several banks

  wide frequencies, to give an estimate of the frequencies

formants in these bands.

  Another way is to represent the speech signal in terms of filter parameters whose spectrum best matches -2- that of the input speech signal. This technique is known as

  the name of predictive linear coding '(CLP). Linear coding predic-

  tif or CLP has grown in popularity due to its efficiency,

  precision and simplicity. An average of the characteristics is established.

  recognition ticks extracted from speech on 10 to 40 millise-

  condes, then they are sampled between 50 and 100 times per second.

  The parameters used to represent speech for the purpose of recognition can be directly or indirectly a function of the amplitude or power spectrum. The formant frequencies and the linear prediction filter coefficients are examples of parameters which are indirectly a function of the spectrum.

  of speech. Other examples are the spectral parameters and the pa-

  reameters of the logarithmic area ratio. In these cases and in the

  In most other cases, the speech parameters used for the re-

  knowledge are or can be derived from spectral parameters. The

  The present invention relates to the selective addition of noise to the parameters.

  very spectral producing parameters of speech recognition.

  The present invention applies to all forms of recognition.

  this speech using speech parameters that are or can

  be derived from spectral parameters.

  In any event, one of the most common approaches

  Speech recognition in the past has been the use of models to achieve adaptation. In this approach,

  words are typically presented in the form of parameter sequences

  very. Recognition is carried out using a measurement of simi-

  the predefined scale for comparing the unknown model element with stored models. In many cases, time alignment algorithms are used to account for variation in the

  frequency of word production. Systems for adapting to

  therefore allow high efficiency to be achieved with a cor-

  reduced from acoustically distinct words. Some researchers have looked into the ability of such systems to perform, over time,

  a fine phonetic distinction among a wide range of speakers.

  See, for example, the article entitled "Performing Fine Phonetic Distinctions: Templates versus Features in Variability and

- 3 -

  Invariance in Speech Processes "by J.S. Perkel and D.H. Klatt, edi-

  Hillsdale, New Jersey: Lawrence Erlbaum-Associates, 1985, au-

  authors R.A. Cole, R.M. Stern and M.J. Lasry.

  As an alternative, many offer an.

  close to speech recognition based on characteristics,

  according to which it is first necessary to identify a group of characteristics

  acoustic ques containing phonetically relevant information contained in the speech signal. With these elements, algorithms can be created to 'extract the characteristics of the signal from

  speech. A workbook is then used to combine the characteristics

  ques and make a recognition decision. Some believe

  that a feature-based system is more capable of performing

  kill fine phonetic distinctions that a system of adaptation to models and that it is therefore intrinsically superior to it. In all cases, adaptation to models is a technique-often used for pattern recognition, in which case an unknown

  is compared to prototypes to determine which one it is

seems the most.

  According to this definition, speech recognitions based

  on the characteristics and using Gaussian models with more

  different variables to perform a classification also realize

  an adaptation to models. In this case, the statistical workbook

  that simply uses a vector cd or mistqu sIose cnfkiguratirn. Similarly, if we take the spectral amplitude and the CLP coefficients as characteristics, the techniques based on the spectrum are

  also based on specifications.

  Regarding use, adaptation systems

  to models and systems based on the characteristic features

  actually kill different points on the same continuum. One of the most complex problems posed by the adaptation to

  of the models resides in the difficulty of defining measures of

  sufficiently sensitive to make phonetic distinctions

  only fine, but sufficiently insensitive to spectral variations

not relevant.

  This problem manifests itself in particular by an excessive importance-

4 - 2627887

  sive tuned to negligible variations between frames in the spectrum of a long and stable vowel. Therefore, the systems

  known, aware of this type of problem, have proposed-several methods

  distance triques which are supposed to be sensitive to phonetic distances and which are insensitive to irrelevant acoustic deviations. See, for example, the article titled "Prediction

  of Perceived Phonetic Distance from Critical Band Spectra "by D.H.

  Klatt, published in ICASSP-82 Procedures, IEEE Catalog no CH1746-7,

pages 1278 to 1281, 1982.

  In any case, to better understand the

  speech transmission, see Proceedings of the IEEE, no-

  vember 1985, volume 73, no 11, pages 1537 to 1696. This publication

  of the IEEE presents various articles devoted to transmission systems

  man-machine speech and gives a good overview of the problems

  individuals they pose. It will be understood that one of the main as-

  pects of any speech recognition system is its ability to

  accomplish the task entrusted to it, namely to recognize a part

  role, whatever the environment.

  As noted, many pa-

  role use models. For the most part, these converted systems

  feels statements in sequences of parameters that are stored in the computer. Sound waves pass from the mouth of a speaker,

  via a microphone, to an analog converter

  digital where they are filtered and digitized with, for example, any background noise. The digitized signal is then

  filtered again, converted to recognition parameters and compared

  in this form, to stored speech models to determine

  the most likely choice for the word. For further information

  examples of these techniques, see IEEE Spectrum, vol. 24.

  no 4, published in April 1977. See the article entitled "Putting Speech

  Recognizers to Work ", pages 55 to 57 by T. Wallich.

  As we can see from reading this article, the uses of speech recognition systems are becoming more and more diverse and there are many models already available.

  which are used for various applications indicated in this article

- 2627887

  key. The training of models is also very well mastered in the known technique. These models are used with many different types of speech recognition systems. One of these types of systems is known as the "keyword recognition system" described in the publication "An Efficient Elastic- Template Method for Determining Given Words in Running Speech" by JS Bridle, "British Accoustical Society Spring Meeting ", pages 1 to

  4, April 1973. In this article, the author talks about the creation of mo-

  elastic models from a parametric representation of examples

  ples spoken of keywords to detect. Parametric representation

  similar incoming speech is continuously compared with these mo-

  flags to measure the similarity between speech and keywords

  from which the models are derived.

  The recognition system determines that a word has been pro-

  spoken when an incoming speech segment is sufficiently similar to the corresponding pattern. Word patterns are considered "elastic" because they can be extended and compressed over time

  to account for variations in speech rate and values

  local riations in the flow of word pronunciation.

  Recognizing keywords is similar to recognizing

  classical speaking session. In the first, models are only mixed

  moralized only for "key" words which must be recognized among arbitrary words or sounds, while in the second, models are memorized for all the words which one thinks that they will be

  pronounced. All of these systems, whether it's recognition systems,

  sance of keywords or .class speech recognition systems

  siques using models, face the same problems, with-

  see their inability to recognize the word spoken when it is spoken for example by different individuals or by the same individual

under different conditions.

  The present invention therefore aims to provide an apparatus

  and methods for making an automatic recognition system

this perfected speech.

  The present invention also aims to provide a speech recognition system automatically adapting to a

- 6 - 2627887

noisy atmosphere.

  - * As we will see later on reading the description -

  In the annex, most speech recognition systems work less well in the presence of noise. This deterioration is particularly marked when the models were derived from words with little or no noise, or with noise from a

  quality different from that existing when trying to recon-

  birth. The prior methods intended to remedy this difficulty

  cult involve creating new patterns in the presence of new noise. This creation requires to collect a new word and a new noise. In this particular system, we add

  noise analysis to the models, which increases the proba-

  recognition and thus significantly improve the return

  of the system, without having to collect new pa-

roles to produce models.

  In a speech recognition system of the compression type

  providing a spectrum analyzer to provide values to an output

  spectral amplitude of the statements and to compare memo models

  laughed at spectral values processed to provide an output in the event of a favorable comparison indicating the presence of speech in these statements, the improvement consists of an apparatus for supplying the stored models, comprising a first means connected to the spectrum analyzer for supplying a signal representative of the noise signal expected from an incoming signal, and a means connected to the first means and responding to the predicted noise signal for in Er des Morles which are

  modified according to the predicted noise signal.

  The different objects and characteristics of the invention are

  will now be detailed in the description which follows, made to

  by way of nonlimiting example, referring to the drawings anne-

  which: - Figure 1A is a block diagram representing a system

  speech recognition using speech recognition parameters

  this derivatives of a spectrum in accordance with the present invention, FIG. 1B is a block diagram repeating an avariant of a

  speech recognition system using speech parameters

-7 -2627887

  knowledge of spectral character in accordance with this invention

  t-ion, FIG. 2, a detailed block diagram representing a technique in accordance with the present invention for forming operational model data,

  - Figure 3, a table giving the definitions of the different

  output annuities indicated in FIG. 2 ,, - FIG. 4, a detailed block diagram of another embodiment of the present invention,

  - Figures 5A to 5C, diagrams illustrating the function

  operation of a speech and noise detector in accordance with the present

the invention,

  - Figure 6, a table giving the definitions of the para-

  technical meters in accordance with Figures 5A to 5C.

  - As we will see, the present invention applies to all

  recognition systems using parameters which are.

  spectral character or which are derived from those which are of spectral character. In the latter case, it may be necessary to memorize the models in two forms: spectral for analytical addition

  noise and operational models.

  FIG. 1A represents a block diagram of a system of

  speech recognition according to the present invention, which uses

  recognition parameters derived from the spectrum.

  A microphone 10, in which a speaker using the

  system, is shown. Conventionally, the microphone 10 con-

  turns sound waves into electrical signals which are amplified at

  by means of an amplifier 11. The output of amplifier 11 is con-

  Connected to a spectrum analyzer 12. The spectrum analyzer can be of the broadband or narrowband type and have a rapid analysis capacity. The operation and structure of the spectrum analyzer are well known and can be achieved by several techniques. The spectrum analyzer intervenes to divide the speech into

  short frames and provides a parametric representation upon its output

  that of each frame. The type of acoustic analysis performed by

- 8- 2627887

  the spectrum analyzer is not essential for the present invention and many known acoustic analyzers or spectrum analyzers can be used. Examples are found in requests for

  US Patent No. 439,018, filed November 3, 1982 for G. Vens-

  ko et al., 'and n 473.422 filed March 9, 1983 for G. Vensko et al.

  These two requests are jointly assigned to ITT Corporation, considered

  dere as the assignee) and integrated here by reference.

  Reference will also be made to American patent application No. 655.958, filed on September 28, 1984 for A.L. Higgins et al. and I!

  titled KEYWORD RECOGNITION SYSTEM AND METHOD USING TEMPLATE-CONCA-

TENATION MODEIú

  The spectrum analyzer 12 can include a 14-channel bandwidth filter arrangement and uses a frame size of 20 milliseconds or more. These spectral parameters are treated as indicated in FIG. 1A. As indicated, the output of the spectrum analyzer is connected to a switch 13 which can be set to put

  Recognize, Train a Model or Modify a Model.

  When the switch 13 is placed in form a Modem-

  the, the output of the spectrum analyzer 12 is connected to a module

  14 called the spectral form of the models. The purpose of module 14 is to help

  der to the training of models from the output of the analyzer

  spectrum. The models formed in module 14 are a spec-

  of models and many techniques for forming these mo-

  these are well known. Essentially, in Train a Model mode, the output of the spectrum analyzer 12 is processed by the module 14 which provides models according to the statements made by the speaker through the microphone 10. The speaker is invited to speak words to be recognized, and representative models of the words spoken are shown. These models are used by module 15 to derive

  recognition parameters from spec- formed models

  for final models, as indicated by module 16 in the presence of little or no noise. As indicated by the

  module 16, the models corresponding to zero noise are then sto-

  ckés and are representative of particular statements, such as

  words, expressions, etc., pronounced by a particular speaker.

AT 9- 2627887

  The memorized models are coupled, via a switch 100, to a processor 160 which performs an algorithm of

  recognition. As can be seen, the processor 160 operates

  So in recognition mode to compare the unknown speech with models which are memorized in the module 16 and which have been pro-

  duits for zero noise conditions. Therefore, as my-

  In FIG. 1A, in Model Formation mode, a spectral form of model form is produced to obtain model parameters which are then used to form models for zero or low noise conditions. As explained later, the processor used can operate with the models stored in module 16

  for zero or low noise conditions. The operation of the pro-

  stopper 160 is also well known and is intended for the essen-

  tiel, to make an adaptation according to different measures of

  distance or other algorithms. When this adaptation is made

  an indication appears to indicate that there is a correct word

  and that the word or sound therefore constitutes the output of the system.

  As can be seen, the switch 13, when in Recognize mode, makes it possible to connect the output of the spectrum analyzer 12 to a parameter derivation module 161 which, for

  essentially, derives from the parameters of the spectrum analyzer, these pa-

  rameters being compared to the stored models, as described for example above and stored in the module 16. As shown in FIG. 1A, the switch 13 can also be placed in position

  central. In this position, also called Modify Mo-

  the output of the spectrum analyzer is passed to a noise statistics estimation module 162. As can be seen, the function of the module 162 is essentially to provide an analysis of

  noise or process the noise to provide an estimate of the statistics

  noise ticks. This is an essential aspect of this in-

  vention that selectively adds noise to form patterns

  in order to achieve speech recognition and improve this

  knowledge in the presence of additional noise.

  Therefore, the function of the statistic estimation module

  noise level 162, which will be described later, aims to 10o-2627887

  modify the spectral models formed in module 164 which is

  nected to module 14 and which receives the information. The output of module 164 derives from recognition parameters in module 165

  and these parameters are used to form models, as indi-

  the module 166, which can be used with noise or at low noise levels. As can be seen, the system illustrated in FIG. 1A therefore makes it possible to carry out reconnaissance with

  models that can be used with noise or with models for

  before being used with very little or no noise through

diary of switch 100.

  As briefly mentioned above, in Recognize mode, the

  spectrum parameter output from spectrum analyzer 12 is approx

  yée at the input of processor 160 via the module de-

  parameter rivation 161; The processor typically performs an

  gorithm which is also not essential for the invention. The processor 160 determines the sequence of the stored models and makes the best adaptation to the incoming speech to be recognized. Therefore, the processor output is essentially a chain of model labels in which each label represents a single

  model in the most suitable model sequence.

  For example, each model can receive a number and a la-

  beautiful. It can be a multi-bit representation of this number.

  ro. This output is sent to a model search system.

  included in processor 160, which, in the presence of a representation at

  several bits, can be for example a comparator with a device

  tif memorization of model labels. Therefore, the process

  seur 160 intervenes to compare each incoming model label with the stored models. The subsystem 160 can then signal that a word

  or a particular phrase was spoken and say which word.

or what expression it is.

  Whether in Train Model or Modify Model mode,

  the user pronounces various words and recognition parameters

  these are derived from the spectrum output of the spectrum analyzer 12.

  In Modify Model mode, the system intervenes to produce different

  various models usable in relation to the system in Re-

- 11 - 2627887

* know, these models being, as indicated above, modified by the selective addition of noise via the noise statistics estimation generator 162. The selective addition of noise by means of the generator 162 ensures more efficient operation reliable system, as will be seen later.

  FIG. 1B represents a recognition system used

  s spectral character recognition parameters. In all cases, the same functional elements have been designated by the same reference numerals in Figure lB. As we can

  note, a microphone 10 is connected to the input of an amplifier

  11 whose output is connected to the input of the

  spectrum 12. The output of spectrum analyzer 12 is again

  connected to a switch 13 which can be placed in train mode

  dele, Modify a Model or Recognize.

  As can be seen in FIG. 1B, in the Form a Model mode, models are formed for low noise conditions or

  null through the module 170. The module 170 forms models

  directly providing them with recognition parameters

  spectral reaction. The shape models are then memorized and also

  linked to module 171 which modifies the spectral models, such as

  for example those derived from module 170 under the influence of a general-

  noise statistics estimator 172 which essentially functions as the noise generator 162.

  modified spectral models 171 is connected to module 173 which memorizes

  make the models usable in noisy surroundings. A processor 177 is also shown and can operate either with the models stored in the module 170, or with the models stored in the

module 173.

  In any case, before going any further, it is known to

  Manage models according to known techniques. There are different ways of ar- = z models. The processes for accomplishing the task of generating models are automatic and

  normally use a two or more step procedure.

  In one of these possibilities, speech data from

  the test statement (Model mode) are divided into segments. These seg-

-12-

- Â2 - 2627887

  then entered for statistical grouping analysis

  allowing to select a subset of segments which

  put a mathematical function based on a distance measurement between segments. The segments belonging to the selected subset are used as models.

  These techniques are described in the patent application below.

  above no 655.958. In all cases, the different techniques for measuring distances are well known, as indicated by certain

  references appearing in the preamble of this description.

  One of the most popular techniques for measuring distances

  is known as Mahalanobis distance calculation.

  For examples of this particular technique, see a patent application entitled IMULTIPLE PARAMETER SPEAKER RECOGNITION SYSTEM AND METHODS, filed January 16, 1987, No. 003.971 by E. Wrench et al. This application gives many other examples of techniques used in speaker recognition systems and describes in detail some of the algorithms used in connection with these systems. In any case, and always with reference to the

  Figure 1, we see that the main aspect of the present invention

  tion relates to speech recognition systems, such as-

  glossy in Figure 1, which use models to compare with incoming lyrics to make a decision

  on the word spoken. This technique can be used in systems

  keyword recognition systems, speech recognition systems, speaker recognition systems, systems

  speaker verification, language recognition systems

  pledge, or in any system using models or a combination of different models to make a decision about an emitted sound. Before undertaking an explanation of the structure and techniques employed in the present invention, certain aspects and

  considerations of the present invention will be addressed.

  The inventor has found that when models have the same

  signal-to-noise ratio that unknown or spoken speech makes it

  recognition is better than with models presenting

-._ 3- 2'627887

  less or more noise. Therefore, if we assume that the report

  signal / noise port of the audio signal can be predicted, the recognition performance can then be optimized by modifying the models before using them, so that they are "as if" they were generated from speech with the same ratio if-

  general / noise as expected unknown speech.

  Therefore, to achieve the present invention, the

  The following considerations apply. The first is to pre-

  say the signal-to-noise ratio of the expected speech and the second to

  modify the models to meet the "as if" requirement. The pre-

  diction is based on both theoretical and em-

  piric. In most applications, one can reasonably

  expect the user to speak at a relatively high level

  either in absolute data in the case of a low level or constant noise, or at a relatively constant level greater than this noise. Speech and noise level can then be used to predict the signal-to-noise ratio of the unknown speech. As explained below, this is accompanied by the use of a noise and speech level detection module. In some cases it is assumed that the speech level and the noise level in each filter channel change slowly enough for the current values

  are useful estimates of impending values.

  By modifying the models that are free of noise or more or less

  te that they are "as if" they had been obtained from a gen-

  noise generator speech is based on empirical considerations

and theoretical.

  Research has shown that excellent app

  Proximity by admitting that the noise and speech powers are added in each filter bank channel. A closer look reveals that the combination of speech and noise has a non-central chi-squared distribution, lerltedeésdeliberté

  being a function of the bandwidth of the wire bank channels-

  very. From this and other considerations it is possible to make more precise estimates of the expected value of combining known speech power with noise having

  known statistical properties. The better accuracy of "ad-

  noise edition "thus obtained effectively increases the accuracy of the models produced, but does not translate into an improvement in

  recognition accuracy higher than that obtained with the

4- 2627887

  gle of "added powers". We will therefore continue to refer below

  following the rule of added powers, even if it is possible to increase the theoretical precision by adopting another method to estimate the expected value of the combination between the powers of speech and noise. This replacement does not change the purposes or the

  practical realization of the present invention.

  It has also been found that the internal electronic noise and the quantization noise combine with the acoustic noise and signal in accordance with the "added powers" rule. They

  may be less than the observed acoustic noise, but this re-

  mark is valid. We can therefore use the result of "powers

  aJoutées "to design different models, so that the applica-

  tion of research results in a continuous effort to use numbers derived from valid models. This will be the subject

further explanations.

  We have shown that models resulting from a power of

  noise equal to its average value give full satisfaction for pro-

  duire reliable recognition outputs. It is therefore not necessary

  be able to predict the variability between frames of the noise power and just use the mean value. Template settings

  that we are looking for are those that would be produced from the same

  speech power than that actually present in the model

  the Basic Shape, combined with the average noise power of the mo-

  is lying. The noise power values of the channels from the system are estimates of the noise power and can be

  used to relate to the noise power

  which can be determined mathematically. To understand by-

  this procedure and the Justifications relating thereto

  tent, the following considerations apply.

  It is first indicated that the probability law of the output

  tie of a single discrete Fourier transformation - (TFD) of a speech signal tainted by an additional zero mean Gaussian noise can be easily calculated. The next factor to consider, which is important in expanding the speech and noise combination model to make it applicable to each channel in a bandpass filter bank, indicates that the channels have or may have a

  much greater bandwidth than a single TFD channel. The para-

  noise power meter and the number of contributing channels can

  therefore be estimated by observing the output of the bandpass filter

  in the absence of speech and in the presence of noise.

  The next step was to make sure that the mod-

  speech recognition created in the absence of noise can

  be improved to be used in the presence of noise, by modifying them

  laughing so that they are equal to their expected value in the presence of noise. The process to be used is therefore that corresponding to each

  speech sample and each bandpass filter channel shown

  felt in each noise-free model, then substituting the expected value of the noise-free model modified by the presence of the

noise of the moment.

  Therefore, by measuring the mean and variance at the output of the bandpass filter channel, we can estimate the properties

  from the channel to the way it transmits Gaussian noise. For the-

  sentiel, as can be seen above (many considera-

  above have been proven mathematically), there is a ba-

  is both theoretical and empirical for the implementation of the

  present invention. As noted, the principle of this invention

  tion lies in the analytical addition of noise to create models

  those who, once trained, intervene to increase reliability

  speech recognition systems.

  In any case, there are two ways to add noise to model data collected in a noise-free environment

  and therefore to form new models that can be used in noisy environments

  you. One rigorous way is to add noise to each element.

  then model the results. An approximate way is to average the elements free of

  noise to form Basic Shape data and to modify these data

  born by adding noise according to current conditions,

  using the "added powers" rule or any other rule

  propriée or more precise. The rigorous way requires to keep

  all models and items and requires excessive storage.

  The approximate way provides essentially the same models

  and the same recognition results. There is a condition es-

  which is implicit in the implementation. The model data

  must be free from noise in relation to the environment in

which they are used.

  Figure 2 shows a detailed block diagram of training

  tion of models used by adding noise to a Basic Shape model. The Basic Shape model is itself an average obtained from a group of "elements" of words. Each element is made up

  parameters from a single word of the given word. One or more

  several elements can be arranged to form a Basic Shape model. The Basic Shape models are formed under silent conditions and stored in a module 16 of FIG. 1A or in a

  module 170 of Figure lB. Note that Figure 3 is a ta-

  beau defining each value appearing in figure 2. As one can

  see, the microphone 10 in which a speaker speaks is also

  shown in Figure 2. The microphone output is connected

  tee at the input of amplifier 11 whose output is connected to

  the spectrum analyzer designated by BPF or bank of pass filters

  band 12. Switch 13 is in the Modify position.

  Model. The output of the bandpass filter bank 12 is the vector

  of the spectral intensity values of the bandpass filters and is applied to a module 20 which is used to average the pairs of

frames.

  The averaging of the pairs of frames is a technique

  well-known and is mainly carried out by numerous

  well known. The output of the module 20 is the result of calculating the average of successive pairs of the input from the spectrum analyzer 12 and a module 20 is used to halve the effective frame frequency. The output of module 20 is applied to a scale bit module 21- and to a squared component module 22. The squared component module 22 provides a vector output which is equal

  to the squared intensity which is essentially the value of

  of the output of the frame pair average module 20.

  -17- - 17 - 262t887 The output of the scale bit module 21 is essentially used to provide twice the average of the successive pairs implemented by a series of offsets to allow one of them to adapt to the maximum vector components in a 7-bit scale. Consequently, the module 21 is a shift register which essentially provides a certain number of right offsets to carry out the operation described. The output of the scale bit module 21 is transmitted to a logarithmic converter 23-which produces, at its

  output, a vector of logarithmic spectral parameters at scale-

  the. The average of this vector of parameters is then calculated on a given group of model elements by the module 24 to provide

  the output the logarithmic spectral parameter at scale, which provides

  essentially denies a parameter of the Basic Shape model. The exit

  tie of the components module squared 22 is sent to an input of

  module 25 called relativized energy module and with a mo-

  dule 26 called noise and speech level detector.

  The output of the relativized energy module 25 is a parameter

  be indicative of the relative energy, for example determined in low-

  the average energy from the output of the control module

  posing squared 22. This average is calculated on elements of

  model by module 36 to provide an indicative average of the vector

  output tor which is the required relative energy parameter

  to provide another Base Shape data value. As ex-

  folded below, the output of the noise level and pa-

  role 26 is indicative of the energy level, the average of which is again

  calf calculated by module 27 to provide at its output the energy level of another type of Basic Shape. The level detector

  noise and speech which will be described later, provides two outputs

  additional parts, one of which is a logarithmic indicator of the speech level, the average of which is calculated over a word time and on a channel which is a scaling device linked to the words, and the other of which is the vector of the noise level in each channel, the average of which is calculated with respect to time, but not with respect to the channel. It is also a vector connected to

  word recognition units. We therefore see that the output of the modu-

  the 27 is applied to a first adder module 30 which receives

  an additional output from the noise and level detector

  the. The output of adder 30 is applied to one of the inputs

  an adder 31 which receives, - at its other input, a derivative output

  of the scale bit module 21. The output of the scale bit module 21 is multiplied, via the module 32, by a

  factor K which is equal to 18.172 and which is defined in detail in ta-

  bleau 3. The average of this value is then calculated by the modu-

  the 33 to produce, at its exit, the value of Basic Form of the

  logarithmic value which is applied to the other input of the addi-

  actuator 31. The output of the adder 31 is applied to the addi-

  actuator 32. The adder 32 receives, as another input, the output of the noise and speech level detector 26 which is again the

  vector of the noise levels of each channel. This output is applied

  quée at one of the inputs of a function module 40 which receives, at its other input, the output of the module 23. The output of the function module is the vector of spectral logarithmic parameters at scale for a model with addition noise. It is applied to a function module 41 to provide, at its output, the recognition parameter vector which is the mel-cosine transformation matrix for the particular statement. Consequently, the output of module 41 and the

  output from module 26 are used to provide model data

operational.

  As indicated, all of the outputs listed on the block

  gram of Figure 2 are described in Figure 3. As can be seen from the above, the actual value of intensity

  of the Base Shape model derived from Figure 2 is essential

  tially given by the following equation: MB X 2SB expb (UB) and the effective power is given by the following equation B m2B 22SB expb (21B)

-19 - 2627887

See definitions in Figure 3.

  In any case, as we can see, before adding

  noise, the power of each frame is changed so that the ni-

  mean speech calf of the model indicated at the output of module 27 of FIG. 2 is the same as the speech level of the moment indicated by

  the output of the noise and speech level detector 6 which is ap-

  plicated at the input of the adder 30. Since the values

  are in recognition units (0.331 dB), the power e-

  fective in the Basic Form is modified, which is indicated at the output of module 26. To this value is added the power level

  noise of the moment and we thus obtain the power level effected

  tif of the model with addition of noise ,. so that the intensity effected

  tive of the model with noise addition is represented as the output

of module 41.

  Therefore, all recognition parameters operate

  are in the mel-cosine transformation of the spec-

  logarithmic paths and are measures of relative energy. As noted, all of the above should be apparent to those skilled in the art after examining Figure 2 and the definitions in Figure 3 and the

  Corresponding mathematical formulas should also be avoided.

  teeth. Using exactly the same techniques, we can therefore form models by adding noise to model elements and then

  by calculating the average. For the most part, the realization process

  tion is similar to that shown in Figure 2, so you can provide the exact same outputs as shown in Figure 2, except that averaging can be done after

the function unit 40.

  Figure 4 shows a more detailed block diagram of a

  typical system using a method of forming analog models

  gue to that indicated above. In FIG. 4, the same reference numbers have been used to designate similar operational components. As can be seen in Figure 4, there is an AGC or automatic gain control module 45 which is connected to one of the inputs of an adder 46 whose output is connected to a -20 -

- 20 - 2627887

  coder / decoder module (CODEC) and a linearization circuit 47.

  For the most part, the encoder / decoder module can be a convert-

  analog-digital generator followed by a digital-analog converter

  than. The output of the codec is applied to the synthesizer or bank of bandpass filters 12. The output of the bandpass filter 12 is sent to a module

  of frame pair average 20 which is also associated with a mo-

  scale module 21 and the speech and noise detection module 26 which will be explained. The output lines shown on the right side of Figure 4 provide the different data values

  operational models that are used to form mo-

noise in the presence of noise.

  As can be seen, the speech and

  noise 26 which will be described below constitutes a functional module

  essential. Figure 4 also shows that the inputs of the mid-

  crophone 10 are designated Nc and Sc which are the sources of significant signals and noise. The index "c" indicates that it is

  the average spectral intensity over the bandwidth of each of the

  filter bank standards forming the spectrum analyzer 12. In any case, each index "c" has 14 varers, one for each filter in the filter bank. Therefore, Sc is the spectral intensity of the channel

  C of the acoustic speech signal, while Nc is the spec-

  quadratic average of the acoustic noise of this channel. The outputs of the adders 50 and 46 are the spectral intensities of the

  electronic noise which is injected before and after the con-

  automatic gain control AGC 45. The output of codec 47 contains

  the spectral intensity of the quantization noise introduced by the co-

  In any case, the output of the bandpass filter bank 12

  is the vector of spectral intensity values of pass-through filters

  band, while the output of the average module of weft pairs is the result of the average of successive pairs of

spectral intensity values.

  The effective output signal from the filter bank 12 is an estimate of the spectral intensity of the signal at the input of the filter bank on the passband of the filter bank and

-21- 2627887

  this is indicated for each channel of the filter bank. The way

  no of successive pairs of these values is calculated to produce the

  output from module 20 at a rate of 50 per second.

  Essentially, the group of all values of the 14 ca-

  naux is fully shifted to the right of the same number S in module 21, so that the largest value occupies 7 bits or less and the resulting values are converted by a table lookup to a number proportional to the logarithm. The table indicates

  127 for an entry of 127, so the result can be viewed

  derived as 26.2 times the natural logarithm of the input or, equivalently, the base logarithm b, where b is. equal to 1.03888. Frame values of 20 milliseconds are also used by the

  detector 26 for producing a measurement of the pointer speech energy

  te and an estimate of the average noise energy of each channel.

  The speech level is an estimate of the base logarithm b of the total speech energy at the microphone 10, at which

* adds an arbitrary constant.

  Essentially, the effect of automatic gain control is suppressed and is therefore not a spectral value. For example, it is a function of the total energy of the bandwidth of the whole

  from the filter bank. The speech level estimate is also

  function of words or expressions. Its time constants

  are such that they are a measure of the level at which brief statements

  these are pronounced. There is therefore only one level value which can be associated with each unknown model or segment of duration of

  model. The time constraints of noise detection estimates

  26 are also such that only one noise level estimate should be assigned to each channel over the time periods of

  the length of the statements. Therefore, the output values of the de-

  speech and noise detector 26 connected to the logarithmic circuit 54 of FIG. 4 are estimates of average energy of the output of

  the filter bank. They are therefore affected by self-monitoring

  gain and are directly proportional to the energy

  mean spectral without logarithmic transformation.

  We assume that the signal and the different noise sources

- 22 - 2627887

  are statistically independent and their energies are added on average. This method is not only convenient for determining internal noise sources, but it has proven to be an excellent

  approximation of acoustic noise and signal sources. In or-

  tre, we assume that there are noise values that can be

  considered as equivalent noise power at mid-level

  crophone. These values are the acoustic noise power and

  very high noise levels of the system, some of which are reduced by

  the automatic gain control module 45.

  Therefore, the scale factors derived from the figure

  4 and shown in Figures 2 and 3 are provided to produce models

  them depending on the noise. Using the method of calculating the mo-

  yenne models, so we can produce an average model which is

  identical or equivalent to that which would be obtained by making the

  yenne logarithmic spectral parameters of all elements at

  same speech level and at the same signal / noise ratio. Therefore

  quent, to simplify the whole problem, it is assumed that there

  signal-to-noise ratios in all models as well as in all model elements. This can be achieved by adjusting the speech levels in all elements to be equal, and equal signal-to-noise ratios therefore result in an equal noise value in all elements. Under this assumption, all forms of averaging the noise equivalent can be performed. As noted above, research has shown that when models have the same signal-to-noise ratio as the unknown,

  speech recognition performance is better than with mo-

  LEDs with less or more noise. It is therefore indicated that, on

  the basis of the above techniques, the signal / noise ratio of the signal.

  audio can be predicted so recognition performance can

  be optimized by modifying models before they are used

  so that they are "as if" they were e from

  speech having the same signal-to-noise ratio as speech

known imminent.

  As indicated, we therefore proceed in two stages. One is

- 23 -2627887

  te predict the signal-to-noise ratio of impending speech, and then modify the models to meet this requirement. As we

  will see, the speech and noise detector 26 does not perform an

  timation of the speech power in each channel, because it varies

  from one word to another depending on the phonetic content of each

  of them. As we cannot predict the words that will be pronounced-

  However, the data would have no predictive power. The important thing is that for normal procedures there would be no estimate

  signal-to-noise ratio for each channel. Therefore, the process

  The modification time of the models indicated above avoids using signal / noise values for each of the channels. It is therefore indicated that the models resulting from a noise power equal to its value

  average give very good results in a recognition system

sance.

  In other words, there is no point in worrying about the

  noise power from one frame to another, - because it

  is sufficient to use the average value. The mo-

  so they are those that would be produced from the same

  speech capacity than that which actually exists in "basic shape" models, combined with the average noise power of the

  moment. For the most part, as indicated above, the module for

  speech and noise detection 26 is a digital processing circuit

  that of signal (DSP) which intervenes to execute an algorithm four-

  providing a measurement of the power level of a speech signal in the presence of additional acoustic noise and also a measurement of the average noise power in the bandwidth filter bank channels in any arbitrary form. The measurement of the noise level obtained is indicative of the level of conversation of the speaker making it possible to adjust the signal / noise ratio for speech recognition. Other measures of speech level vary rapidly and / or with the relative frequency of occurrence of vocal sounds and not

  inside the spoken word. The measurement obtained by the speech and noise detector remedies this problem by

  detecting the slightly smoothed peak power of the vocal nuclei

ques.

-24- 2627887

  More specifically, it detects the light peak power

  the smoothed vocal nucleus containing the most energy. In ne-

  gliding power peaks during unstressed syllable nuclei and during speech intervals which are not vocal nuclei, the measurement constitutes a continuous indication of the general level of speech. The detector is intended for use in

  presence of additional noise without correlation with speech present

  te when total noise power usually varies slowly-

  ment with respect to the frequency of production of vocal nuclei in speech (typically from 5 to 15 per second). The detector also has the function of catching up with faster changes in the noise level. The speech and noise detector 26 uses a logarithmic or compression technique and performs a measurement of

  the total speech power over the frequency range concerned.

  This measurement is first subjected to a filtering process with slow rise and rapid descent, the constraints of rise time and

  descent being chosen so that there is a great difference

  tive between the instantaneous signal power and the filtered value during the first few milliseconds of the vocal nucleus,

  whereas this difference does not adopt high negative values.

  A non-linear function of the difference between the power-

  this instantaneous signal and the filtered value with lowering time

  pide and with slow rise time is then directed towards a process of integration with long pulses of an appropriate duration, so that the resulting value exceeds an appropriate threshold only during normal vocal nuclei or accentuated speech intervals, in

  usually omitting unstressed vowel nuclei. Not-

  wise this threshold is then used to identify an interval of

  high signal strength corresponding to speech cores.

  Only intervals thus identified are used for the detection

  speech levels. The values that come from the process

  integration with long pulses and which are greater than one

  cond threshold below the threshold of the speech nuclei are then used

  read to identify intervals that contain a power

  speech and noise power. Only the intervals, for the-

AT-.25 - 2627887 '

  which the long pulse integration value is less than the second (lower) threshold and for which the instantaneous power

  is not greater than a third threshold greater than its fil-

  fast down and slow up are used as input to the noise power detection function. As indicated, the noise power detection module can essentially comprise a signal processing module

  digital realized by an integrated circuit chip. There are many

  such large chips which, for the most part, are programmable and designed to execute various types of algorithms. The aim of the algorithm which is associated with the noise and signal detection function is to determine both the energy content of the signal and the

  energy content of the noise and proceeds as follows.

  We first obtain a mathematical value indicative of

  the energy of the channels. This is done in each of the frames.

  The total energy is then calculated. The system can then continue.

  vre to account for changes in automatic gain control. As soon as the energy is calculated, the results are smoothed over given time intervals. When the energy value smoothed

  is obtained, the logarithmic value of the total energy is calculated.

  After calculating the logarithmic value of the total energy, a long pulse integration or an average is carried out for each estimate of speech level at the input of the arrangement of

  bandpass filter. The next step involves using a wire-

  be asymmetric allowing to filter the logarithmic energy for

  detect speech by monitoring the rise time of the speech signal

  role. It will immediately be noted that the speech signal is designated at

  generic title and that the incoming signal may be noise, an arte-

  fact which is not a noise or speech signal and which may be due to heavy breathing or other characteristics of the voice

  the speaker who, for the most part,. do not constitute information

  neither noise. In any case, it can also be a real

speech signal.

  To determine this, we therefore monitor the instantaneous values

  born of logarithmic energy over smoothed energy. The algorithm has

- 26 - 2627887

  function of dividing the time interval into given periods

  associated with signal rise and fall times. When the world

  ted is positive rather than negative, some decisions are made regarding the nature of the incoming signal to be recognized. These decisions, indicated above, determine whether it is speech, an artefact

  or pure noise. For example, for an interval for which the

  Tee is negative, it is absolutely accepted that if the climb remains ne-

  negative, this is a noise signal. The noise signal is accepted

  and the system continues to detect the signal by smoothing the

  noise values, using these values to help fuel it

  average noise and using the calculated values to apply them to the noise estimate. This value is then used

  to form the model. The operation is more difficult for a tran-

positive position.

  A positive transition can translate a noise, an artifact

  or a word. To make this determination, we carry out and

  works on an integral of a nonlinear function. Based on the comparison between the value of the integral and certain thresholds,

  we can therefore determine whether or not a positive rise is indicative of

  ve of a word, a noise or an artifact. In this way, the values that come from the speech and noise detection module

  are indicative of the true speech value. The programs intended

  born at the speech and noise detector are shown in Figures A to 5C o the complete program is indicated. FIG. 6 gives the definition of the technical parameters necessary to understand the programming formats illustrated in FIGS. 5A to 5C. For further explanation, the procedure is performed once for each frame and proceeds as follows. The

  first step in the procedure shown in Figure 5A is to ob-

  hold the energy of each channel as well as the total energy. This is shown in steps 1 and 2. Then the energy is filtered in

  each channel, taking into account changes in the scale of the control

  the automatic gain, as indicated in steps 3 and 4. The next step

  Vante consists in smoothing the energy values to obtain values

  their smooth energy logarithmics, which are corrected for

- 27 - 2627887

  tion of the automatic gain control. This is shown in steps, 6 and 7. The next step is to get an impulse average

  long terms for the speech level estimate in step 8.

  Next, we obtain the asymmetric filter value of the energy and the rise of the momentary energy over the filtered value which is represented in steps 9 and 10. We then leave this particular aspect of the program to go to Figure 5B. The variable r represented at

  step 10 of FIG. 5A corresponds to the value whose energy lo-

  garithmic of the moment ex-cedes its smooth asymmetric value. During the

  vowel nuclei, r becomes positive and the rest for an interval

the considerable.

  This value is of particular importance for its inter-

  positive and negative values and special treatment is therefore necessary

  saire when it first becomes positive or first negative. This is shown in detail in Figure 5B. In any case, if r- first becomes positive, the frame number is recorded as marking the possible start of a defined speech kernel. Then we reset

  the value of P which is used to decide whether it is speech.

  We continue by suspending the noise detection. In any case, during

  that r remains positive, we accumulate the values of p and we place the in-

  artifact and speech indicators if P exceeds specified thresholds.

  These are shown on the left side of Figure 5B. Yes

  r first becomes positive, we return the noise detector to the last

  last known noise values and we therefore resume detection of

  noise after a given delay if speech or artifact has been detected

  while ensuring that the assumed noise level is sufficient

  high in relation to the noise level. If speech has been detected

  in this climb, the frame number is saved as

  marking the end of a known speaking interval.

  In any case, while r remains negative, we continue to

  detect noise after a given delay. This is represented in totali-

  tee in the attached diagrams which clearly describe the differences

your planned operations.

  Figure 5C essentially shows the generation of output variables which, as shown, are used to provide - 28-

262 788 7

  operational model data as shown

  tees, for example, in Figures 2 and 4. As can be seen from the above, the main aspect of this system is therefore to provide models, adding noise correctly and in advance to achieve a model to which is associated, for the es

  sentiel, a level of signal / noise ratio. The associated noise level

  linked to the model is indicative of an estimate of the noise which will be pre-

  feels in the impending signal. In this way, we increase

  significantly the probability of recognition of a recognition system

birth of speech.

  It should be noted that the generation of models of this type uses

  reading the noise addition, as shown above, can be used

  yeas in any speech recognition system used

  models to provide a comparison of these models with an incoming signal, to determine if that signal corresponds to the

  speech, artefact or noise. Therefore, the system in-

  tervient to provide speech recognition models which are first trained in the absence of noise and which are improved to be used in the presence of noise, by modifying them so that they

  are equal to their expected value in the presence of noise.

-29-

* - 29 - 2627887

Claims (21)

  1. Speech recognition system of the type comprising an ectre analyzer (12) by supplying spectral intensity values of utterances at an output and for comparing stored models with   spectral values processed to provide output in case of   favorable parison indicative of the presence of speech in said   statement, including the corresponding development of the apparatus for   said stored models includes first means connected to said eutr analyzer (12) for providing a signal representative of the   noise signal expected from an incoming signal, and a means connected to it   said first means and responding to said predicted noise signal to ert cker models which are modified according to said predicted noise signal.   2. Speech recognition system according to -the-   dication 1, characterized in that said first means comprises a mo-   noise and speech level detection yen, the function of which is to provide, at an output, a first signal indicative of the power level of a speech signal in the presence of noise, and a second   signal indicative of average noise power.   3. Speech recognition system according to the res-   dication 1, characterized in that said spectrum analyzer (12) comprises   takes several. bandpass filters disposed in a filter bank arrangement, each filter being adapted to pass a given spectral component in accordance with the bandwidth of said filter.   4. Speech recognition system according to the res-   dieation 1, characterized in that said second means comprises a mo-   yen to generate models under low noise conditions and   to modify said models in accordance with said pre-noise signal said.   5. Speech recognition system according to the res-   dication 1, characterized in that said first means comprises a mo-   yen to predict the signal-to-noise ratio of an immediate speech signal - 30 -2627887 nent.   6. Speech recognition system according to the res-   dication 3, characterized in that said first means comprises a mo-   yen to measure the mean and the variance of the so-called bandpass filters   of to provide an estimate of the noise passage properties of each filter.   7. Speech recognition system according to the res-   dication 6, characterized in that said noise estimate is made   read on the basis of said filter response to Gaussian noise.   8. Speech recognition system according to the res-   dication 4, characterized in that said models oe _en absen-   this decit stsssu 'deelmpts iphdeuit in that it caigtun men   responding to said models to provide an average value for four-   supplying basic shape data to outputs and means for modifying said basic shape data in accordance with a signal of noise predicted of the moment.   9. Speech recognition system of the type comprising a   spectrum analyzer to provide intensity values to an output   spectral utterance of statements and to compare memorized models given   born at spectral values processed to provide an output in case   favorable comparison indicative of the presence of speech in the-   said statement, the corresponding improvement of the apparatus for tajk = said memorized models comprises a processing means connected to said analyzer for ensra models for memorization by modification of said given models in accordance with a calculated calculated value indicative of the presence of noise, and means for comparing said ergdr models to incoming signals for provide said output.   10. Speech recognition system according to the res-   dication 9, characterized in that said means for processing said expected calculated value is indicative of the presence of Gaussian noise.   11. Speech recognition system according to the res-   dication 9, characterized in that said processing means comprises means for averaging the noise-free models for -31- - 3 '- 2627887   provide Basic Form data outputs and to modify   said Basic Shape data outputs by adding, to said data   born, calculated noise data.   12; Speech recognition system according to the res-   dication 9, characterized in that said processing means comprises an averaging means for supplying, at the output, the average value of successive pairs of said spectral intensity values   provided by said analyzer, and a connected scaling means   tee at the output of said averaging means to provide a si-   gnal of field of given length and a means for converting said si-   gnal of -field of length given in a logarithmic signal for   providing one of said Basic Shape data outputs.   13. Speech recognition system according to the res-   dication 12, further comprising a square elevation means   connected to said averaging means to provide, at an output, a   vector signal indicative of the squared intensity of said moderate value   yenne of successive pairs, and a means connected to said output of the   said means of squaring to provide other data outputs born from Basic Form.   14. Speech recognition system according to the res-   dication 13, characterized in that said means connected to the outlet of said squaring means comprises a means of production   of relative energy responding to said vector signal to provide a pa-   Energy meter of Basic Shape, and a means of detecting ni-   noise and speech calf to provide, at an output, a Basic Shape parameter indicative of the speech and noise power level. 15. A method of forming models which can be used in a speech recognition system, comprising the following steps: providing a signal indicative of an expected noise level   for an imminent signal, and modify a given model in accordance with-   said signal supplied to provide a model showing said level of expected noise.   16. Method according to claim 15, characterized in that said supplying step consists in measuring the response of a - 32 - 2627887   speech processing channel given in relation to noise and to be estimated   said signal to be provided on the basis of said measurement.   17. Method according to claim 15, characterized in   what said modification step first consists in forming a mo-   relatively noiseless base shape model and to modify said base shape model in accordance with said indicative signal of said expected noise level.   18. The method of claim 15, characterized in that the modifying step consists in forming relatively noise-free Basic Shape models, adding noise to each model, and averaging said noise model data.   to form new models according to said analyzed data   sées. 19. Method according to claim 15, characterized in   what the step of providing a signal is to provide the report   signal / noise port of an imminent signal to be recognized by modifying the strength of a signal present by averaging the logarithmic spectral parameters of all models at the same speech level   and at the same signal / noise ratio, and to use said modulated parameters   yen to form modified patterns.   20. A method of forming models which can be used in a speech recognition system, comprising the steps   following: modify the trained models before they are used   entered for comparison by adding to said models a noise signal indicative of a predicted value, so that said models   modified behave as if they were enrpjrlfrom a si-   general speech with the same signal-to-noise ratio as a signal imminent to recognize.   21. Method according to claim 20, characterized in   what the modification steps are to predict the ratio if-   general / noise of an imminent speech signal using such a ratio   general / noise of the moment as said predicted value on the basis of a ni-   speech calf of the moment, and to average the power of   noise and speech power of the moment to define said if- general noise added.