DE4317991A1 - Speech recognition system using neural network and fuzzy logic processing - divides speech signal into frequency bands from which energy levels are obtained for use in fuzzy logic processing. - Google Patents

Abstract

The speech recognition system receives an input from a microphone (10) that is amplified (20) and received by an analyser (30). The analyser output is transferred via an interface (40) to a personal computer (60) complete with hard and floppy discs (50), keyboard (70) and display screen (80). The speech analyser divides the speech signal into predetermined frequency bands in which energy levels are defined. The personal computer received the defined energy signal levels and compares the signal levels of adjacent bands. A binary data value is then generated for identification. The processing is based upon a neural network and fuzzy logic theory. USE/ADVANTAGE - Esp. for mono-syllabic Korean speech. Improved speech identification process.

Description

The invention relates to a speech recognition system, in particular especially a speech recognition system for monosyllabic Koreani language.

When extracting features for speech recognition is the perception of one spoken for the distinction syllables essential characteristic due to speech speed, accent and pronunciation habits of a be agreed speaker, changes in the environment during the Speaking, emotional state of the speaker etc. very much difficult.

Also represented due to the fluctuations in articulated language even a single phoneme dene phonetic characteristics, which in turn by previous and subsequent phonemes are affected. These factors bring difficulties for the development of an algorithm must be used to extract the special language features and complicate the presentation and integration of the knowledge obtained by the algorithm.

Research into established systems has been carried out on a broad basis sis performed to solve the above problems. As a single method there z. B. a formant ana lysis for the classification of voiced sounds with regard to egg a vocal frequency component, a dynamic time Deformation process (Dynamic Time Warping or DTW-Ver drive), in which temporal disturbances between the individu ellen articulations of a single word using a dynamic programming technology can be reduced so that the greatest possible uniform language for recognition can be used, and a veiled Markov model Process (Hidden Markov Model or HMM process) for Speech recognition by presenting a phonetically together set signal used.

However, since most of the previously realized the before speech recognition systems using existing methods an enormous amount of computation to recognize, of course  human language and accepting the various It requires factual articulations to be articulated usability and meaning doubtful, and genuine time speech recognition becomes difficult. A fuzzy or un neural network model using sharpness theories has been proposed as a process that is capable of general problems of pattern recognition, e.g. B. Speech recognition to solve.

In contrast to a von Neumann computer, this is assumed from that the neural network model is not only the most suitable neten rules for solving themselves due to ambiguous, un complete and contradictory data Learns questions but also a variety of neurons par processed allel, so that the neural network model in an environment requiring parallel processing as in for example, voice recognition can be used. The neural network model offers three basic advantages le.

First, it is highly adaptable. That is, human The corresponding language is ambient noise and egg characteristics of a speaker different from what is new ronal network model can be learned effectively.

Second, the learning process is useful. While an algo rithmic processing by suitable extraction of a indefinite characteristic from a variety of Voice data is very difficult, the neural network werk-Modell extract the characteristic yourself and by darge learn offered examples.

Third, the above parallel processing can be performed will. The result is obtained in the neural network model nis by the numerous, parallel processing leading base neurons, so that is required for learning enormous time expenditure due to parallel processing becomes manageable.  

A special Refe border pattern determined in advance or an abundance of oneself Rules relating to voice data are individually programmed. Im that system using neural network, however, can the outward appearing characteristics or Information is learned, patterns are classified, without any special information about a change in characteristics lubricating instruction is required, and a larger lei Ability to a modified pattern can be formed.

A representative neural network model, which be is already using the neural network model as speech recognition procedures for improving speech recognition performance introduces is a time-delaying neural network (Time Delay Neural Network, TDNN) which performs well when recognizing syllable phoneme language. Fer ner found an experiment in which a phonological Group corresponding subordinate network as a module is shown, an extended area of recognized ob projects without the high recognition rate of the subordinate Network.

To the properties of the neural described above Network actually apply and sufficient Ge need to make is a hard-wired hardware scarf required, which, however, in contrast to software Simulation that presents many limitations. For this Basically, a neural network in the form of an input controlled multi-layer perceptrons (input-driven multi-layer Perceptron, IDMLP).

Furthermore fuzzy or blurring theory is introduced, because due to the diversity of the voice data the Frequency characteristics corresponding to each individual spectrum cher differentiates, even if the same word in each case is spoken.

The invention is therefore based on the object, a language  detection system for Korean single syllables in which a neural network and one based on fuzzy Theory-based pattern comparison algorithm is used will.

According to the invention, this object is achieved by a speech recognition system with a voice recording device for Record a speech signal, a speech analysis direction for dividing the voice signal from the voice recording device in predetermined frequency bands, where the divided speech signal in each frequency band as Energy level is represented and a processing computer to take over the signal from the speech analysis Setup, binary conversion of the speech signal by comparison the energy level of each frequency band in relation to the energy level of a neighboring frequency band, and Output a voice signal corresponding to the binary data.

Advantageous refinements are in the subclaims characterized the invention.

The invention will now be described with reference to an embodiment game described with reference to the drawings.

Fig. 1 shows a block diagram of a circuit structure for performing a speech analysis according to the embodiment example.

FIG. 2 shows an embodiment of the analog amplifier from FIG. 1.

FIG. 3 shows an exemplary embodiment of the speech analyzer from FIG. 1.

Figure 4 shows a single frame output related to a spoken Korean vowel.

Fig. 5 shows a flowchart of the voice analysis of the voice analyzer.

Fig. 6 shows a flowchart of a speech analysis data extraction step.

Fig. 7 shows the spectral distribution of a speech signal.

FIG. 8 shows the binary-converted spectral distribution of the speech signal from FIG. 7.

Figure 9 shows the steps for binary converting blurred data.

Fig. 10 shows a structure of a neural IDMLP network.

Fig. 11 shows a table of syllables to be classified into one-syllable consonant-vowel pairs.

FIG. 12A to 12E show the individual modules related respective classification rates.

In Fig. 1, a speech recognition system includes a microphone 10 for voice recording, an analog amplifier 20 for amplifying the speech from the microphone 10 , a speech analyzer 30 for analyzing a speech signal from the analog amplifier 20 , an interface board 40 for adaptation to one Personal computer (PC), hard disk and floppy disk drives (HDD + FDD) 50 for exchanging data with the computer, a processing computer or host computer 60 connected to the interface board 40 and the hard disk and floppy disk drives (HDD + FDD) 50 , ei ne keyboard 70 forming input means of Hostcompu ters 60, and an output means of the Ho stcomputers 60 forming screen 80th

The analog circuit for the voice input of FIG. 2 is used for adjusting the volume of the captured Spra che using a variable resistor. A differential amplifier 90 (e.g. a TL072CP module) has a frequency bandwidth of 10 kHz, since normal human voice frequencies range up to 7 kHz. Since the digital and the analog signals are mixed or combined in the field of speech analysis, great attention should be paid to noise. In the present exemplary embodiment, noise is eliminated by a capacitor with a capacitance of approximately 0.1 μF. The output signal of the analog amplifier stage serves as an input signal for the speech analyzer 30 , which outputs 8-bit data.

Fig. 3 is a block diagram of an embodiment of the subdivided into two areas voice analyzer 30 and. Here, one of the areas is an interface area with an 8-bit comparator 31 (e.g. a 74LS688 chip) and an interface chip 32 (e.g. an 8255PPI circuit), this area performing the speech analysis (Reference numeral 33 denotes, for example, a µPD7763 module). Here, since the 74LS688 module is an open collector type, a pull-up resistor 35 is connected between the output (pin 19 ) and a voltage source (V _cc ). By using the 8255PPI module to adapt to the personal computer 60 , the interface board 40 was built and manufactured in a simple manner. The 8255PPI and µPD7763 devices are controlled by the host computer 60 , which will be described in more detail later with reference to the software. The input of address and data signals into the speech analyzer module 33 takes place via the address and data bus. A 4 MHz clock is required to synchronize the operations of the internal circuits of the speech analyzer module 33 , which is provided by the 4 MHz crystal oscillator.

An input "RESET" of the interface module 32 automatically resets it by contacting the reset terminal in an I / O slot of the host computer 60 when the host computer 60 is started up.

The resetting of the speech analyzer module 33 must be done before setting an input mode, but the overall circuit is enlarged excessively.

Therefore, this problem is solved in the present exemplary embodiment using the 8255PPI module and the associated software. Specifically, a value corresponding to the reset signal is placed on the data bus and on an output of the interface module 32 at the reset input of the speech analyzer module 33 .

When a frame signal port 33 a outputs "FRAME" of the speech Analysa tor block the completion of the analysis of a Rah mens representing logic "1", the HostCom reads computer 60 the value of a disposed within the voice analyzer module 33 16- Channel filter bank or a 16-channel bandpass filter. Using the interface module 32 serving as the interface of this area, the preprocessing area which requires the most time for speech recognition is formed by hardware, as a result of which a real-time speech recognition system is formed as a whole.

This means that although an enormous amount of time is spent on the Ana lysing speech signals using general software Simulation is wasted in this execution example the output signal of the 16-channel bandpass filter is achieved by hardware so that the time required ver wrestles and thereby a system is formed which for Real-time voice recognition can be used.

Table 1 below shows the frequency characteristic of the sixteen bandpass filters of the speech analyzer module 33 from FIG. 1.

Table 1

FIG. 5 shows output signals of a frame of the speech analyzer 30 ( FIG. 1) with respect to a Korean vowel, which enables the formant of the vowel to be monitored. Consequently, the speech analysis system designed and manufactured according to the exemplary embodiment has no disadvantages in the form of e.g. B. the need for a preprocessing circuit or board.

The structure of the software is divided into two. First, a control program allows a user to easily use the speech analysis system designed and manufactured as above. Such a control program sets the operating modes of the interface module 32 and of the speech analyzer module 33 and directs the input and output of data. Secondly, another program detects a speech interval on the basis of data which, after passing through the speech analyzer module 33 , are considered to be within a certain frequency range, normalizes lengths of different pronunciations, and ultimately binarizes or converts data into binary form that are used for input into the IDMLP neural network.

The setting of the operating mode of the interface module 32 serves to designate a connection via which the input and output is carried out in a basic operating mode. In the present exemplary embodiment, "PAO" and "PB1" are configured as outputs, the connection "PB1" also being identified as an input. The above operating mode is set by software commands in the programming language C:

outportb (0x307, 0x82); / * A + B connection output, B- Connection input * /.

In the hardware structure, the interface module 32 is used to initialize the speech analyzer module 33 and to determine the end of the speech analysis. Accordingly, the above first program for controlling the interface module 32 is very simple.

The above second program for controlling the speech analyzer module 33 is also required. At present, the time required for the analysis of the data and the reading out of the analyzed data must be skillfully calculated in order to ensure a stable overall system.

The speech analyzer module 33 should be initialized using the interface module 32 by the following C commands:

outportb (0x00); / * µPD7763 reset signal * /
delay (1); / * Duration of the reset signal * /
outportb (0xff); / * Remove the reset signal * /.

The above delay (int); Command becomes necessary because the return set signal initializes the system.

The I / O control area of the speech analyzer module 33 controls the external input and output data processing using the data bus connections DB0∼DB7.

Table 2 below serves to explain the Be drive sequences corresponding to five control signals CS, WR, RD, A0 and A1.

Table 2

After the reset signal has been released, an operating mode must be set within 378 µs. The operating mode of the speech analyzer module 33 is set by writing data into a command / status register (CSR) contained in the module via the data bus, the operating modes controllable by the host computer 60 ( 1) the duration of an analyzed frame, (2) preamplification, (3) switching an equalizer on and off, and (4) the cutoff frequency of a low-pass filter.

These four modes of operation become available by writing data to the command / status register CSR over the data bus of the computer 60 using the programming language C again as follows:

outportb (0x304, 0x4c); / * 0dB, 16 ms * /
outportb (0x304, 0x02) / * 25Hz, EQ off * /

Fig. 5 shows an overall flow diagram of the voice recording until the reading of the analyzed results.

Fig. 6 shows a step of detecting the voice interaction Valls and obtaining binarized data for a handover in the neural IDMLP network, which is executed after the output of the speech analyzer 30 via the interface component 32 in a memory of the Ho stcomputers 60 was saved.

In the present embodiment, a frame length set to 16 ms. The speech interval should be in full continuously analyzed data can be determined. In the present the embodiment, the frame is equal to the length of the speech interval when the energy level of a Frame larger than a preset threshold voltage is. If a person repeats the same word several times speaks, the lengths of the spoken words are rare same, so that a time axis normalization is carried out should. When a monosyllabic word is spoken repeatedly it generally lasts for 8 to 26 frames As a result, 15 frames are set as the reference value Timeline standardization performed and received as above binarized data.

According to the present embodiment, the Step from the input signal of the microphone to as on  output signal of the neural IDMLP network used Aus output signal of the 16-channel filter bank through hardware reali to reduce the collection time of the input data shorten.

The input data extraction step is used to form the above, standardized and binarized in the final state Data. It is carried out as follows:

• 1. The voice signal is received via the microphone 10 .
• 2. Using the circuit of FIG. 2, the received speech signal is raised to a suitable volume and used as the input signal of the speech analyzer 30 .
• 3. The result of the analysis is read out from the speech analyzer 30 .
• 4. The speech interval is set using the pre set threshold determined.
• 5. The language interval is standardized in relation to the Frame of reference corresponding timeline.
• 6. The output signal of each bandpass filter is compared with the of the neighboring filter compared, their relative Energy levels are binarized.

Table 3 below shows that from the above Speech interval determination step resulting data.  

Fig. 7 shows the spectral distribution of the speech signal.

Fig. 8 shows a binarized frequency spectrum. In this case, the energy level read from the speech analyzer 30 of each bandpass filter is compared with the output signal of the filter whose frequency band is immediately below, so that a logic "1" is set as the output signal of the filter when its output value is greater is than that of the lower frequency filter (if EfN-1 <EfN), and a logical "0" in the case of a smaller value (if EfN-1EfN).

Table 4 below shows the binarized speech Output signal using the method described above has been turned (here is the one corresponding to a frame Number of input bits 15).  

In the present embodiment, in one precede recognition of monosyllabic Korean language the step, the blur theory and the above he imagined adaptability of the neural IDMLP network when recognizing spoken numbers from "0" to "9" (the, pronounced in Korean, are monosyllabic). Furthermore, in the detection using the neurona len IDMLP network each syllable of the voice data with the ge learned result blurred, so that the recognition versu che in terms of using a voice data syllable learned result will be carried out and it will be possible Combination of neural network and Un sharpness theory measured.

Fig. 9 shows the step of binarizing the blurred data.

Each syllable by speaking the numbers of ten times Speech data obtained from "0" to "9" is convoluted and misused sharpens what the blurred data relates to suitable threshold value can be binarized.

Table 5 below shows the result of the above Binarization of blurred data.  

Since the number of input nodes during the learning of the Network is regular, the lengths of the under should different spoken sounds in relation to the timeline be standardized. In the present, 15 frames as a reference embodiment using the time axis Nor mation introduced gradually to an appropriate framework interval if the frame of an input mu is longer than the reference frame.

Here, 200 languages are spoken by a speaker data syllables as learning data of the neural IDMLP network is used, and the recognition attempt is 100 using recognizing speech data syllables performed each during three different periods (morning, noon, in the evening) should be recognized to adaptability of the presented speech recognition system with regard to the Determine diversity of voice data.

After learning both show the binarized data as also the blurred data in relation to the learning data a detection rate of 100%. Furthermore point in relation on the test data learning processes using both the binari As well as the blurred data, a high level of awareness rate (over 94%). These experimental results will be illustrated in the tables below.

If the neural IDMLP network using the binarized Learning data is the recognition rate of the recognition attempt 94% in the morning (Table 6A), 99% at noon (Table 6B), and in the evening 96% (Table 6C), with an average Er identification rate of 96.3%.  

Table 6A

Table 6B

Table 6C

On the other hand, if the neural IDMLP network with the blurred data learns the recognition rate of the Detection attempt in the morning at 97% (Table 6D), at noon at 99% (Table 6E), and in the evening at 98% (Table 6F), at an average detection rate of 98%.  

Table 6D

Table 6E

Table 6F

As can be seen from the tables above, is the midday detection rate is the highest among the three Trial times, and the date with the worst orchestration is the number "6" spoken (in Korean).

Learning the IDMLP neural network was after one Passage in terms of learning using either the bi narized or blurred data ended. Even though the structural properties of the neural IDMLP network works that have not been tested are shown in the recognition result not much of a difference when using the verun sharpened data is learned, except that the detection rate recognition of spoken numbers is slightly higher.

For the purpose of applying the IDMLP neural network to Korean monosyllabic words are recognized first a type-classifying neural network for classification adornment of the words in five vowel-oriented modules det, and the entire network of six modules sets to the final recognition by each of the un subordinate networks based on the classified types  perform.

During the type classification, input sound is classified into five groups using five vowel sounds. The concrete phonemes to be classified are 70 Korean single syllables with a consonant vowel (KV) structure, which, as shown in FIG. 11, are formed in conjunction with 5 Korean vowels.

Those learning the input sound in five groups ordering type-classifying neural network ver data is extracted by extracting only the Vo derived from the input sound, whereby the learning process takes place.

In any case, the data used is monosyllabic K-V structure, which is why the vowels are simply extracted that can.

Because there is a vowel at the end of a spoken word the extraction of some at the end of the frame of reference is sufficient (e.g. 15 frames) existing frames for type classification tion. However, the above algorithm is unsuitable if the gradual expansion of the concrete words to be recognized is taken into account, d. i.e. if language with a monosyllabic gene consonant-vowel-consonant structure (K-V-K structure) is taken into account.

For this reason, in the present embodiment just extract the center area of the entire frame and as learning data of the type-classifying neural Network used. Using a like the extracted data described above trained neural Type classification in the morning, at noon and in the evening as a spoken digital recognition experiment carried out.

As learning data, data are used that are obtained by extraction the vowel parts from 350 speech data syllables (70 by verb syllable formed with 5 vowels, each five  were spoken by a speaker) the. About 420 voice data syllables for use each of the five modules, and a total of 2500 syllables with which the module analysis test is carried out.

FIG. 18A-F show a rate of analysis with respect to speech data of a monosyllabic KV structure reaches the egg in the middle NEN type classification success of 98.4%. The respective type classification rates became 97.6% for the "a" module, 98.6% for the "e" module, 98.1% for the "i" module, 98.3 % for the "o" module and 98.1% for the "u" module. Here, a type classification rate greater than 90% was achieved when examining the type classification with respect to KVK monosyllabic speech data, although the learning was continued by extracting the data from the KV monosyllabic speech data.

Ultimately, the speech described above can recognize system in such a recognition attempt the result of both recognition koreani single syllables as well as those beyond Speech recognition from single syllables, d. H. achieve multi-syllable language. Furthermore, a new one Speech recognition system can be realized.

The speech recognition system described above has thus a voice recording device for recording a Speech signal, a speech analyzer for dividing the Speech signal from the speech recording device in vorbe tuned frequency bands, the divided signal in the individual frequency bands as energy levels and a host computer to take over the Si gnals from the speech analyzer, binary conversion of the input signals by comparing the energy level of each Fre quenzbands in relation to the signal, and outputting one has binarized data corresponding voice signal, whereby the realization of a new speech recognition system stems possible.

Claims (3)

1. Speech recognition system, characterized by

• - a voice recording device ( 10 ) for recording a voice signal,
• - A speech analysis device ( 30 ) for dividing the speech signal from the speech recording device ( 10 ) into predetermined frequency bands, wherein the divided speech signal is represented in each frequency band as energy level, and
• - A processing computer ( 60 ) for taking the signal from the speech analyzer ( 30 ), binary changes of the speech signal by comparing the energy level of each frequency band with respect to the energy level of an adjacent frequency band, and outputting a speech signal corresponding to the binary converted data.

2. Speech recognition system according to claim 1, characterized by an adaptation device ( 40 ) for mutual adaptation of the speech analysis device ( 30 ) and the processing computer ( 60 ). 3. Speech recognition method for use in a speech recognition system with a speech recording device ( 10 ) for recording a speech signal, a speech analysis device ( 30 ) for dividing the speech signal from the speech recording device ( 10 ) into predetermined frequency bands, the divided Speech signal in each frequency band is represented as an energy level, and a processing computer ( 60 ) for inputting the signal from the speech analyzer ( 30 ), binary converting the speech signal by comparing the energy level of each frequency band with respect to the energy level of an adjacent frequency band, and outputting a voice signal corresponding to the binary converted data, characterized by

• a first step for inputting speech analysis data into the processing computer ( 60 ),
• a second step for determining a voice interval using data from the first step,
• a third step for performing a time axis normalization using data from the second step,
• - A fourth step for binary conversion of the temporally standardized data, and
• - A fifth step for storing the result of the fourth step in a memory of the processing computer ( 60 ).