JPS6336680B2 - - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION The present invention relates to an automatic speech recognition device for automatically recognizing speech signals uttered by humans.

Automatic speech recognition devices that automatically recognize speech uttered by humans are considered to be extremely effective in the future as a means of providing data and instructions from humans to electronic calculators and various machines. For example, if you use a device that recognizes numeric audio by connecting it to a computer, it becomes possible to input numerical data such as slips.In particular, audio signals can be transmitted to remote locations via telephone lines, so it is possible to issue slips, etc. You can immediately make inventory inquiries, etc. Furthermore, considering that it is possible to input voice signals while using hands and feet for other purposes, the effects brought about by automatic voice recognition devices are considered to be extremely large.

The pattern matching method is often adopted as the operating principle of automatic speech recognition devices that have been researched or published in the past. In this method, standard patterns are memorized in advance for all types of words that need to be recognized, and the degree of matching (hereinafter referred to as similarity) is calculated by comparing them with unknown input patterns. The word is calculated and determined to be the same word as the standard pattern that yields the maximum match. In this pattern matching method, standard patterns are prepared for all words to be recognized, so if the speaker changes, it is necessary to input and memorize a new standard pattern. Therefore, when recognizing hundreds of different words, such as the names of cities across Japan, it takes a huge amount of time and effort to pronounce and register all the words, and the memory required for registration is also large. It is expected that the capacity will increase enormously. Furthermore, the amount of processing required to pattern-match an input pattern with a standard pattern becomes enormous as the number of words increases.

On the other hand, input speech is divided into phoneme units and recognized as combinations of phonemes (hereinafter referred to as phoneme recognition).
The method of determining similarity with a word dictionary written in phoneme units has the advantage that the memory capacity required for the word dictionary is significantly smaller, and the contents of the dictionary can be easily changed. An example of this method is "Word speech recognition system using the outline form of the speech spectrum and its dynamic characteristics," Sanpo et al., Journal of the Acoustical Society of Japan 34 (1978).
It is stated in FIG. 1 shows the block configuration of a speech recognition device using this method. The input speech 50 enters a filter group 40 and is converted into a frequency spectrum, and then an acoustic processing section 41 calculates parameters Pe ₁ , Pe ₂ , Pe ₃ , G, H, V, and W necessary for phoneme recognition.

The phoneme recognition unit 42 uses these parameters to perform phoneme recognition to determine what each phoneme is based on a phoneme segmentation operation (hereinafter referred to as segmentation) and a standard phoneme pattern 45.
However, since the arrangement of phonemes at this stage is incomplete, the error correction unit 43 corrects the arrangement of the phonemes mainly using the rule 46 for combining Japanese phonemes, and completes the creation of the phoneme sequence. The word matching unit 44 uses a Confusion Matrix 47 (hereinafter referred to as Confusion Matrix) that represents the probability of each phoneme being replaced with another phoneme, being dropped, or being inserted into another phoneme, which has been statistically determined in advance.
Using the word dictionary 48 consisting of phoneme names and phoneme names, the degree of similarity with the entire word dictionary is calculated, and the dictionary item with the greatest degree of similarity is output as a recognition result.

This method performs phoneme recognition by focusing on the position of the peak in the spectrum, and the algorithm is simple. Also, since it focuses only on the position of the peak and its relative size, it is difficult to recognize phonemes due to differences in speakers or environments. This method has the advantage of being less affected by changes in the overall shape of the spectrum pattern, and is considered to be a method suitable for targeting unspecified speakers.
However, this method has the following drawbacks because it uses a frequency spectrum obtained by a group of filters. Speech has a pitch component based on the length of the vocal tract.The pitch frequency of a male voice is generally below 200Hz and rarely overlaps with the first formant of a vowel, but the pitch frequency of a female voice is generally between 200 and 300Hz, which makes it difficult to pronounce a vowel. In the frequency spectrum of female voices, a large peak appears in the low frequency range due to the influence of the pitch frequency, and unnecessary peaks occur due to the influence of harmonics of the pitch frequency. There was a drawback that it became impossible to detect the exact peak position. For example, FIG. 2 shows an example of the frequency spectrum of the vowel /e/ in a female voice according to a filter group consisting of 29 channels between 250 and 6300 Hz and Q=6 filters every 1/6 octave. In the figure, the vertical axis is the channel number divided at 1/6 octave intervals,
The horizontal axis is a frame number divided into 10 ms units, and correct phonemes are named by inspection in advance. According to this, peak B, which does not correspond to the formant of the vowel /e/ due to the influence of pitch, appears in channels 2 and 12 on the vertical axis, and it is difficult to distinguish it from peak A, which corresponds to the true formant.

An example of phoneme recognition using this method is shown in FIG.

Figure 3 shows the word "cheap" uttered by an adult female, with the horizontal axis divided into frames. In the diagram a
is a phoneme named manually, the bar indicates the beginning of the phoneme, and the boxed area indicates the center. b represents the vowel recognition result, c represents the semi-vowel recognition result, d represents the silent and consonant sections, Q represents the silent period, and C corresponds to the consonant period. e indicates the consonant recognition result. f is various parameters for segmentation,
g is the position of the peak of the spectrum on the frequency axis expressed as 1, 2, and 3 in descending order of power. From this figure, we can see that in the /a/ part, a peak appears around 250Hz due to the influence of pitch (indicated by area A in the figure), which means that /a/ is mistaken as /i/ (indicated by area D in the figure). I understand. Also/
A peak appears around 250 Hz at o/ due to the influence of pitch (areas B and C in the diagram), which is incorrectly written as /n/ (areas E and F in the diagram). As described above, it can be seen that in the conventional method, the pitch component of the voice is strongly influenced by female voices, and it is not possible to cope with female voices.

The present invention reduces the pitch component of speech by linear predictive analysis instead of the conventional filter group in order to accurately determine the position of peak A in Figure 2, which corresponds only to formants, for both men and women. The present invention provides a phoneme recognition method and a speech recognition system that can solve the above-mentioned problems by obtaining a frequency spectrum that corresponds to a specific frequency spectrum, and that can be applied to unspecified speakers regardless of gender. Linear predictive analysis is a method of approximating the frequency spectrum with an all-pole model and separating the frequency spectrum envelope characteristics and vocal fold wave characteristics, and the influence of the pitch frequency and its harmonics should be reduced. Furthermore, since the frequency spectrum does not include components other than the specified model, there is an advantage that a smooth spectrum pattern can be obtained. FIG. 4 is an example in which the frequency spectrum of the vowel /e/ in a female voice, which is the same as in FIG. 2, was obtained using the method according to the present invention.
It can be seen that unnecessary peaks due to the influence of pitch have been removed and only peak A corresponding to the formant is depicted.

FIG. 5 is an example of phoneme recognition of the same word as in FIG. 3 using the method of the present invention. When you look at g, /a/
No peak appears near 250Hz at the position (indicated by area ① in the figure), indicating that the pitch component has been removed. From this, if we look at b, /a/
It can be seen that is correctly recognized as /a/.
Also, at the position of /o/, the pitch component around 250Hz that appeared in the conventional example in Figure 3 is removed (indicated by area C in the figure), and /o/ is segmented as a vowel. I know that there is. In this way, if vowels, consonants, and semi-vowels are segmented correctly, word recognition can be performed correctly.

As described above, the present invention can be applied to both male and female voices by accurately extracting peaks corresponding to formants using linear predictive analysis, and can be applied to voices suitable for unspecified speakers. This enables automatic recognition.

FIG. 6 shows an outline of the configuration of this automatic speech recognition device. Speech input enters the acoustic processing unit 1 and performs linear predictive analysis to calculate parameters necessary for phoneme recognition such as frequency spectrum and power. Phoneme recognition unit 2
Now, the parameters W, A, obtained by the acoustic processing section 1 are
G, H (details will be described later), and the peaks of the frequency spectrum obtained from the frequency spectrum Pe ₁ , Pe ₂ ,
Using Pe ₃ (hereinafter referred to as local peak), 10
Phoneme recognition is performed every ms analysis interval (hereinafter referred to as a frame). The phoneme sequence creation unit 3 corrects the sequence of phonemes (hereinafter this work is referred to as error correction) using rules that have been created in advance to connect common Japanese phonemes to each other (hereinafter referred to as phoneme combination rules).
Create a phoneme sequence (hereinafter referred to as a phoneme sequence) for each word. In the word matching section 4, the probability of each phoneme being replaced with another phoneme, addition of another phoneme, or omission error is calculated statistically in advance.
Using the Confusion Matrix, the degree of similarity between a word dictionary created from pre-registered phoneme sequences and the recognized phoneme sequence is calculated, and the word with the highest degree of similarity is output as the recognition output.

The configuration of the sound processing section 1 is shown in FIG. After the audio input is A/D converted and the pre-emphasis section 10 performs high-frequency emphasis of 6 dB/octave to correct the spectral slope, the window section 11 converts the audio input cut out for each frame using equation (1). T expressed = 20m
Apply a Hamming window every s.

y(t)=0.56+0.44cos2πt/T...(1) However, when |t|>T/2, y(t)=0 In the linear prediction analysis unit 12, "Speech Analysis
and Synthesis by Linear Prediction of the
Speech Wave” BSAtal etc, J.Acoust.Soc.
S ₁ , S ₂ , ..., S _o , ...
If S is _N , the prediction error S^ _o at the analysis order p is expressed by equation (2).

S^ _o = _P 〓 ^k=1 a _k ^(p) S _ok ...(2) Here, a _k ^(p) (k = 1, 2, ..., p) is a linear prediction coefficient. On the other hand, the autocorrelation coefficient of the analysis interval is r _k
If (k=1, 2,..., p), r _k can be found by equation (3).

r _k = 1/N _NK 〓 ⁿ⁼¹ S _o S _o+k …(3) In order to obtain the minimum average 2 errors of the prediction error S^ _o in equation (2), set r in (3) as N≫p. _k is used to determine the linear prediction coefficient a _k ^(p) in the p-element simultaneous linear equations in equation (4). Therefore, by solving equation (4), the linear prediction coefficient a _k ^(p)
(k=1, 2,..., p) is calculated, and it is generally known that this can be calculated accurately using Levinson's method.

The frequency spectrum calculation unit 13 calculates the spectrum envelope ^* (n) from the linear prediction coefficient a _k ^(p) obtained in the previous stage. Find it with Here, σ ² is the residual power, A(i)= _pi 〓 ^j= 〓a _j (p) a _j+i (p) …(6) θ _o =2π(n)T, and the frequency (n ) are set to have equal octave intervals, and ^* (n) is determined by setting the residual power of equation (5) to σ ² =2π.

The peak extraction and parameter calculation unit 14 determines the position and size of the local peaks on the frequency axis from the local maximum point and inflection point of the spectrum envelope calculated using equation (5), and calculates the positions and sizes of the local peaks on the frequency axis in descending order of peak size. Let Pe ₁ , Pe ₂ , and Pe ₃ be P ₁ , P ₂ , and P ₃ on the frequency axis in ascending order of frequency, and at the same time, determine G, H, and A as parameters necessary for phoneme recognition as follows. First, the least squares approximation straight line Y(n) of the spectrum X(n) obtained by logarithmically transforming the spectrum envelope ^* (n) is obtained using the following equation.

Y(n)=A·n+B (7) Here, the coefficient A indicates the overall slope of the spectrum, and B is a value indicating the overall level of the spectrum. Spectrum
If the spectrum normalized by (n) is Z(n) (called a normalized spectrum), then Z(n) = Low range (177-400
Hz), midrange (400~1100Hz), high range (1100~2800Hz)
The ratio of the average power of the normalized spectrum to the average power of the low frequency range is determined as G, and the ratio of the average power of the high frequency range to the power of the middle frequency range is determined as H.

Furthermore, the power for each frame of 10 ms length is determined by the sum of the squares of the audio signal, and the power is logarithmically transformed and used as the parameter W.

W=10/Nlog ₁₀ ( _N 〓 ⁱ⁼¹ Si ² ) …(9) While calculating the above parameters, simultaneously calculate the value of W and
The values of Pe ₁ and Pe ₂ are used to determine whether each frame is silent or has sound.

The phoneme recognition unit 2 first determines the start and end of the utterance from the silence/voice information obtained by the acoustic processing unit 1 using the continuity and duration of utterance or silence. Next, the local peak Pe ₁ obtained by the acoustic processing unit 1,
Pe ₂ Pe ₃ and P ₁ , P ₂ , P ₃ and parameters W, G,
Phoneme segmentation and phoneme determination are performed using W _s , G _s , H _s , and A _s obtained by smoothing H and A. To explain using Figure 5 as an example, first, consonant segmentation is performed by capturing the minimal changes in W _s and A _s , and then the local peaks Pe ₁ ,
Pe ₁ , Pe ₂ , constructed based on the distribution of Pe ₂ , Pe ₃ ,
Pe ₁ , Pe 2 , Pe ₂ for each frame in the standard pattern of Pe ₃
The results are shown in column d, which determines consonant candidates for each frame by applying Pe ₃ , and then determines the consonant in that section by applying a rule based on the number of consonant candidates.
Next, the semi-vowels are segmented using the maximum and minimum changes in G _s and H _s , and a two-dimensional map of P _{1 and} P ₂ (dotted line in Figure 8) is created based on the distribution of P 1 and P ₂ . P ₁ and P ₂ of each frame are applied to determine the semi-vowel in that section. Column c′ is the result. Finally, the vowels are recognized based on the distribution of P ₁ and P ₂ , with five vowels /i/, / as shown in the solid line in Figure 8.
e/, /a/, /o/, /u/ and the middle vowel /
P ₁ of ie/, /ea/, /ao/, /ou/, /ui/,
After fitting P _{1 and} P ₂ of each frame into the two-dimensional layout diagram based on P 2 , median smoothing is performed every 5 frames. Finally, each vowel is divided into vowels whose distance from the previous and subsequent frames is within 1, and those whose duration is 4 frames (40 ms) or more are determined as one vowel.

The phoneme sequence creation unit 3 corrects errors in the phoneme sequence based on pre-prepared Japanese phoneme combination rules, separates long vowels and simple vowels based on vowel duration, and creates devoiced vowels /i/, /u/ is inserted to create a phoneme sequence.

The word matching unit 4 uses a simple word dictionary created only from phoneme sequences and a Confusion Matrix created from a large number of recognized phoneme sequences in advance to calculate the degree of similarity between the recognized phoneme sequence and the entire word dictionary. The word with the highest similarity is output as the recognition result.

FIG. 9 shows the configuration of an automatic speech recognition device according to the present invention. The audio signal input from the microphone 20 is amplified to an appropriate level by the amplifier 21, and then sent to the A/D converter 2.
2, 12kHz sampling, 12 bit A/
D-convert. After this is subjected to 6 dB/octave pre-emphasis and a Hamming window in a signal processing circuit 29, a linear prediction analysis processor 23 calculates a frequency spectrum and parameters necessary for phoneme recognition from the linear prediction coefficients. The main processor 24 uses the main memory 25 to perform segmentation, phoneme recognition, and create a phoneme sequence example sequence, and transfers the obtained phoneme sequence results to the word matching processor 27. The word matching processor 27 refers to the word dictionary and the data in the Confusion Matrix memory 26 to calculate the degree of similarity for each word, and transfers the results to the main processor 24 . The main processor 24 outputs the word with the maximum similarity to the I/O 28 as a recognition result, or rejects the word. The I/O 28 sends the received results to other computers or causes other I/O devices to perform work. In this way, speeding up can be achieved by providing a dedicated processor in addition to the main processor to share the calculations.

Using this device, we conducted a word recognition experiment with a total of 3,320 data using 166 names of major cities across Japan uttered by 20 adult males in a soundproof room, and the average recognition rate was 84%. This is almost the same as the conventional filter bank method. On the other hand, according to the data uttered by 20 adult women, the conventional filter bank method could only recognize about 30% of the voices and was completely unable to respond to female voices, but with this device, male voices can be recognized. The effectiveness of the present invention was confirmed by making it possible to recognize up to 84%, which is the same as that for men and women, and opening the way for it to be applied regardless of gender.

As described above, the present invention is based on phoneme-by-phoneme recognition for unspecified speakers, and focuses on the positions of local peaks in the spectrum and their relative sizes to identify phonemes. By performing recognition, it is less affected by variations due to the speaker or the environment, and by using linear predictive analysis instead of a conventional filter group for spectrum analysis, it is not affected by pitch components of speech. It is possible to extract peaks corresponding to stable formants, and since the filter output is a discrete quantity, the extraction error is large, whereas the LPC cepstrum of the present invention is a continuous quantity, so peak extraction accuracy is improved. As a result, it is possible to create an automatic speech recognition device that is capable of responding to unspecified speakers of both genders.

[Brief explanation of the drawing]

Fig. 1 is a block diagram of a conventional automatic speech recognition system, Fig. 2 is a frequency spectrum diagram of the female vowel /e/ using a filter bank, which is a conventional example, and Fig. 3
The figure shows an example of phoneme recognition by conventional filter analysis, Figure 4 is a frequency spectrum diagram of the feminine vowel /e/ by linear predictive analysis of the present invention, and Figure 5 shows an example of phoneme recognition by linear predictive analysis of the present invention. Figure shown, No. 6
The figure shows the block configuration of the entire automatic speech recognition device according to the present invention, FIG. 7 is a block diagram of the acoustic processing section of the present invention, and FIG. 8 shows the local peak of the present invention.
FIG _{. 9} is a block diagram showing details of the automatic speech recognition device of the present invention. 1... Acoustic processing unit, 2... Phoneme recognition unit, 3...
Phoneme sequence creation section, 4...Word matching section, 10
...Pre-emphasis section, 11...Window section, 12...
...Linear prediction analysis section, 13... Frequency spectrum calculation section, 14... Peak extraction parameter section, 22...
...A/D conversion circuit, 23...Linear predictive analysis processor, 24...Main processor, 26...Word dictionary, memory for configuration matrix, 2
7...Word matching processor.

Claims (1)

[Claims] 1. An acoustic processing unit that processes voice input to calculate parameters necessary for phoneme recognition, a phoneme recognition unit that performs segmentation and phoneme recognition using the parameters, and a phoneme recognition unit that processes voice input and calculates parameters necessary for phoneme recognition; a phoneme sequence creation unit that corrects the phoneme sequence of the phoneme sequence to create a phoneme sequence; and a word matching unit that matches the phoneme sequence with a word dictionary; Tate uses linear prediction analysis to perform speech recognition based on the peak position and relative size of the peak in the frequency spectrum obtained from the maximum point or inflection point of the spectrum envelope determined from the obtained linear prediction coefficient. An automatic speech recognition device characterized by being configured as follows.