JPH0455520B2 - - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION Field of Industrial Application The present invention relates to a phoneme recognition method in a speech recognition method characterized by performing phoneme recognition.

Configuration of conventional examples and their problems The conventional method divides the input speech into phoneme units and recognizes them as combinations of phonemes (called phoneme recognition), and outputs the recognition result by calculating the similarity with a single dictionary written in phoneme units. A block diagram of the word recognition system is shown in FIG.

First, the voices of multiple speakers are analyzed in advance by the acoustic analysis section 1 using a filter bank for each 10 ms analysis interval, and the feature parameters are determined by the feature extraction section 2 based on the obtained spectrum information. . From these feature parameters, a standard pattern is created for each phoneme or phoneme group represented by vowels such as /a/, /o/ and consonants such as /m/, /b/, and is registered in the standard pattern registration section 5. I'll keep it. next,
The input voice of an unspecified speaker is similarly analyzed by the acoustic analysis section 1 for each analysis section, and the voice of the unspecified speaker is analyzed by the acoustic analysis section 1
Find the feature parameters by Using these feature parameters and the standard pattern in the standard pattern registration section 5, the segmentation section 3 performs a separation operation between vowels and consonants (hereinafter referred to as segmentation). Based on this result, the phoneme discriminator 4
By comparing the standard pattern with the standard pattern in the standard pattern registration section 5, the phoneme corresponding to the standard pattern with the highest degree of similarity is determined as the phoneme in that section. Finally, the time series of phonemes created as a result (hereinafter referred to as the phoneme series) is sent to the word recognition unit 6, and the word corresponding to the item with the highest similarity to the word dictionary 7 similarly expressed in the phoneme series is recognized. Output as result.

When phonemes are discriminated by the phoneme discriminator 4 in this configuration, conventionally, the feature parameters indicating the characteristics of the phoneme are obtained for each frame for the section determined to be a consonant section, and the standard for each phoneme or phoneme group is prepared in advance. Consonant classification is performed for each frame by comparing it with the pattern. This result is applied to the consonant classification tree, and those that meet the conditions are recognized as consonants. However, in this case, initial consonants are not clearly determined, but only phoneme groups are determined. For example, pa /b/, /d/, /g/ are voiced plosives.

Discrimination of voiced plosives has been reported, for example, in ``Analysis of Japanese voiced plosives'' by Hosoya and Fujisaki, Acoustical Society of Japan Speech Study Group (S80-67). However, due to the analysis time and complexity of the algorithms, these methods have not been reported to be used in actual word recognition systems.

As described above, the conventional method only discriminates phoneme groups for word-initial consonants, which may cause problems depending on the word to be recognized. In addition, a discrimination method within a phoneme group has been reported, but it has not yet been used in actual systems due to problems such as analysis time and algorithm complexity.

Purpose of the Invention The present invention was developed in order to solve the above-mentioned conventional problems, and provides a phoneme recognition method that can be used in an actual system by considering the analysis time and algorithm for recognizing initial consonants. The present invention achieves the above-mentioned object, and is capable of segmenting initial consonants of input speech by methods based on voiced/unvoiced determination, methods based on vowel nasality determination, and methods based on power change. , the method using cepstral distance is arbitrarily applied, and depending on which method was used for segmentation, initial consonants are divided into voiceless consonants, voiced consonants, consonants characterized by power changes, and duration. is recognized as a group of multiple phonemes, such as a group of short consonants, and then automatically detects a feature part (a part that is effective for phoneme discrimination) in the phoneme interval, and performs previous recognition on the feature part. The present invention provides a word-initial consonant recognition method characterized in that a phoneme is determined by calculating the similarity of each phoneme belonging to a phoneme group with a standard pattern.

Description of examples The outline of this example is as follows.

B. Using the results of the four segmentation methods, the voiceless consonant group, voiced consonant group, and
Consonants are classified into four groups: consonants characterized by power changes and consonants with short duration.

(b) A characteristic part is set for each polyphoneme group, and a standard polyphoneme pattern is created in advance for that characteristic part.The phoneme standard pattern uses a lot of data that has been accurately labeled by visual inspection. and create it. In addition to the phoneme standard pattern, one type of standard pattern for surrounding information of the characteristic part is created for each phoneme group.

C. Phoneme discrimination Perform word-initial consonant segmentation of input speech to find consonant intervals. Then, a part of the consonant interval (for example, an end point) is set as a reference point. on the other hand,
It is determined which phoneme group this consonant interval belongs to among the major classifications in A above. Next, the standard pattern belonging to the determined phoneme group is applied to the feature part in the phoneme interval to discriminate the phoneme. By the way, since it is generally difficult to automatically and accurately obtain characteristic parts, the following procedure is performed. In other words, by referring to the reference points above, find a candidate section for the characteristic part with some width, and then calculate the degree of similarity with each phoneme by applying the surrounding information standard pattern to the entire range of the candidate section. do. When calculating the degree of similarity with each phoneme, the degree of similarity with the standard pattern of the surrounding information of the phoneme group described in A above is removed from the degree of similarity between the standard phoneme pattern and the unknown human power. By doing this, it is possible to remove information on parts that do not correspond to the feature part (in other words, the parts around the feature part) out of the feature candidate interval, and to capture the accurate feature part and make a phoneme. can be determined.

Hereinafter, one embodiment of the present invention will be described in detail with reference to the drawings, taking consonant recognition as an example.

This example uses the results of the following four segmentation methods to classify word-initial consonants into (1) voiceless consonants, (2)
voiced consonants, (3) consonants characterized by power changes,
(4) Classify into four groups of consonants with short duration.

Every frame (in this example, one frame is 10m)
sec) method by voiced/unvoiced determination Method by frame-by-frame vowel/nasality determination Method to capture temporal changes in power Method by cepstral distance Methods used in conjunction with ~ and when an initial consonant is detected, the following methods are used Segmentation is performed based on the detection results. The methods up to ~ will be explained below.

The first method, which uses voiced/unvoiced judgment, will be described.

Segmentation of voiceless consonants at the beginning of words can be performed accurately by using the voiced/unvoiced determination results performed for each frame.

There are methods for determining voicedness/unvoicedness that use zero-crossing waves, spectral slope, first-order autocorrelation coefficient values, etc., but any method may be used. In this embodiment, the determination is made by comparing with a standard pattern for voiced/unvoiced determination.

Here, when the unvoiced state continues for a certain number of frames or more from the beginning of the word (for example, 4 frames or more), this section is determined to be a consonant section. This method is valid for all voiceless consonants.

Next, the method using vowel/nasal sound determination, which is the method of (2), will be explained.

In this embodiment, phoneme recognition for each frame is performed using LPC cepstral coefficients by comparison with a standard pattern of each phoneme prepared in advance. The standard pattern is 5 vowels (/a//
i/, /u/, /e/, /o/)/nasal (/N/
(represented by ) and voiceless consonants (represented by /s/) were used. In this way, the phoneme with the highest degree of similarity (first candidate phoneme) and the phoneme with the second highest degree of similarity (second candidate phoneme) are determined for each frame. A series in which the first candidate phoneme and second candidate phoneme for each frame are arranged in order of frame number is defined as a first candidate phoneme time series and a second candidate phoneme series. When the above phoneme sequence is viewed in order from the beginning of the word, if /N/ continues for a certain number of frames or more (for example, 4 frames or more) including the first candidate or second candidate phoneme sequence, this section is determined to be a consonant section.

This method is particularly effective for voiced consonants, mainly nasal sounds.

Next, a method based on a temporal change in power, which is the method of (2), will be explained.

When the beginning of a word is mainly a plosive consonant, the temporal change in power value is plotted as shown in Figure 2a. This is because the power rises rapidly due to its plosive nature, and the part where it crosses with the following vowel becomes concave like a.

b is the value obtained by differentiating the value of the temporal change in power of a. _P1 to _P3 indicate frame numbers of inflection points of a. Here, the frame number at which the voiced section begins is set to 1. Here, like a and b
The differential values of P ₁ and P ₃ are positive, the differential value of P ₂ is negative, and P ₃
<m (m is a threshold value indicating a frame number), the period from the beginning of the word to _P3 is determined as the initial consonant section.

Finally, the method using cepstral distance will be explained.

The cepstral distance is used by comparing the spectral pattern of a certain reference frame with the spectral pattern of each frame from the beginning of the word to the reference frame.

In this embodiment, LPC cepstral coefficients C ₁ to _Co (where n is a positive integer) are used as parameters representing the characteristics of the spectral pattern.
As a method of selecting a reference frame in which a spectrum appears stably, the m-th frame from the beginning of a word (m=7 in this embodiment) is fixed. This means that consonants with a relatively long duration (duration of m frames or more) are ~
This is because it can be detected using the following method.

Equation 1 is used as a method for comparing spectral patterns between two frames.

〓(i,j)= _o 〓 ^l=1 (C _l (i)−C _l (j)) ² ...Equation 1 In Equation 1, C _l (i) is the l-th LPC cepstral coefficient in the i-th frame from the beginning of the word. It represents. Similarly, C _l (j) is the l-th frame in the j-th frame.
It represents the LPC cepstral coefficients. 〓(i,
j) is larger, the spectrum of the two frames
The pattern will be different.

Using this equation 1, calculate 〓(i, m) (where 1≦i≦m-1) between the reference frame and each frame from the beginning of the word to the reference frame, and calculate the maximum value as 〓 _nax
shall be. The presence or absence of an initial consonant is determined depending on whether the value of 〓 _nax is larger or smaller than a certain threshold value.
When detected using this method, the consonant section is defined as the word-initial consonant section up to the frame in which the value of 〓(i, m) has the largest change. This method is effective for detecting consonants with short durations.

The initial consonants are segmented by the methods described above, and the consonants detected by the method are divided into unvoiced consonants (/z/, /h/, /
Consonants detected by the method of s/, /c/, /p/, /t/, /k/) are combined into voiced consonants (/
m/, /n/, /b/, /d/, /g/, /
The consonants detected using the method r/, /z/) are divided into consonant groups (/b/, /d/, /
g/, /z/, /p/, /t/, /k/) are combined into short-duration consonant groups (/m/, /n/, /b/, /d/). , /g/, /
r/, /z/, /h/, /p/, /t/, /
It is roughly classified into four consonant groups such as k/).

After narrowing down the candidates by major classification in this way, subclassification is performed within each consonant group. The method of subdivision is to find the degree of similarity with the standard phoneme pattern,
Consonants are determined by comparing the degree of similarity for each phoneme.

Voiceless plosives and voiced plosives are characterized by the transition from the plosive point to the following vowel. Therefore, in order to perform subclassification within a voiceless plosive group or a voiced plosive group, it is necessary to perform a similarity calculation that takes into account the temporal movement around the plosive point. Nasal sounds are characterized by the transition to a vowel, and it is necessary to calculate the degree of similarity by taking into account the temporal movement of this part. Ruon/
r/ is characterized by the spectral change and duration of the entire interval. /z/ is characterized by having a buzz part and a friction part following it.

As described above, although there are differences in the characteristic parts depending on each consonant group, it is common that the temporal movement based on the characteristic point is important information. The method for automatically detecting feature points is to use the word-initial frame, which is the beginning of a phoneme, for voiceless consonants, the frame that changes from nasal to vowel for voiced consonants, and the rising frame of power for consonants characterized by a change in power, and for consonants that are characterized by power changes. A consonant group with a short duration is considered to be the end of a phoneme. However, it is not easy to automatically detect feature frames accurately. Therefore, in order to reduce erroneous recognition due to automatic detection errors, the degree of similarity is calculated over several frames before and after the automatically detected feature frame, and the value of the frame with the maximum degree of similarity is taken as the degree of similarity for that phoneme.

Next, regarding the calculation of similarity, the following formula 2 or formula 3 Distance based on Bayesian judgment: P _i =1/(2π)d/2・|Σ|1/2・exp{−1/
2 (〓−〓 _i ) ^T・Σ ^-1・(〓−〓 _i )}...Formula 2 Mahalanobis distance: L _i = (〓−〓 _i ) ^T・Σ ^-1・(〓−〓 _i ) ...Formula 3 is used to calculate similarity considering temporal movement. That is, instead of the feature parameters of a single frame, the feature parameters of a plurality of frames (here, 1 frame) are used as data used for similarity calculation. Input feature parameters 〓=(x ₁ ⁽¹⁾ , x ₂ (1) …x _d ^{(1) ,} x 1 (2) , x ₂ ⁽ ₂ ⁾ …x _d ⁽²⁾ …
，
x ₁ (l), x ₂ (l), ...x _d (l)) Average value of standard pattern μ = (μ ₁ ⁽¹⁾ , μ ₂ ⁽¹⁾ , ...μ _d ⁽¹⁾ , μ ₁ ^{(2 )} , μ ₂ ⁽²⁾ ,
…μ _d ⁽²⁾ ,
..., μ ₁ (l), μ ₂ (l), ... μ _d (l)). Similarly, the covariance matrix Σ is assumed to have d×l dimensions (not described because it would be complicated). By using multiple frames of data in this way, it is possible to simultaneously compare the spectral characteristics of the parameters and the characteristics of their temporal fluctuations with the phoneme standard pattern.

Next, we will explain how to create a standard pattern. The standard pattern is created using a lot of data that is accurately extracted from the audio by visual inspection.

The phoneme standard pattern is created by cutting out 1 frames of data corresponding to the feature part from a lot of data of the same phoneme, finding a d×l dimensional feature vector, and then finding the average value and covariance matrix of a lot of data. Create one for each phoneme.

One standard pattern of surrounding information is created for each phoneme group. This is because surrounding information is common to each phoneme within a phoneme group.
The standard pattern of surrounding information is thus a standard pattern of universal surrounding information for that phoneme group. Figure 3 shows how to create it.
In the vicinity of the characteristic part (the shaded part in the figure), a section that is sufficiently long in time compared to the characteristic part is defined as the surrounding information section L.
Set as . For this intermediate, as shown in the figure, the feature parameters of l frame (d x l dimensions)
is extracted over the entire interval while shifting it one frame at a time. Such a procedure is applied to a lot of data belonging to the same phoneme group to obtain an average value vector and a covariance matrix, and this is used as a standard pattern of surrounding information. In this way, although the standard pattern of surrounding information includes data on the characteristic part, the data in the vicinity of the characteristic part has a much greater weight than that.

Next, we will describe a specific method for subdividing broadly classified data using the standard pattern created by the above method.

In the following explanation, for simplicity, we will use the distance measure of Equation 2, and one phoneme group will be divided into two phonemes (phoneme 1,
Let us consider the case consisting of phoneme 2). The idea remains the same even if the number of phonemes increases.

The feature section detects a feature frame using the method described above, and calculates a rough candidate section using that frame as a reference. This interval is defined as t ₁ to _{t 2} in terms of time. Now, the unknown input vector (data to be subclassified) at time t is 〓 _t (t = t ₁ ~ _{t 2} ) Standard pattern of phoneme 1 (average value) 〓 Standard pattern of ₁ phoneme 2 (average value) 〓 ₂ Let _e be the standard pattern (average value) of surrounding information, and let Σ be the covariance matrix common to all of phoneme 1, phoneme 2, and surrounding information. Σ is created by averaging each covariance matrix.

Letting the similarity (distance) of unknown input to phoneme 1 at time t be L ₁・t, L ₁・t=(〓 _t −〓 ₁ ) ^T・Σ ⁻¹・(〓 _t −〓 ₁ )−(
〓 _t −〓 _e ) ^T・Σ ^-1・(〓 _t −〓 _e )…Formula 4 Similarly, if the distance to phoneme 2 is L ₂・t, then L ₂・t=(〓 _t −〓 ₂ ) ^T・Σ ^-1・(〓 _t −〓 ₂ )−(
〓 _t −〓 _e ) ^T・Σ ⁻¹・(X _t −U _e )…Equation 5 is used. What these expressions mean is that the time t
The similarity between the unknown input and the phoneme standard pattern minus the similarity with respect to the surrounding information is set as the new similarity with the phoneme. and formula 4
The calculation of Equation 5 is performed for the period from t ₁ to t ₂ , and the phoneme that becomes the minimum during this period is set as the recognized phoneme among L ₁ ·t and L ₂ ·t.

In reality, Equations 4 and 5 can be expanded into simple equations as follows (the derivation is omitted).

L ₁・t=〓 ₁・〓 _t −B ₁ equation 4′ L ₂・t=〓 ₂・〓 _t −B ₂ equation 5′ 〓 ₁ , 〓 ₂ , B ₁ , B ₂ are standard including surrounding information It's a pattern.

The meaning of the above method will be conceptually explained with reference to FIG.

Consider the case where consonant discrimination is performed in a situation where the phoneme section is shown in FIG. 4a. For the true characteristic part (hatched part) of this consonant, the characteristic part candidate section T is time
It is assumed that the values are obtained as t ₁ to t ₂ . b was determined by Equation 3. Phoneme 1 (solid line), Phoneme 2 (diagonal line)
This figure shows temporal fluctuations in the degree of similarity. A, B, and C indicate positions where the degree of similarity is minimal. In the true characteristic portion (point B), phoneme 1 is smaller than phoneme 2, and this consonant should be determined as phoneme 1. However, within the feature candidate section automatically determined using the segmentation parameters, phoneme 2 is at its minimum at point A, so if this continues, it will be misclassified as phoneme 2. FIG. 6c shows a standard pattern of surrounding information of unknown input and distances, and the value increases near the true feature. This is because the standard pattern is created mainly using peripheral information. FIG. 6 d is the distance to the phoneme standard pattern containing surrounding information, and is equivalent to b minus c. At point d, the value at point B is smaller than at point A, and this consonant is correctly determined as phoneme 1.

As described above, by using the method of this embodiment, it is possible to automatically extract true features accurately and discriminate phonemes from the rough feature candidate sections determined by the segmentation parameters. can.

In addition, although the Mahalanobis distance based on Formula 3 was explained above, a similar method can be used for other distances.

Further, although the above explanation has been made using consonants, a similar method can also be applied to phonemes that change over time, such as semi-vowels.

In this way, the method of narrowing down the number of candidates through major classification and discriminating phonemes using a statistical distance scale that takes into account temporal movement based on automatically extracted feature parts for subclassification is a method that uses phonemes (especially This is a rational recognition method that utilizes the phonetic properties of consonants and semi-vowels.

In this example, all initial consonants (/
p/, /t/, /k/, /c/, /b/, /
d/, /g/, /m/, /n/, /r/, /
z/, /s/, /h/), on average about
A recognition rate of 70.3% was obtained. The data uses 212 single sets uttered by 20 men and women, and is sufficiently reliable. Furthermore, considering that in the conventional method, subdivision within a consonant group is not performed, it can be seen that the effects of the embodiment according to the present invention are large.

Effects of the Invention To summarize above, the present invention performs segmentation of the initial consonant of an input word by using four methods in combination, and determines the initial consonant depending on which of these four methods is used for segmentation. It is recognized as a plurality of phoneme groups, such as voiceless consonants, voiced consonants, consonants with power changes, and consonants with short duration. automatically detects
The present invention provides a word-initial consonant recognition method characterized in that a phoneme is determined by calculating the degree of similarity between the characteristic part and a standard pattern of each phoneme belonging to a previously recognized phoneme group, Automatic segmentation of word-initial consonants can be performed to recognize phonemes with high accuracy.

(b) It is possible to automatically and accurately extract parts (feature parts) that are effective for phoneme discrimination and perform matching.

C. Subclassification within voiced plosives, voiceless plosives, and nasals, which have been difficult to distinguish in the past, can be performed in combination with automatic segmentation.

D. Using the results of the four initial consonant segmentation methods, it is possible to simplify the algorithm for major consonant classification.

There are advantages such as

[Brief explanation of drawings]

FIG. 1 is a functional block diagram of a conventional speech recognition system, and FIG. 2 is an explanatory diagram of a method for detecting a word-initial consonant by changing power in an embodiment of the present invention.
FIG. 3 is a diagram illustrating a method for creating a surrounding information standard pattern according to the same embodiment, and FIG. 4 is a diagram illustrating a method for detecting characteristic parts and discriminating phonemes according to the same embodiment. 1... Acoustic analysis section, 2... Feature extraction section, 3...
Segmentation section, 4... Phoneme discrimination section, 5...
...Standard pattern registration section, 6...Word recognition section, 7...
...word dictionary.

Claims (1)

[Claims] 1. First method for detecting word-initial consonants by voiceless determination
A second method of detecting the initial consonant by vowel nasality determination, a third method of detecting the initial consonant by capturing temporal changes in power, and a fourth method of detecting the initial consonant by cepstral distance. By segmenting the input speech by applying these in any order, the initial consonants are divided into four phonemes: unvoiced consonants, voiced consonants, consonants characterized by power changes, and consonants with short duration. A characteristic part (a part effective for phoneme discrimination) is automatically extracted within the phoneme interval, and a standard pattern of each phoneme belonging to the previously recognized phoneme group is compared to the characteristic part. A phoneme recognition method characterized by determining phonemes by calculating similarity. 2. First, segmentation parameters are used to determine candidate sections for feature parts, and then feature extraction and The phoneme recognition method according to claim 1, characterized in that phoneme discrimination is performed. 3. The phoneme recognition method according to claim 1, wherein the standard pattern similarity calculation is performed using a statistical distance measure and a standard pattern that includes temporal movement of phonemes.