JPH0120440B2 - - Google Patents

Description

[Detailed description of the invention]

INDUSTRIAL APPLICATION FIELD The present invention relates to a speech segmentation method (a method for extracting phoneme intervals in continuously uttered speech) in speech recognition. Conventional configurations and their problems Automatic speech recognition devices that automatically recognize speech uttered by humans are considered to be very effective as a means for providing data and instructions from humans to computers and various machines. The pattern matching method is often adopted as the operating principle of automatic speech recognition systems that have been researched or published in the past. This method memorizes standard patterns for all types of words that need to be recognized in advance, and compares them with unknown input patterns to calculate the degree of matching (hereinafter referred to as similarity). The word is calculated and determined to be the same word as the standard pattern that yields the maximum match. In this pattern matching method, standard patterns must be prepared for all words to be recognized, so if the speaker changes, a new standard pattern must be input and stored. Therefore, in cases where hundreds of types of words are to be recognized, such as the names of cities across Japan, it takes a huge amount of time and effort to pronounce and register all types of words. It is expected that the memory will easily become huge. Furthermore, there is a drawback that the time required for pattern matching between the input pattern and the standard pattern increases as the number of words increases. On the other hand, input speech is divided into phoneme units and recognized as combinations of phonemes (hereinafter referred to as phoneme recognition).
The method of determining similarity with a word dictionary written in phoneme units has the advantage that the memory capacity required for the word dictionary is significantly reduced, the time required for pattern matching is shortened, and the contents of the dictionary can be easily changed. I have it. An example of this method is described in ``Word speech recognition system using the outline form of the speech spectrum and its dynamic characteristics'' by Miwa et al., Journal of the Acoustical Society of Japan 34 (1978). A block diagram of a word recognition system using this method is shown in FIG. First, the voices of multiple speakers are analyzed in advance by the acoustic analysis unit 1 using a filter bank for each 10ms analysis interval, and the feature parameters are determined by the feature extraction unit 2 based on the obtained spectrum information. . From this feature parameter |a
A standard pattern is created for each phoneme or phoneme group represented by vowels such as |, |o|, and consonants such as |n|, |b|, and is registered in the standard pattern registration section 5. Next, the input speech of the unequal fixed speaker is similarly analyzed by the acoustic analysis unit 1 for each analysis section,
Feature parameters are determined by the feature extractor 2.
Using these characteristic parameters and the standard pattern of the standard pattern registration section 5, the segmentation section 3 performs a separation operation between vowels and consonants (hereinafter referred to as segmentation). Based on this result,
The phoneme discriminator 4 compares the phoneme with the standard pattern to determine the phoneme that corresponds to the standard pattern with the highest degree of similarity as the phoneme in that section. Finally, the time series of phonemes created as a result (hereinafter referred to as the phoneme series) is sent to the word recognition unit 6, and the word corresponding to the item with the highest similarity to the word dictionary 7 similarly expressed in the phoneme series is recognized. Output as result. As can be seen from the overall operation described above, if the segmentation unit 3 makes a mistake in segmentation, a phoneme that should be present may be overlooked (dropped phoneme), or a different phoneme may be inserted where there is actually no phoneme. This means that phonemes will be added (phoneme addition). When these errors occur, when a word is expressed as a phoneme sequence, the omission or addition of a phoneme causes it to resemble another completely unrelated word, increasing the risk of misrecognition. . As described above, segmentation is the most important task in a method of word recognition based on phoneme recognition, and the performance of a word recognition system is greatly influenced by the accuracy of segmentation. By the way, the conventional segmentation method uses power information of the entire voice band as a parameter for segmentation, calculates the power dip from its temporal movement, and determines the dip interval to be a consonant. There is also a method of using spectral slope instead of full-band power information, or a method of using both in combination. (For example, "Word speech recognition system using the outline form of the speech spectrum and its dynamic characteristics" Miwa et al., Journal of the Acoustical Society of Japan 34, 1978) All of these methods use parameter dips, and the following is There was a problem. (a) There are consonants that cannot be detected using the full-band power or spectrum slope, and there are many consonants added to vowels and other intervals. (b) There are consonants that cannot be detected only by the temporal movement of parameters, and these are dropped. OBJECTS OF THE INVENTION It is an object of the present invention to provide a speech segmentation method that eliminates the above-mentioned conventional drawbacks and can perform segmentation of a wide range of consonants from voiced consonants to voiceless consonants with high accuracy. . Structure of the Invention In Japanese, words are essentially composed of alternating combinations of vowels and consonants. Consonants other than phlegmatic consonants and other consonants do not occur consecutively. Therefore,
When recognizing Japanese, accurately separating vowels and consonants greatly contributes to improving speech recognition rates. The present invention efficiently combines the information in (1) shown below with the information in (2) to (3) to accurately separate consonant intervals in a word from other intervals such as vowels. be. (1) Power dip information generated by the temporal movement of low-frequency power and high-frequency power of an audio signal. (2) Results of voiced/unvoiced judgment for each frame (one frame is 10 msec). (3) Results of frame-by-frame phoneme recognition. In the present invention, to solve the drawback a of the conventional example,
This is solved by effectively using the low-frequency power and high-frequency power information as parameters.In addition to the temporal movement of the parameters, we can also solve the problem by effectively using the low-frequency power and high-frequency power information as parameters. This problem is solved by using the phoneme recognition results for each frame. DESCRIPTION OF EMBODIMENTS An embodiment of the present invention will be described. In this embodiment, low frequency and high frequency power information,
A highly accurate segmentation method is achieved by using both the phoneme recognition results for each frame and the voiced presence/absence determination results. Voiced/unvoiced determination is effective for phonemes that are highly voiceless and have a relatively long duration, such as voiceless fricatives. The low-frequency power dip is effective for phonemes that have a short duration and are strongly voiceless (such as voiceless plosives). High-frequency power dips are effective for phonemes with short duration and strong voicing (voiced plosives, flowing sounds, etc.). Also,
The frame-by-frame recognition results are effective for nasal sounds in which power dips are difficult to appear and voiced consonants with long durations. In this way, these pieces of information have complementary properties, and by using them in combination, most consonants in Japanese can be detected with high accuracy. The method according to this embodiment will be explained in detail below. First, a first consonant interval detection method using low-frequency and high-frequency power information will be described. In this embodiment, the low-frequency power and high-frequency power of the audio spectrum are used together as segmentation parameters. The former is effective for distinguishing between vowels and voiceless consonants, and the latter is effective for discriminating between vowels and voiced consonants. Low frequency power is 250
Pass it through a ~600Hz band filter and rectify it frame by frame. Also, the high frequency power is 1500 ~
Similarly obtained with a 4000Hz bandpass filter. FIG. 2 shows a method for extracting dips from low frequency or high frequency power information. A is a time series plot of the rectified output of the filter, which is affected by many small dips in addition to the large dips in the consonant section. Since the latter is an unnecessary dip, it is removed by smoothing (Fig. 2b). Then b
The signal of c is obtained by differentiating the signal of c.
Then, from the signal of c, the magnitude p between the maximum value and the minimum value
Then, the time length (number of frames) L from the minimum value to the maximum value is determined. The conditions p>p _nio and L<L _nax are applied, and for dips that satisfy the conditions, the section from the minimum value to the maximum value at c is defined as a dip section (consonant candidate). This method replaces calculation of the size of the power dip with detection of the rate of change in power, and its maximum value,
By calculating the minimum value, the dip section can be detected easily and with high accuracy. Next, a method for specifying a consonant section from consonant candidates detected by one or both of the low-frequency power dip and the high-frequency power dip will be described. Let pl be the magnitude of the dip obtained from the low-frequency power information using the method described above, and let _ph be the magnitude of the dip obtained from the high-frequency power information. When the consonant candidate section based on the low-frequency information and the consonant candidate section based on the high-frequency information overlap, the two-dimensional coordinates (p _l , p _h ) are applied to the discriminant diagram shown in FIG. 3. If (p _l , p _h ) is located in the additional section (inside the diagonal line) on the discriminant map, that consonant candidate is rejected. If (p _l , p _h ) is located in the consonant interval,
A portion corresponding to the logical sum of the low-frequency power dip section and the high-frequency power dip section is specified as a consonant. If there is no overlap between the consonant candidate sections based on the low-frequency and high-frequency information, one is set as o (for example, (p _l , o)) and applied to the discriminant map. Using the low-frequency power information and high-frequency power information that have complementary properties as parameters, search for consonant candidate intervals using each of them, and then determine the consonant interval by applying them to a discriminant map. Compared to conventional methods, this method is effective for a wide range of consonants, from voiced to unvoiced, and can detect consonant intervals with high accuracy. Especially the voiced consonant |b
It is valid for |, |d|, |η|, |r|, the vocal consonant |h|, and |z|, which exhibit both voiced and unvoiced properties. Next, a second consonant interval detection method using phoneme recognition results for each frame will be described. The above-mentioned segmentation method using dip information has a detection rate of about 73% for nasal sections, which is not sufficient compared to other voiced consonants. Another drawback is that the duration of a plucked sound is too long, so depth information cannot be used. In this embodiment, the above-mentioned weaknesses are covered by using phoneme recognition results for each frame. In this embodiment, phoneme recognition is first performed for each frame, and frames recognized as the same phoneme are combined to obtain the phoneme recognition result for that section. Various methods can be considered for phoneme recognition for each frame, but in this example, the parameter
The LPC cepstrum coefficients C _i (i=1 to d) obtained by LPC analysis (using the autocorrelation method) are used in the following manner. Assuming that a standard pattern for phoneme k is an average value μ _k and a covariance matrix Σ _k , the probability P _k that a certain frame is phoneme k can be obtained by the following equation. (The subscript T represents the transpose, and the subscript -1 represents the inverse matrix.) The log likelihood L _k is L _k = -1/2 (C-μ _k ) ^T・Σ _k ^-1・(C- μ _k ) − A _k (Formula 2) However

[Formula] The standard pattern is created in advance using a large amount of data in which phonemes are labeled visually. Five vowels |a|, |i|, |u as a standard pattern
Prepare those for |, |e|, |o|, and |N|. |N| represents the nasal group, |m|,
It is a collection of |n| and pakuon. (Formula 2) is applied to all frames of the speech section, and the phoneme with the largest likelihood and the phoneme with the second highest likelihood are determined for each frame. This is the result of frame-by-frame phoneme recognition for vowels and nasals. If we apply the five vowel and nasal patterns to all frames in this way, each frame in the section corresponding to nasal sounds |m|, |n|, and pellic sounds will be recognized as nasal sounds |N|, and other spectral patterns will be similar to nasal sounds. phonemes (|b|, |d|, |η|, |r|)
also has a high probability of being recognized as |N|. Therefore, by referring to the section recognized as |N|, voiced consonants can be detected even in sections where dips do not exist. In this example, |N|
A consonant interval is defined as a consonant interval in which five or more consecutive frames including the frame with the second highest likelihood are recognized. FIG. 4 shows an example of the recognition results for each frame for phonemes with the first and second likelihoods. In this example, the first to fifth frames are |a|, and the sixth to fifth frames are
The 10th frame is |o|, and the 17th and subsequent frames are |u|
Then, the phoneme is determined. The 11th to 16th frames are segmented as a consonant section because N is 6 consecutive frames including the second likelihood. A standard consonant pattern is applied to the consonant section, and phonemes are determined later. The segmentation method based on looking at the continuity of frames recognized as nasal sounds described above is
This is valid for |m|, |n|, 框音, |b|, |d|, and |η|. Next, a third consonant interval detection method using voiced/unvoiced determination results will be described. The unvoiced consonants |s|, |c|, |h|, and |z|, which have long durations, have a duration longer than L _nax , and dips may not be detected. In this case, segmentation can be performed based on the temporal continuity of voiced/unvoiced determination results for each frame. Voiced/unvoiced determination may be performed using any of the methods that utilize zero-crossing waves, the slope of the spectrum, the value of the first-order autocorrelation coefficient, and the like. In this embodiment, highly accurate determination is made by preparing voiced and unvoiced standard patterns and applying equation (2). In this embodiment, segments in which unvoiced segments continue for seven or more consecutive frames are segmented as consonant segments. Next, an application example of the method for detecting the first to third consonant sections described above will be described. Various combinations of detection methods for the first to third consonant intervals are possible, but the first consonant interval detection method uses low-frequency and high-frequency power information, and the phoneme recognition results for each frame are used. It is desirable to combine one or both of the second consonant interval detection method and the third consonant interval detection method using the voiced/unvoiced determination results. Here, an example will be shown in which the third, first, and second consonant interval detection methods are applied in this order. The applicable law is as shown below. (i) First, apply the third rule (denoted as ) to the voiced section, and define a section in which the unvoiced section continues for 7 or more frames as a consonant section. (ii) Apply the first rule (
) is applied to find the consonant interval by dip. (iii) Second rule for voiced sections (written as)
is applied, and a section in which the section recognized as |N| continues for five or more frames is defined as a consonant section. (iv) All the intervals found in (i) to (iii) above are considered consonant intervals. However, if overlapping sections are determined by the rules (i) and (ii) or (ii) and (iii), priority will be given to the section determined by the dip, as a general rule. This example was evaluated using 212 words uttered by 10 men and 10 men. This word set is data for evaluation in which consonant sections are labeled in advance by visual inspection. Table 1 shows the evaluation results (addition rate 4.7%) for each phoneme.

[Table] The table shows the number of phonemes and the recognition rate at each stage when the rules are applied in the order of . The right column shows the final recognition rate, and it can be seen that a high segmentation rate of over 90% was obtained for each phoneme. Looking at them individually, |s| and |
The rule is valid for a part of h|, |z|, and the rule is valid for a part of the pixel sounds, |m|, |n|. The rules are valid for other phonemes. For most phonemes, the combination of the three rules improves the recognition rate at each stage. This result demonstrates the effectiveness of using the three rules together. Compared to the conventional example, this embodiment is characterized in that segmentation of a wide range of consonants from voiced consonants to voiceless consonants can be performed with high precision. For example, in the conventional example, the segmentation rate for nasal sounds was very low, whereas in this example, the segmentation rate for nasal sounds was very low.
More than 90% results were obtained, |r|, |η|, |h|,
It also improves by more than a few percent compared to |z|. Improvement rates of around 1% were obtained for all other phonemes. Note that in this example, all information is used,
I have given an example of applying , , in the order, but you can also get pretty good results using only and or and. Furthermore, the application order is not fixed. That is, one or both of or can be combined in any order. Effects of the Invention As described above, the present invention detects power dips caused by temporal fluctuations in the low-frequency power and high-frequency power of the speech spectrum, and uses the magnitude of each power dip in combination to detect consonant sounds. The first detection method for detecting sections is to detect all frames (one frame length is e.g.
10 msec of data) is recognized as a vowel or a nasal sound, and when a certain number or more frames recognized as a nasal sound are consecutive, the second detection method detects that interval as a consonant interval, and for the entire speech interval, All frames are judged as voiced or unvoiced, and when a certain number or more of consecutive unvoiced frames occur, the consonant interval is detected by combining at least one of the third detection method, which detects the interval as a consonant. This detection allows segmentation of a wide range of consonants, from voiced consonants to voiceless consonants, with high accuracy.

[Brief explanation of drawings]

Fig. 1 is a block diagram of a conventional speech recognition system, Fig. 2 is a diagram explaining the method of detecting a power dip from low-frequency power information or high-frequency power information according to the present invention, and Fig. 3 is a diagram showing a method for detecting a power dip from low-frequency power information or high-frequency power information. A discrimination diagram for distinguishing consonant intervals and addition of consonants according to the size of the power dip. Figure 4 is a diagram explaining a method for recognizing all frames as vowels or nasals and detecting consonant intervals from this result. It is.

Claims (1)

[Claims] 1. Detecting power dips caused by temporal fluctuations in the low-frequency power and high-frequency power of the speech spectrum, and detecting consonant intervals by using the magnitude of each power dip in combination. A speech segmentation method characterized in that a consonant interval detection method is combined with another detection method to detect a consonant interval. 2. Another detection method is to recognize all frames of an entire speech interval as vowels or nasals, and when a certain number or more of consecutive frames recognized as nasals occur, that interval is detected as a consonant interval. A speech segmentation method according to claim 1. 3. Claims that another detection method is one in which a voiced/unvoiced determination is made for all frames of an entire speech interval, and when a certain number or more of consecutive unvoiced frames occur, that interval is detected as a consonant. The speech segmentation method described in paragraph 1. 4. Consonant section detection, in which another detection method recognizes all frames of a whole speech section as vowels or nasals, and detects that section as a consonant section when a certain number or more consecutive frames are recognized as nasals. By combining this method and a consonant section detection method that performs voiced/unvoiced judgment on all frames of all speech sections and detects the section as a consonant when a certain number or more of consecutive unvoiced frames occur, the consonant section detection method detects the section as a consonant. The speech segmentation method according to claim 1, which detects intervals.