JPH0682275B2 - Voice recognizer - Google Patents

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a speech recognition apparatus, and more particularly to segmentation of input speech into phoneme groups, and this segmentation method and a phoneme identification neural network applied thereto. The present invention relates to a speech recognition apparatus that recognizes phonemes by merging with.

[Problems to be Solved by Conventional Techniques and Inventions] Conventional speech recognition methods include a method of performing phoneme recognition after segmenting a continuous speech waveform with time divisions, and a method of performing continuous speech waveform timing. A so-called phonological spotting method has been proposed in which phonological recognition of a segment and its part is performed simultaneously.

However, in the former method, segmentation is performed with a combination of simple parameters such as uniform power and spectrum change regardless of the phoneme environment in which each phoneme exists. Can't get Furthermore, the segmentation method is
There is no proposal to improve the phoneme recognition rate by using the phoneme groups obtained by the segmentation method to determine the final phoneme recognition result, which is used only for identifying phoneme intervals. Also, the latter method has a drawback in that there are many phoneme misrecognitions and insertion errors near the boundaries of consecutive phonemes, and as a result, a high phoneme recognition rate cannot be obtained.

Therefore, the main object of the present invention is to solve phonological misrecognition due to segmentation error and phonological misrecognition and insertion error at phonological boundaries by phonological spotting method, and further to use the phonological groups obtained by segmentation method to make a final decision. It is an object of the present invention to provide a speech recognition device capable of determining a phoneme recognition result and performing high phoneme recognition.

[Means for Solving the Problem] The invention according to claim 1 is a voice recognition device for recognizing an input voice, the magnitude of the power of the input voice in a certain frequency band, and the power in the certain frequency band. Based on the acoustic characteristics of the amount of change in the spectrum, the amount of change in the spectrum in a certain frequency band, and the ratio of the power in a certain frequency band to another frequency band, the input speech is divided into sections and confidence factors for each phonological group. Segmentation means for determining and performing segmentation, each provided corresponding to the phoneme group, a plurality of time-delayed neural networks for recognizing the phonemes of the phoneme group given the maximum certainty factor among the segmented phoneme groups, Interval of phonological group determined and certainty factor and time delay neural network The recognition means for recognizing a voice based on the phoneme recognized by.

The invention according to claim 2 is the same segmentation means as in claim 1, a time-delay neural network for recognizing the phonemes of a segmented phoneme group,
Function deciding means for deciding a function representing the validity of the decided phonological group and the recognition result of the time-delay neural network, and speech recognition based on the decided phonological group section, the certainty factor and the decided function. And a recognition means for doing so.

The invention according to claim 3 comprises the segmentation means of claim 1 and a plurality of time-delayed neural networks, and determines a function representing the validity of the phoneme group determined by the segmentation means and the recognition result of the time-delayed neural network. Function determining means, and a recognizing means for recognizing speech based on the determined phoneme group function and certainty factor and function.

[Operation] The speech recognition apparatus according to the present invention performs segmentation by determining an interval and a certainty factor of an input speech for each phoneme group and segmenting the phoneme of the phoneme group given the maximum certainty factor into a time-delayed neural network. The speech is recognized based on the section of the phoneme group that is recognized and determined by the certainty factor and the phoneme recognized by the time delay neural network.

[Embodiment of the Invention] FIG. 1 is a schematic block diagram of a voice recognition apparatus to which the present invention is applied. Referring to FIG. 1, the voice recognition device includes an amplifier 1, a low pass filter 2, an A / D converter 3 and a processing device 4.
Including and The amplifier 1 amplifies the input audio signal, and the low-pass filter 2 removes aliasing noise from the amplified audio signal. The A / D converter 3 samples the audio signal and converts it into a digital signal. The processing device 4 includes a computer 5, a magnetic disk 6, terminals 7, and a printer 8. The computer 5 performs voice recognition based on the sampled voice digital signal input from the A / D converter 3 by using a method shown in FIGS. 2 to 5 described later.

2 to 5 are diagrams showing various methods of recognizing speech by identifying phonemes of the present invention.

First, a common configuration in each method shown in FIGS. 2 to 5 will be described. Each of the systems shown in FIGS. 2 to 5 is composed of three parts, each of which comprises a phoneme segmentation section, a phoneme identification section, and a phoneme determination section. A detailed description thereof is given in a patent application (Japanese Patent Application No. 1-61928) previously filed by the inventor of the present application, and will be briefly described here. The phonological segmentation unit performs rule-based detection of phonological candidates by using global acoustic features on the spectrogram for each phonological class to detect a rough position where a phonological unit can exist. The phonological class here is, for example, an unvoiced fricative or a voiced fricative.

Next, a hypothesis of the phonological environment is made. That is, for each detected phoneme candidate, a phoneme type is hypothesized before and after each. Next, phonological boundary detection and hypothesis verification are performed under the phonological environment hypothesis. Under the correct hypothesis,
A high degree of certainty is obtained for each hypothesis, and as a result, the phonological environment is detected. On the other hand, the wrong hypothesis leads to lower confidence.
It does not reach the phonological environment. Whether or not the hypothesis is correct is determined by the acoustic features on the spectrogram, that is, the magnitude of power in a certain frequency band of the input voice, the amount of change in power, the amount of change in spectrum, and the power for other frequency bands. Judgment is made based on acoustic characteristics such as the ratio. Next, the phoneme boundary that gives the maximum certainty factor for each hypothesized phoneme class is set as the segmentation result, and the start and end of the phoneme and the phoneme class are output with certainty factor.

FIG. 6 is a diagram showing an example of a time delay neural network (TDNN) for identifying phonemes. Next, the sixth
A method of identifying the phoneme of the segmentation detected as described above will be described with reference to the drawings. Sixth
The time-delayed neural network shown in the figure groups 18 consonants into 6 classes: voiced plosives, unvoiced plosives, nasal sounds, voiced fricatives, unvoiced fricatives, and stream sounds, and each group is input to the input layer 11. It The input layer 11 is learned from the conventionally known back propagation,
Identify segmented phonemes. The identification of each class is performed by the middle layer 12. In this embodiment, the learning of the time-delayed neural network is performed by adjusting the end positions of all the consonants to the positions 2/3 from the front of the input layer 11. Similarly, in the phoneme identification, the end of the segmentation result is applied to the same position of the input layer 11, and the phoneme giving the maximum certainty degree output by the output layer 13 of the time delay neural network is used as the identification result.

The phoneme determination unit shown in FIG. 2 to FIG. 5 uses the segmentation result for each phoneme class and the phoneme identification result output by the time-delay neural network applied to that section to determine the phoneme that gives the maximum certainty factor and its phoneme. The section is determined.

The system shown in FIG. 2 identifies a phoneme by a combination of the simplest segmentation method and phoneme identification method to recognize a voice. After the input voice is analyzed and feature extraction is performed, the segmentation unit determines that the confidence of unvoiced fricatives is 0.62 and the confidence of voiced fricatives is 0.51, for example. And
An unvoiced fricative with a high degree of certainty is selected, and this unvoiced fricative is input to the time-delayed neural network shown in FIG. 6, and phonological discrimination is performed using the method disclosed in Japanese Patent Application No. 1-61928. The phoneme recognition is performed.

In the example shown in FIG. 3, the phoneme is identified by the means using the segmentation method for narrowing down the phoneme group, and the voice is recognized. In this example, the input speech is analyzed, and as a result of feature extraction, the result that gives the maximum certainty factor in the segmentation unit is determined, and voiced consonant identification is performed depending on whether the phoneme group is a voiced sound group or an unvoiced sound group. A time delay network or unvoiced consonant recognition time delay neural network is selectively applied to identify phonemes within the interval.

In general, the smaller the number of distinct phonemes, the better the discrimination ability of the time-delayed neural network. Therefore, if there is no confusion among the phoneme classes of the segmentation results, it is better to use a time-delayed neural network that performs phoneme discrimination for each class. It is expected that the identification rate will be improved. That is, the phoneme class is narrowed down by the segmentation unit, and the phoneme identification within the class is performed.

FIG. 7 is a diagram showing an example of the time-delayed neural network for voiced consonant identification and the time-delayed neural network for unvoiced consonant identification described in FIG. The unvoiced consonant discrimination neural network shown in FIG. 7 (a) is for discriminating eight unvoiced consonants (p, t, k, ch, ts, s, sh, h). 22 and an output layer 23. In addition, the time-delayed neural network for voiced consonant identification shown in FIG. 7 (b) is used for voiced 7 consonants (b, d, g, m, n, r, z).
Input layer 31, intermediate layer 32 and output layer 33.
Including and

The example shown in FIG. 4 identifies a phoneme by using a phoneme group of the segmentation method and a function representing the validity of the result of the phoneme identification method to recognize a voice, and in FIG. 2 and FIG. Similar to the embodiment described, the segmentation unit determines the certainty factors of unvoiced fricatives and voiced fricatives, and then uses the time-delayed neural network shown in FIG. Phonological recognition is performed. That is, in the example shown in FIG. 4, phoneme segment candidates and their phoneme groups are output, and the validity of the phoneme class of the time delay neural network and the phoneme class of the segmentation result can be taken into consideration. It is expected that both the ability of segmentation and phoneme identification will be improved.

Here, as an example of a function representing the validity, the maximum confidence factor (Ce
This is shown as a method of using the phoneme that gives the rtainty factor) as the recognition result.

CFrec = combine (CFseg, CFnn) (1) CFnn = k · Wnn · f (arg (seg), arg (nn))
(2) where CFrec: confidence of final phoneme recognition CFseg: confidence of segmentation result CFnn: confidence of phoneme identification result Wnn: output value of identification phoneme of time-delayed neural network arg (seg): phoneme of segmentation result Class arg (nn): time-delayed neural network identification phoneme k: coefficient (time-delayed neural network reliability, k
The larger is, the more reliable the output result of the time delay neural network. ) F (): a function indicating the validity of the identified phoneme and the phoneme class.
It gives 1.0 if the identification phoneme of the time-delay neural network belongs to the phoneme class of the segmentation result, 1.0 if it does not, and 0.5 if the voiced / unvoiced sounds match.

combine (): MYCIN confidence calculation model In the example shown in Fig. 5, the phoneme identification means is selected by the means using the segmentation method to narrow down the phoneme groups, and the result of the phoneme group and phoneme identification method of the segmentation method is selected. The phoneme is identified and the voice is recognized by using the function representing the validity of.

FIG. 8 is a table showing the result of consonant recognition by each method of the present invention. 18 consonant recognition time delay neural
When a network and two time-delayed neural networks of voiced / unvoiced sound are used, with and without consideration of the validity of the identification phoneme of the time-delayed neural network and the phoneme class of the segmentation result,
Further, when considering the validity, an experiment was carried out by changing the conditions such as how much time delay neural network output results should be trusted. In Figure 8, 18-CONS-TD
NN shows the case of using the 18 consonant discrimination time delay neural network, V / UV-TDNN shows the case of using two time delay neural networks of voiced sound / unvoiced sound, and NO COMB is the time delay neural network. The case in which the validity of the identification phoneme and the phoneme class of the segmentation result is not considered is shown, and with COMB is taken into consideration.

The degree of dependence of the equations (1) and (2) on the time delay neural network is k = 0.4,0.
Two values of 8 were used. The larger k is, the more reliable the output result of the time delay neural network is. Recognition Rate indicates the case where both phoneme segmentation and phoneme identification are performed correctly.
Error Rate indicates the additional error rate, and Segmentation Rat
e indicates the rate at which the start / end boundary error of the phoneme was detected within 50 msec and was judged to be correctly segmented.
Boundary Alignment Error indicates the deviation of the correctly detected boundary with respect to the inspection label.
The “mentation rate” indicates the phoneme identification rate in the segment that is correctly segmented according to the present invention. The table shown in FIG. 8 shows the effectiveness of the method of applying the time-delay neural network after narrowing down the phoneme class, and the validity of the identification phoneme of the time-delay neural network and the phoneme class of the segmentation result. It demonstrates the effectiveness of the method considered.

Note that the phoneme groups can be narrowed down not only by dividing voiced sounds / unvoiced sounds, but also by separating frictional sounds, nasal sounds, plosive sounds, and the like. do it.

Further, although the time-delay neural network is used in the phoneme identification method of the above-described embodiment, a phoneme identification method within a phoneme group by another general statistical method may be used. For example, phoneme identification methods using general neural networks, phoneme identification methods using HMM, phoneme identification methods using Bayesian rules, phoneme identification methods using linear discrimination, and phonemes using standard patterns designed by methods such as LVQ. An identification method or the like can be applied.

[Effects of the Invention] As described above, according to the present invention, the segment and the certainty factor of the input speech are determined for each phoneme group, segmentation is performed, and the phoneme of the phoneme group given the maximum certainty factor is recognized. Then, the speech can be recognized based on the determined section of the phoneme group and the phoneme recognized by the certainty factor and the time-delay neural network, which enables high-performance phoneme recognition.

[Brief description of drawings]

FIG. 1 is a schematic block diagram of an entire voice recognition device to which an embodiment of the present invention is applied. FIG. 2 is a diagram showing an example in which a phoneme is identified and a voice is recognized by a combination of the simplest segmentation method and phoneme identification method in an embodiment of the present invention. FIG. 3 is a diagram showing an example of recognizing a voice by identifying a phoneme by means using a segmentation method for narrowing down a phoneme group. FIG. 4 is a diagram showing an example in which a phoneme is identified and a voice is recognized by using a phoneme group of the segmentation method and a function representing the validity of the result of the phoneme identification method. FIG. 5 shows that phonemes are identified by the means using the segmentation method for narrowing down the phoneme groups, and the phonemes are identified by using the function indicating the validity of the result of the phoneme group and the phoneme identification method of the segmentation method. It is a figure which shows an example to recognize. FIG. 6 is a diagram showing an example of the time-delayed neural network for identifying 18 consonants used in FIGS. 2 and 4. FIG. 7 is a view showing an example of a time delay neural network for distinguishing voiced sounds / unvoiced sounds used in the embodiment of FIGS. 3 and 5. FIG. 8 is a table showing the result of phoneme recognition by each method of the present invention. In the figure, 1 is an amplifier, 2 is a low-pass filter, and 3 is
A / D converter, 4 processing unit, 5 computer, 6 magnetic disk, 7 terminals, 8 printer, 11,21,31 input layer, 12,22,32 intermediate layer, 13,23 Reference numerals 33 denote output layers.

Claims (3)

[Claims] 1. A voice recognition device for recognizing an input voice, the magnitude of the power of the input voice in a certain frequency band, and the amount of change in the power in the certain frequency band,
Based on the acoustic characteristic of the amount of change in spectrum in a certain frequency band and the power ratio in a certain frequency band and another certain frequency band, the segment and the confidence level of the input speech are determined for each phonological group. A plurality of time-delay neural networks for recognizing the phonemes of the phoneme group to which maximum confidence is given among the phoneme groups segmented by the segmentation means, and A speech recognition apparatus, comprising: a recognition unit for recognizing a voice based on a phoneme group section determined by the segmentation unit, a certainty factor, and a phoneme recognized by the time delay neural network. 2. A voice recognition device for recognizing input voice, the magnitude of power in a certain frequency band of the input voice, and the amount of change in power in a certain frequency band,
Based on the acoustic characteristic of the amount of change in spectrum in a certain frequency band and the power ratio in a certain frequency band and another certain frequency band, the segment and the confidence level of the input speech are determined for each phonological group. A time-delayed neural network that recognizes the phonemes of the phoneme group segmented by the segmentation means, and a function that represents the validity of the phoneme group determined by the segmentation means and the recognition result of the time-delayed neural network. A voice having a function deciding means for deciding, and a recognizing means for recognizing a voice based on the section of the phoneme group decided by the segmentation means, the certainty factor and the function decided by the function deciding means. Identification equipment. 3. A voice recognition device for recognizing an input voice, the magnitude of power in a certain frequency band of the input voice, and the amount of change in power in a certain frequency band,
Based on the acoustic characteristic of the amount of change in spectrum in a certain frequency band and the power ratio in a certain frequency band and another certain frequency band, the segment and the confidence level of the input speech are determined for each phonological group. A plurality of time-delay neural networks for recognizing the phonemes of the phoneme group to which the maximum certainty factor is given among the phoneme groups segmented by the segmentation means. Function determining means for determining a function representing the validity of the phoneme group determined by the means and the recognition result of the time-delay neural network, and the interval and the confidence factor of the phoneme group determined by the segmentation means. Having a recognition means for recognizing speech based on the function determined by said function determining means, the speech recognition device.