JP3410756B2 - Voice recognition device - Google Patents

Description

Description: BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a speech recognition apparatus capable of recognizing an arbitrary word. 2. Description of the Related Art In a conventional speech recognition apparatus, in order to recognize an arbitrary vocabulary, a single syllable or a feature sequence of phonemes is used as a unit, and recognition is performed using a combination of these. [0003] However, in the above-described conventional speech recognition device, the standard pattern for matching is configured for the entire time series corresponding to these units. For this reason, it is necessary to perform time normalization matching for normalizing a difference in time structure for each utterance by a dynamic programming (DP) or the like, and there has been a problem that the configuration becomes complicated. . Furthermore, the above-described conventional speech recognition apparatus has a problem that a standard pattern corresponding to a phoneme or the like can only calculate the likelihood to its own phoneme category, and the pattern recognition performance is low. Therefore, in the above-described conventional speech recognition apparatus, an efficient way of taking a recognition unit, automatic scanning of phoneme features,
There were problems with the reduction of the computational complexity of dynamic programming, and the learning method of recognition units and standard patterns. SUMMARY OF THE INVENTION An object of the present invention is to provide a speech recognition apparatus capable of reducing the amount of calculation in dynamic programming and performing pattern recognition efficiently in view of the above-mentioned problems in the conventional speech recognition apparatus. . SUMMARY OF THE INVENTION An object of the present invention is to provide a plurality of voice event detecting means for extracting a characteristic portion of an input voice, and a likelihood of a word based on the outputs of the plurality of voice event detecting means. Word detection means for determining the degree of speech, and the voice event detection means connected to the input voice in accordance with the characteristics of the recognition target word are independently scanned on the time axis, and each voice event detection means and word a Ruoto voice recognition system to recognize the input speech based on the output of the detecting means, input
Positive reference vectors corresponding to characteristic parts of speech and characteristic
Based on the likelihood with the anti-reference vector that does not correspond to the
Recognize the input voice and convert it to an acoustic signal containing the voice to be recognized.
On the other hand, each audio event detecting means is scanned and
Assuming the end of word matching, find output from word detection means
The corresponding word based on the maximum value of the output time series
And a continuous speech recognition device . In the voice recognition device of the present invention, the plurality of voice event detection means extracts a characteristic portion of the input voice and detects a word. The means obtains the likelihood of the word based on the outputs of the plurality of audio event detection means, and independently generates the audio event detection means connected to the input voice according to the characteristics of the recognition target word on the time axis. The input voice is recognized based on the output of each voice event detecting means and word detecting means by scanning. At this time, the ginseng corresponding to the characteristic part of the input voice
Reference vectors and anti-reference vectors that do not correspond to characteristic parts
The input speech is recognized based on the likelihood of
Scan each audio event detection means for audio signals including voice
At each time, the word detection
Output from the stage is calculated based on the maximum value of the output time series.
A corresponding word is detected and recognition is continuously performed. An embodiment of the speech recognition apparatus according to the present invention will be described below in detail with reference to the drawings. FIG. 1 is a block diagram showing the configuration of an embodiment of the speech recognition apparatus of the present invention. The speech recognition apparatus shown in FIG.
1. An analog / digital (A / D) converter 12 connected to a microphone 11, a microprocessor 13 connected to the A / D converter 12, a voice event detecting means and a word detecting means connected to the microprocessor 13 Read only memory (ROM) 1
4. Random access memory (RAM) 15 connected to microprocessor 13, microprocessor 1
The external interface 16 is connected to the external interface 3. Next, the operation of the speech recognition apparatus shown in FIG. 1 will be described. The input sound is collected by a microphone 11 and converted into an electric signal. After being subjected to a low-pass filter, the A / D converter 12 converts the analog signal into a digital signal. The audio signal converted into a digital signal by the A / D converter 12 is transferred to a microprocessor 13 via a bus. The microprocessor 13 uses the voice recognition program stored in the ROM 14 to execute the RO
The neural network corresponding to the recognized word phoneme string stored in M14 is called, and the working area is set to RAM1.
In step 5, recognition processing is performed while temporarily storing data, and the recognition result is output to the outside through the external interface 16. FIG. 2 shows an example of a speech waveform. FIG. 2 is an example of the unvoiced plosive / k /, where the plosive part appears in the noise. This burst time may vary depending on the utterance attempt. As described above, it can be considered that the sound event representing the sound feature occurs while a characteristic time series determined to some extent fluctuates on the time axis. In the present invention, a neural network is configured based on the characteristic time series using the characteristic time series of the L frame in order to capture a voice event. The neural network may be a layered perceptron type neural network as shown in FIG.
Learning vector quantization (LVQ (Learni
ng Vector Quantization)) type neural network, but here, the LVQ type neural network of FIG. 4 will be described. In the LVQ type neural network, there are a plurality of reference vectors, and the output of the neural network is calculated based on the distance between the reference vectors and the inner product. Also,
As a comparison with the layered neural network, each reference vector itself is called an output unit, the value of the reference vector is called a unit weight, and the inner product of these is called an output from the output unit. Also, a reference is made to a phoneme event neural network including a reference vector group learned in response to a phoneme event and an inner product operation. Although there is a debate about whether or not the LVQ type is a neural network, it is currently regarded as a type of neural network. For simplicity, a phoneme will be described as an example of a unit of speech recognition. In the LVQ type neural network, as shown in FIG. 5, reference vectors Vkiｋi = 0,. . , N}. When learning data in this category is presented, the reference vector is moved in the direction of the vector, and in a different category, learning is performed so as to move away. However, since it is not possible to sufficiently discriminate only with the positive reference vector indicating that the category belongs to the category, in the present invention, the anti-reference vector Ukj ｛j = 0,. . , M}. Therefore, when learning data belonging to the category is presented, the reference vector Vki is moved in the direction of the learning data, and the anti-reference vector Ukj is moved away. Conversely, when learning data that does not belong to the category is presented, the reference vector Vki moves away from the reference vector Vki.
j is corrected in the approaching direction. When an input to be recognized is given, an inner product of these positive and negative reference vectors is calculated, and the likelihood of the category is calculated as in the following equation. D (l, k) = h (f (d (X1, V
k))-g (d (Xl, Uk))) ls <l <le where Xl is a time-series pattern starting from the first frame of the input speech to be recognized, and ls is the matching start time , Le are collation end times. Also, the function d (Xl,
Vk) is the similarity between Xl and the positive reference vector of category k, and the function d (Xl, Uk) is the similarity between Xl and the anti-reference vector of category k. For example, the similarity to the positive reference vector obtained by the function d can be formed by the maximum value function of the output to each positive reference vector, and the similarity to the anti-reference vector can be formed by the maximum value function of the output of the anti-reference vector. . F is a function for calculating the likelihood of the input, each of the reference vector groups and their categories, and g is a function of calculating the likelihood of the input, each of the anti-reference vectors and its category, for example, max. H is an optimal positioning function when scanning is performed between the target section ls and le. Similarly, max can be considered as this function. FIG. 6 shows the actual distance from each reference vector. At the event position of the target phoneme, the similarity with the normal reference vector increases, and the similarity with the anti-reference vector decreases. In the case of the discrete word recognition, a phoneme event neural network corresponding to a phoneme string of the vocabulary to be recognized is connected, and each network scans a time axis while considering a time constraint to obtain a maximum value. A weighted sum is obtained by a detection neural network to obtain a recognition result. The structure of the word detection neural network is shown in FIG. Learning is performed so that a reliable phoneme event is weighted in the recognition target word. Next, learning of the phoneme event neural network will be described. First, each phoneme event neural network performs initial learning from a speech database to which a certain amount of labeling has been performed. A human designates a feature point for each phoneme and learns the part. The learning is the above-mentioned LVQ learning. Next, as shown in FIG. 7, a learning word is recognized using this phoneme event neural network. At the time of recognition, each phoneme event neural network performs re-learning of each phoneme event neural network at the position of the phoneme event obtained by scanning on the time axis to perform optimization. This is called optimization learning. Next, learning of the word detection neural network is performed using the phoneme event neural network. In this embodiment, the word detection neural network is configured as a sum of the phoneme event neural networks. However, the word detection neural network may be configured by a set of LVQ-type reference vectors and may be learned using learning word data.
This has the function of storing the output pattern of each phoneme event neural network in the target word. Each of the above-described processing procedures will be described with reference to FIGS. FIG. 8 is a flowchart showing the operation of the initial learning. First, initialization is performed (step S
1) The learning data used for the neural network is classified by phoneme label information assigned in advance to obtain a feature parameter sequence of the voice (step S2). The processing in step S2 will be described later with reference to FIG. Using the parameter series of the learning data classified and analyzed for each label, the correct / inverse reference vector of the LVQ neural network is learned (steps S3 to S7). This learning is repeated until a predetermined repetition termination condition (such as satisfying a certain number of times). In the learning, after the presentation order of the learning data is determined by the random number of the phoneme label so that the learning is not biased by the presentation order of the learning data (step S4), the learning data is read (step S6), and the reference is made. Vector learning is performed (step S7). This learning is performed in step S
This is performed for all learning data in a loop of 5. The learning of the forward / backward reference vector in step S7 is described in FIG.
0 will be described later. The reference vector and the average phoneme length learned by repeating the process until a predetermined condition is satisfied in step S3 are stored (step S8), and the process ends. Next, the processing in step S2 will be described with reference to FIG. After the initial setting (step S20)
1) After specifying a label file (step S20)
2) The label file is read (step S)
203). Next, the first phoneme in the label file is specified (step S204), and the learning phoneme is further specified (step S205). Then, the label of the learning data read in step S203 is compared with the current learning phoneme. (Step S206). As a result of the comparison, if they are the same phoneme (step S207), the parameters of the voice data corresponding to the phoneme position of the label file are read (step S208), and the position given in advance based on the label information is determined (step S208). (S209), and store it in the learning data buffer (step S210). The above processing is performed on all phonemes in one label file in step S204. Further, in step S202, the processes in and after step S203 are performed on all label files. Next, with reference to FIG. 10, the learning of the normal / reverse reference vector in step S7 of FIG. 8 will be described. First, the learning order of phonemes is determined by random numbers so as not to be affected by the learning order of phonemes (step S701). Next, a learning phoneme is designated to compare the learning data phoneme label with the learning phoneme (step S70).
3). If they match (step S704), learning of the positive reference vector is performed (step S705). Here, assuming that the input vector is Xl, the positive reference vector Vk
i and the anti-reference vector Ukj are as follows. Vki = Vki + α (X1−Vki) Ukj = Ukj−α (X1−Ukj) If the values do not match in step S704, learning of an anti-reference vector is performed (step S706). Here, assuming that the input vector is Xl, the positive reference vector Vk
i and the anti-reference vector Ukj are as follows. Vki = Vki-α (X1-Vki) Ukj = Ukj + α (X1-Ukj) Next, with reference to FIG. 11, the processing in the output layer which is the recognition unit of the LVQ type neural network of FIG. 4 will be described. . First, after initial setting (step T)
1) Read unit weights of all phonemes, that is, correct / inverse reference vectors (step T2), read a word dictionary composed of phoneme strings of words to be recognized (step T3), and read one frame of input speech (step T4) ), The voice detected flag is checked (step T5), and if confirmed, the process is terminated; if not detected, the input voice is analyzed, norm is calculated, and normalization is performed (step T5).
6). In the loop starting from step T7, each phoneme is collated with the reference vector in order, and at the same time, the word detection is confirmed. The likelihood of each phoneme with the reference vector is determined and compared with the firing threshold (step T8). If the phoneme has not fired (step T9), the process proceeds to the matching with the next phoneme. On the other hand,
If the ignition threshold is exceeded in step T9, the ignition time and the ignition level are stored in a first-in first-out (FIFO) memory having an appropriate size (step T10).
The word dictionary is checked for this firing, and if it is not the terminal phoneme of any word (step T11), the process proceeds to the verification of the next phoneme, and if it is the terminal phoneme, the output of the word net is checked (step T12). Next, the word net output confirmation processing in step T12 in FIG. 11 will be described with reference to FIG. After performing the initial setting (step T10)
1) The matching process with the entire recognition target word is performed in step T10.
This is performed in a loop from Step 2 to Step T107. First, one recognition word is designated (step T102), and it is confirmed whether or not the terminal phoneme of the word has been fired (step T103). If not, the word is collated with the next word. If it is fired, the time at which the phoneme in the dictionary is fired is checked in reverse time from the time at which the terminal phoneme is fired, and it is checked whether each firing time is within the allowable range by the duration (step T105, T10
6). It is sufficient that the allowable range based on the duration is, for example, 0.75 to 1.5 times the average duration of the phoneme, but it is also possible to learn from the learning data. If it falls within the allowable range, the firing value P of the phoneme is added to Ow, and the phoneme in the next dictionary is checked (step T107). When the matching with all the words is completed in step T102, the likelihood for each word is sorted, and the word detection flag is turned on (ON) (step T10).
8) Output the top M results (step T10)
9). Next, the optimization learning of the LVQ type neural network shown in FIG. 4 will be described with reference to FIG. The optimization learning is a process for performing optimal learning according to a recognition algorithm, and has a great effect on improving the recognition performance. First, after performing the initial setting (step V
1) The unit weight of all the phonemes that have been initially learned, that is, the reference vector is read (step V2), and the phoneme sequence of the word to be recognized is read (step V3). The optimization learning is performed at a predetermined repetition condition in step V4.
For example, repetition is performed until a certain number of repetitions is reached. Further, in the loop of step V5, the reference vector of each phoneme is updated for the entire learning word. First, one learning word is specified, and its data is connected to phonemes corresponding to the word to perform recognition processing (step V6). Thereby, each phoneme event position determined in the recognition processing is obtained. If the output value of the word net obtained as a result is equal to or smaller than the threshold value, it is determined not to learn (step V7). This is to prevent words with too low precision from being used for learning. As the learning progresses, the threshold value is exceeded, so that all words are finally set to be used for learning. This has the effect of improving the learning speed. For words that exceed the threshold value in step V7, word W optimization learning is performed (step V8). This process is performed on all words, the reference vectors of all phonemes are updated, and the repetition is further performed to optimize the recognition unit and the learning unit. When the fixed learning is completed, the unit weights, that is, the reference vector and the average phoneme length are stored (step V9), and the process ends. Next, with reference to FIG. 14, a description will be given of the word W optimization learning method in step V8 of FIG. After performing the initial setting (step V80)
1) In the loop from step V802 to step V808, learning is performed in the order of phonemes in the word to be learned. First, a phoneme is specified in order from the beginning of the word, and the firing position of the phoneme is read (step V803). In the loop from step V804, learning of the normal reference vector and the anti-reference vector is performed by sequentially referring to all phonemes. First, the first learning phoneme is designated and the recognition data phoneme is compared with the learning phoneme (step V80).
6). If they match in step V806, learning of the reference vector is performed (step V807), and if they do not match, learning of the anti-reference vector is performed (step V808). The learning of the forward / backward reference vector is the same as that in steps S705 and S706 in FIG. After the learning of all phonemes included in the word is completed in step V802, the process ends. In the case of continuous speech recognition or word spotting, continuous collation is necessary. This is based on the output of each phoneme detection neural network, as shown in FIG. It is assumed that each time is hypothesized as an end point by using the above method, and a point at which the likelihood of the entire word at that time exceeds a certain threshold and reaches a maximum is detected. In this case, if each phoneme event neural net is not targeted unless there is a likelihood that is equal to or greater than a certain threshold, the amount of calculation of the continuous DP can be reduced. The feature of the voice is such that the voice event which is the feature of the voice in the time series is generated while fluctuating in position, and it is unnecessary to target the entire phoneme as a recognition unit. May include parts. Therefore, the feature of the voice event part is learned by a neural network, and a voice sequence is obtained by scanning over time to recognize the time of all acoustic parameters serving as a recognition unit. The recognition accuracy can be improved. Further, since only the portion having an output larger than a certain likelihood is targeted, the calculation amount of the dynamic programming (DP) used for obtaining the optimal phoneme sequence can be reduced. In the conventional method, the coincidence between the input speech and the standard pattern (model) is measured only by the distance and the similarity. However, in the speech recognition apparatus of the present invention, the distance between the input speech and the standard reference vector for measuring the similarity is measured In addition, by learning an anti-reference vector for obtaining the degree of difference, and calculating the original likelihood from both, accurate identification is possible. Also, as a neural network learning method for voice events, initial learning is performed based on label information obtained by visual observation or the like, and this is connected to form a learning word model. By re-learning and optimizing the speech event neural net at the position of the speech event detected by the net, a neural network most suitable for the speech event to be recognized is constructed. The speech recognition apparatus of the present invention comprises a plurality of speech event detection means for extracting a characteristic portion of an input speech,
Word detection means for calculating the likelihood of a word based on the outputs of the plurality of voice event detection means. The voice event detection means connected to the input voice in accordance with the characteristics of the recognition target word is displayed on the time axis. in a Ruoto voice recognition system to recognize the input speech based on the output of the independently scanned each audio event detection means and word detection means, the input speech
The positive reference vector corresponding to the characteristic part and the characteristic part
Based on the likelihood with the anti-reference vector not corresponding to
Recognizes speech and responds to audio signals containing speech to be recognized.
Scans the audio event detection means and verifies words at each time
Output from word detection means assuming the end of
The corresponding word is detected based on the maximum value of the time series of
Since recognition is performed continuously , any word can be recognized efficiently and with high accuracy.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a block diagram showing a configuration of one embodiment of a speech recognition device of the present invention. FIG. 2 is an explanatory diagram of a voice and a voice event. FIG. 3 is an explanatory diagram showing a configuration example of a layered neural network. FIG. 4 is an explanatory diagram showing a configuration example of an LVQ type neural network. FIG. 5 is an explanatory diagram of the arrangement of reference vectors. FIG. 6 is an explanatory diagram of an output of each phoneme event neural network. FIG. 7 is an explanatory diagram of an optimization learning position. FIG. 8 is a flowchart for explaining initial learning. FIG. 9 is a flowchart for explaining initial learning. FIG. 10 is a flowchart illustrating initial learning. FIG. 11 is a flowchart illustrating an operation of a recognition unit. FIG. 12 is a flowchart illustrating an operation of a recognition unit. FIG. 13 is a flowchart for explaining optimization learning. FIG. 14 is a flowchart for explaining optimization learning. FIG. 15 is an explanatory diagram of word spotting application. [Description of Signs] 11 Microphone 12 A / D converter 13 Microprocessor 14 ROM 15 RAM 16 External interface

──────────────────────────────────────────────────続 き Continuation of front page (72) Inventor Seiji Hamaguchi 22-22 Nagaikecho, Abeno-ku, Osaka-shi, Osaka Inside Sharp Corporation (56) References JP-A-64-81999 (JP, A) JP-A-1- 204099 (JP, A) JP-A-3-269500 (JP, A) JP-A-1-116869 (JP, A) JP-A-2-170265 (JP, A) JP-A-5-334276 (JP, A) (58) Field surveyed (Int. Cl. ⁷ , DB name) G10L 15/08

Claims (1)

(57) [Claims 1] A plurality of voice event detection means for extracting a characteristic part of an input voice, and a likelihood of a word is obtained based on outputs of the plurality of voice event detection means. Word detecting means, wherein the voice event detecting means connected to the input voice according to the characteristics of the recognition target word is independently scanned on the time axis, and each of the voice event detecting means and the a recognition <br/> be Ruoto voice recognition device said input speech based on the output of the word detection means, and a positive reference vector corresponding to the characteristic parts of the input speech
Likelihood with anti-reference vector not corresponding to the characteristic part
And recognizes the input voice based on the voice event.
Scans the detection means and assumes the end of word matching at each time
To obtain the output from the word detection means,
Detects the corresponding word based on the maximum value of the column and continuously
A speech recognition device for performing recognition .