JPH03120600A - Continuous voice recognizing system by neural network - Google Patents

Abstract

PURPOSE:To perform the recognition of continuous voice at high speed and with high accuracy by predicating a phoneme by using a syntax analysis method, and taking the matching of a predicted phoneme and a phoneme spotting result by a neural network with dynamic programming. CONSTITUTION:An inputted voice 1 is frequency-analyzed, and is supplied to a phoneme spotting part 2 after being changed to the form of time series of a feature parameter, and the phoneme spotting part 2 outputs the spotting results of 24 phonemes. Next, an LR table 6 is generated according to context- free grammar stored in a context-free grammar storage part 4 with an LR table generator 5, and an LR purger 7 predicts a phoneme series permitted in grammar as referring to the LR table 6. A predictive phoneme storage part 8 stores a predicted phoneme series, and a phoneme recognition result verifica tion part 3 takes the matching of the predicted phoneme series and the result obtained at the phoneme spotting part 2 with the dynamic programming, and a series which takes the maximum likelihood out of verified phoneme series is outputted to a recognition result output part 9 as a recognition result.

Description

[Detailed Description of the Invention] [Field of Industrial Application] This invention relates to a continuous speech recognition method using a neural network, and in particular, to a speech recognition device using a neural network that recognizes continuously uttered speech. Concerning continuous speech recognition method using neural network.

[Prior art and problems to be solved by the invention] Conventionally, when recognizing continuously uttered speech, first the segmentation of the phoneme in the continuous speech is performed, then the segmentation is performed. A method that recognizes the recorded voice is generally adopted. In addition, in the conventional method,
It is difficult to establish a high-precision phoneme segmentation method and a phoneme recognition method, and the recognized phonemes are once output in the form of an ambiguous "phoneme lattice" and then top-down from information such as dictionaries. Usually, the content of the utterance is identified.

However, such a method has problems in that not only the recognition system becomes complicated, but also it is difficult to construct a highly accurate continuous speech recognition system.

Therefore, the main purpose of this invention is to combine the phoneme spotting results in continuous speech obtained by phoneme spotting technology using a neural network and the phonemes predicted by the extended LR parser using dynamic programming rDyna.
mLc Time-Wrapping Matchi
An object of the present invention is to provide a continuous speech recognition method using a neural network that can be integrated with ngJ to construct a highly accurate continuous speech recognition system.

[Means for Solving the Problems] The present invention is a continuous speech recognition method using a neural network, which includes an analysis means for analyzing continuously uttered input speech, and a time series of characteristic parameters for analyzing the analyzed speech. a normalizing means for normalizing the feature parameter time series over a certain time domain; and a means for spotting phonemes in continuous speech using a neural network using the normalized feature parameters. It uses a syntactic analysis method to predict phonemes in continuous speech, and matches the predicted phonemes with the phoneme spotting results from a neural network using dynamic programming, which has the ability to time-normalize speech. be.

[Operation] The continuous speech recognition method using a neural network according to the present invention uses a time-delay neural network (TDNN), which is a type of neural network.
Neural Net.

rk) [11] and the extended LR parsing method, which is a type of parsing method, to predict the phonology, integrate the predicted phonology and the phonological recognition result by TDNN by dynamic programming, Recognize continuous speech with high accuracy.

[Embodiment of the Invention] FIG. 1 is a block diagram showing a time delay neural network in an embodiment of the invention. Referring to FIG. 1, continuous speech is input to input layer 11, and this continuous speech is provided to subnetworks 12 to 20 as intermediate layers.

Among these subnetworks 12 to 20, subnetworks 12 to 17 and 19 are the 24th subnetwork of all Japanese phonemes.
Type Cb+ dlgo p* LlkIm+ '
n+ NI S, Sh, h, z, ch,
ts, r, w, y, a, i.

Spot us e+ o, Q (silence)).

That is, the subnetwork 12 has three phonemes, d,
g, and the network 13 identifies p, t.

k, and subnetwork 14 identifies m, n, N
, and the subnetworks 15 are s, sh, h.

2, subnetwork 16 identifies ch, ts, subnetwork 17 identifies r, w, y, and subnetwork 19 identifies a, t, u, e.

Identify 0. Subnetwork 18 identifies six phoneme groups, subnetworks 12 to 17, and subnetwork 20 identifies speech or silence.

These sub-networks 12 to 20 are integrated by an integration network 21, and the spotted 24 phonemes are output to an output layer 22. Note that network learning is performed using the error back propagation method (Error Back-Pr
pagation) [2]. This method is a method in which an error, which is an evaluation function, is sequentially reduced locally in the feature space based on the steepest descent method.

FIG. 2 is a diagram for explaining a method for spotting phonemes in continuous speech in one embodiment of the present invention.

Referring to FIG. 2, input voice 11a is given as input data. In FIG. 2, the vertical axis represents frequency and the horizontal axis represents time. The input speech 11a is given to the input layer 11 of the neural network shown in FIG.
This is done by scanning frame by frame in the time direction. Every time one frame is shifted, the output layer 22 outputs the phoneme spotting result of one of the 24 phonemes. Note that the middle layers 12 to 21 of the network shown in FIG. 1 are omitted. The method shown in FIG. 2 is an extremely simple and excellent method that does not require phoneme segmentation unlike conventional methods.

FIG. 3 is a block diagram showing the configuration of a recognition system using the TDNN-LR method. Referring to FIG. 3, input speech 1 is frequency-analyzed, converted into a time-series format of feature parameters such as FFT output, and provided to a time-delay neural network 2. As explained in FIG. 1, the time delay neural network 2 outputs the spotting results of 24 phonemes.

On the other hand, a context free grammar is stored in the context free grammar storage unit 4, and an LR child table is generated by the LR table generator 5 according to this context free grammar. The LR parser 7 predicts the phoneme sequence to be grammatically determined while referring to the LR child table. The predicted phoneme storage unit 8 stores the predicted phoneme sequence in advance, and the phoneme verification unit 3 uses the predicted phoneme sequence stored in the predicted phoneme storage unit 8 and the predicted phoneme sequence obtained by the time delay neural network 2. The phonological spotting results and D
Verification is performed using TW matching. Among the verified phoneme sequences, the sequence with the maximum likelihood is output to the recognition result output unit 9 as a recognition result.

Here, the phoneme prediction method using the LR parser 7 will be briefly explained. The LR parser 7 analyzes a grammar generated from a limited grammar called an LR grammar among context-free grammars. This parser can accept input signals and parse them deterministically without backtracking. The LR parser 7 performs the analysis while looking at the 2F1 class tables, ie, the operation clothes table and the destination table. The action code is a table showing the next action the parser will perform.
The destination table is a table that indicates the next state the parser will take. There are four types of parser operations:

■Move (shift) ■Reduce (reduce) ■Accept (accept) ■Error (error) ■Move is the action of putting the parser state on the stack,
■Reduction groups symbols on the stack according to grammatical rules. ■Acceptance indicates that the input text could be parsed by the LR parser; ■Error indicates that it could not be analyzed.

Next, the analysis procedure is shown.

"Definition" S: Parser status a: Grammar symbol (non-terminal, terminal symbol) Input pointer: Indicates the input symbol string currently being processed.

State stack: saves parser state.

GOTO (s, a): Find the next state from state S and grammatical symbol a.

ACTION (s, a): Find the parser action from the state S and the grammar symbol a.

"Algorithm" ■ Initialization: Position the input pointer at the beginning of the input symbol string. Push 0 onto the state stack.

■ From the current state S and the symbol a indicated by the input pointer to AC
Examine TION (s, a).

■ If ACTION (s, a) - shift, push GOTO (s, a) onto the state stack and advance the input pointer by one.

■ACTION (s, a) Kasumi’reduce.

n”, pop the states of the stack equal to the number of grammar symbols on the right side of the nth grammar rule.If the state at the top of the stack is S', then pop S' and the grammar rule on the left side of the nth grammar rule. From A, next state GOTO (!!', A)
Ask for and put it on the stack.

■ If "ACTION (s, a)-accept", the analysis ends.

■ ACTION (s, a) m e r r o r
If so, the analysis has failed.

■ Return to ■.

The extended LR parser is capable of parsing ambiguous syntax that the LR parser could not handle. Extended LR
In the parser, multiple items are written in the action code. When the parser examines this table of multiple items, it performs parallel operations. In this way, the syntax is analyzed definitively.

FIG. 4 is a diagram showing an example of phoneme spotting results. The example shown in FIG. 4 is a case where the user utters "to the meeting", and shows the spectrum llb of the input voice and the phoneme spotting result 22a. In order to verify the validity of the input speech and phoneme spotting results, phoneme labels are assigned in advance by inspection. In FIG. 4, black squares represent activated outputs.

FIG. 5 is a diagram showing the operation of the phoneme recognition result verification section 3 shown in FIG.
2, a DP matching pass 31, and a phoneme sequence 32 predicted by the LR parser. In Figure 5, /
It can be seen that the input speech uttered as kaigini/ is matched between the predicted phoneme sequence 32 and the phoneme spotting result 22 by the DP matching path 31.

[Effects of the Invention] As described above, according to the present invention, a simple and highly accurate phoneme spotting method using a time delay neural network (TDNN) and a phoneme sequence predicted by an extended LR parser are combined using dynamic programming. Since matching is performed using (DTW), continuous speech can be recognized with high precision and at high speed.

[Brief explanation of drawings]

FIG. 1 is a block diagram showing a time delay neural network used in an embodiment of the present invention. FIG. 2 is a diagram showing a method for spotting phonemes in continuous speech. FIG. 3 is a block diagram showing the configuration of a recognition system using the TDNN-LR method. FIG. 4 is a diagram showing an example of phoneme spotting results according to an embodiment of the present invention. FIG. 5 is a diagram showing the operation of the phoneme recognition result verification section shown in FIG. 3. In the figure, 1 is input speech data, 2 is a phoneme spotting unit, 3 is a phoneme recognition result verification unit, 4 is a context free grammar storage unit, 5 is an LR child table generator, 6 is an LR table, 7
(R parser, 8 is a predictive phoneme storage unit, 9 is a recognition result output unit, 11 is an input layer, 12 to 20 are networks as intermediate layers, 21 is an integration network, and 22 is an output layer. Amendment C7j Ka6, Subject of amendment March 1, 1990 Drawing 7, Contents of amendment (1) The engravings of Figures 4 and 5 of the drawings are as attached (no change in content). 2. Invention Continuous speech recognition method using neural network 3 Name of person making the correction Name of ATR Automatic Translation Telephone Research Institute Representative Akihito Akira 4 Address of agent 2-1-29 Minamimorimachi, Kita-ku, Osaka City Sumitomo Bank Minamimorimachi Building 5, date of amendment order

Claims (1)

[Scope of Claims] Analysis means for analyzing continuously uttered input speech; conversion means for converting the speech analyzed by the analysis means into a time series of feature parameters; and time series of feature parameters converted by the conversion means. normalizing means for normalizing a sequence over a certain time domain, and means for spotting phonemes in continuous speech by a neural network using feature parameters normalized by the normalizing means, and using a syntactic analysis method. A continuous speech recognition method using a neural network, which is characterized by predicting the phoneme in continuous speech using a dynamic programming method that has the ability to time-normalize speech, and then matching the predicted phoneme with the phoneme spotting result using a neural network. .