JPH0457099A - Voice recognizing device - Google Patents

Abstract

PURPOSE:To recognize the recognition object vocabulary of an input voice even under the noise environment by constituting the device so that feature amounts inputted to each event net are shifted timewise from each other, a position in which an output value becomes maximum is selected therein, and a supernet inputs an output from a word net and a value corresponding to a recognition word to which an input voice belongs. CONSTITUTION:A voice inputted from a microphone 21 is amplified by an amplifier 22, A/D-converted 23, and thereafter, inputted to a sound analyzing part 24. Subsequently, after it is inputted to the sound analyzing part 24, a value of output power of each BPF 25 is outputted. A compressing part 26 decreases a dimension of a feature vector of the input voice. Next, all frames outputted from a feature vector store part 27 are inputted to an event net 28. Thereafter, a word net output store part 32 writes in an output from a word net 31 at the time when the event net selects the word head, selects the maximum one and outputs it to a supernet 33. As for the output of each word net, only the maximum output in plural outputs is selected, inputted to the supernet, and the output of the supernet is calculated. The calculated output of the supernet is sent to a result deciding part 34, and by deciding the threshold, the result of recognition is outputted to a result display part 35.

Description

DETAILED DESCRIPTION OF THE INVENTION [Field of Industrial Application] The present invention relates to a speech recognition device using a neural network.

[Prior Art] In general, a speech recognition device requires a process of extracting only a voice section by removing silent sections and noise sections before and after utterance from a signal input from a microphone, that is, detecting a voice section. It is.

Detection of voice sections is not so difficult if the signal-to-noise ratio (hereinafter referred to as S/N ratio) is good. In that case, it is sufficient to detect an interval in which the value of the power time series of the audio signal exceeds an appropriate threshold value as an audio interval.

However, in a real environment, the S/N ratio deteriorates due to various noises, such as weak fricatives, and voiced sounds with small amplitudes that exist at the beginning (hereinafter referred to as word-initial) and at the end (hereinafter referred to as word-final) of speech. etc. becomes difficult to detect. Furthermore, non-stationary noise may be erroneously detected as a voice section.

As one of the methods for detecting speech sections in a noisy environment,
There is a method of selecting a suitable speech interval from a plurality of interval candidates.

In the above method, speech recognition is actually performed for each candidate segment, and the segment with the highest matching score is selected as a qualified speech segment.

A further development of the above method is a method in which all times in the data are used as word beginning and word ending candidates, speech recognition is performed on all sections, and sections with high matching scores are found. An example of this is word spotting. A continuous dynamic programming method (hereinafter referred to as continuous DP method) is used in word spotting matching.

As a word speech recognition device, there is a ``speech recognition device'' by Yamaguchi and Itamoto (Japanese Patent Application No. 69248/1999).

This word speech recognition device inputs the features obtained by acoustically analyzing speech into each unit of the input layer of a multilayer perceptron neural network, and then outputs speech recognition results according to the output value of each unit of the output layer. get.

The word speech recognition device described above uses a predetermined method to detect features obtained by acoustically analyzing input speech for each frame based on time interval information when inputting them to each unit of the input layer of the event net. The features input to each event net are temporally shifted within a predetermined range starting from near the beginning of the word, and the position where the output value of each event net is maximum is selected among the temporally shifted features. By doing this, the time warp of the input audio is corrected, and
The maximum output position of the final event net is the end of the input audio.

[Problems to be Solved by the Invention] A speech recognition device using word spotting using the continuous DP method described above has problems in that it has a low ability to reject input other than the vocabulary to be recognized and has low noise resistance. In addition, since only local distances are observed for each frame, it is easy to add unnecessary words and drop words and phonemes. Also, since DP matching must be constantly performed, the amount of calculation and memory required for inter-frame distances is increased. The problem is that there are many.

The word speech recognition device described above has the problem that the beginning of a word must be detected in advance by some method, and if the detection error is large, erroneous recognition and rejection occur.

SUMMARY OF THE INVENTION In view of the above-mentioned problems with conventional speech recognition devices, an object of the present invention is to provide a speech recognition device using a neural network that can recognize word nests to be recognized in input speech in a noisy environment. .

[Means for Solving the Problems] The above-mentioned object of the present invention is to provide an event net that inputs feature quantities obtained by acoustically analyzing input audio for each frame, and an input system that inputs and inputs the output from the event net. WordNet outputs a value corresponding to the degree of similarity of speech to a specific word in the recognition target vocabulary, and inputs the output from WordNet and outputs a value according to the recognized word to which the input speech belongs. Based on time interval information obtained by analyzing voice samples from multiple speakers, EventNet temporally shifts features within a predetermined range, starting at an arbitrary time as the beginning of a word. , speech recognition configured to select the position where the output value is maximum among the temporally shifted features and output a value corresponding to the similarity of the partial phoneme sequence in a specific word. achieved by the device.

[Operation] The event net inputs the feature values obtained by acoustically analyzing the input audio frame by frame, and based on the time interval information between adjacent event nets obtained by analyzing the audio samples of multiple speakers. , the feature amounts input to each event net are temporally shifted within a predetermined range with an arbitrary time as the beginning of the word, and it is determined whether the word is handled by that event net or not, and the temporally shifted features are The position where the output value is maximum is selected among the amounts, and the value corresponding to the similarity of the partial phoneme sequence in a specific word among the recognition target word nests is output, and the word net uses the output from the event net. The supernet inputs the input speech and outputs a value corresponding to the degree of similarity with a specific word, and the supernet inputs the output from the wordnet and outputs a value corresponding to the recognized word to which the input speech belongs.

[Embodiment] Hereinafter, an embodiment of the speech recognition device of the present invention will be described in detail with reference to the drawings.

FIG. 1 shows the configuration of a speech recognition device according to this embodiment.

The speech recognition device shown in FIG. 1 includes a microphone 21, an amplifier 22 connected to the microphone 21, an analog/digital converter (hereinafter referred to as an A/D converter) 23 connected to the amplifier 22, and an A/D converter. A plurality of band pass filters (hereinafter referred to as BPF) 2
5 are connected in parallel, the acoustic analysis section 24 and the acoustic analysis section 24
A compression unit 26 connected to the compression unit 26, a feature vector storage unit 27 connected to the compression unit 26, and a plurality of event net groups each having a plurality of event nets 28 connected to the feature vector storage unit 27 in parallel. 29. Event net output storage unit 30 connected to each event net 28 and provided in each event net group 29
．． A plurality of wadnets 31 . each connected to each eventnet group 29 . A plurality of wordnet output storage sections 32 each connected to each wordnet 31, a supernet 33 connected to a plurality of wordnets 31, a result judgment section 34 connected to the supernet 33, and a connection to the result judgment section 34. It is configured by a result display section 35.

Next, the operation of the speech recognition device shown in FIG. 1 will be explained.

First, the audio input from the microphone 21 is transmitted to the amplifier 2.
2 and converted from an analog signal to a digital signal by an A/D converter 23, and then input to an acoustic analysis section 24.

The acoustic analysis unit 24 acoustically analyzes the input audio using the BPF 25 and outputs the output power value of each BPF 25 for each frame.

Note that the above acoustic analysis is not limited to the analysis using the BPF group.
Linear Predictive Coding ((Line
ar Predictive Coding), hereafter L
(referred to as PG) or parameters obtained by cepstral analysis, etc. may be used.

The compression unit 26 uses KL transformation to reduce the dimensions of the feature vector of the input speech in order to reduce the scale of the network.

The feature vector storage section 27 sequentially receives the feature vectors compressed by the compression section 26 by the 1'L transformation.

However, immediately after the start of the operation, there is no actual input from the microphone 21 yet, so the feature vector storage unit 27 pseudo stores the feature vector of the silent section for one second as the initial value of the feature vector ( Here, the value of T represents a number that depends on the vocabulary to be recognized).

Since the speech recognition device shown in FIG. 1 does not detect the beginning of a word, all frames output from the feature vector storage section 27 are input to the event net 28.

Note that, as shown in the figure, a plurality of event nets 28 are connected in parallel to form an event net group 29.

The feature vector storage section 27 is a ring buffer as shown in FIG. 2, and the current storage location of the feature vector is indicated by a W pointer (for writing). The F pointer in the figure represents the time (frame) of the assumed beginning of a word. In reality, the duration differs depending on the word, so the value of T above is determined corresponding to each word r (r = 1.2..., R1, where R represents the number of word nests). This improves processing efficiency. Note that the word r is a standard pattern composed of an event net and a word net.

If the current time is tb, the W pointer is t6, word r
The beginning of the word is represented by t and r, respectively.

The value of T may be set to approximately the maximum duration during the word search, and in this embodiment, T=1.2 seconds.

When the current time is tb, the beginning of the word r is assumed to be all frames belonging to the interval (, tS,, r+Δ]. Here, Δ is Δ-tbt1r-TlIlinr. Mata, 1m111r is the minimum possible duration of the has word r.

FIG. 3 shows the beginning t of word r at the current time tb1.

The relationship between minimum duration T, r and Δ is shown.

1n Next, a method for detecting the beginning of a word using the speech recognition device shown in FIG. 1 will be explained.

First, all frames within the interval [,I,,++Δ], i.e., t, t, +1. t, r+f 2,... t, r+Δ are all assumed to be word beginnings.

When t and I are at the beginning of a word, frames (K
(generally varies depending on the word, but is set to 3 here), the center of the frame that is the target of the operation in event net E "11 is tl3, t
1-2. ..., tfr+3.

Also, when tl +1 is the beginning of a word, "event net"
The center of the frame that is the target of rl calculation is t'-
2. t,'-1,·,tfr+4.

However, among these frames, tf 2. t+-1,
..., t'+3 is calculated when tfr is the beginning of the word, and has already been stored in the event net output storage unit 30.
Since it is stored in , use the calculation result.

This event net output storage section 30 also has a ring buffer structure like the feature vector storage section 27. Further, the event net output storage section 30 is provided for each event net group 29 corresponding to the word r. That is, N event net output storage units 30 exist for one word r (N is the number of event net groups 29, and in this embodiment, N=5).

As mentioned above, regarding event net E+1, tl
When +1 is the beginning of a word, only the frame t1+4 is newly calculated.

Below, each event net E, 2. E, 3.
E, 4. For E and 5, the duplicated calculation portions are similarly read from each event net output storage section 30. In addition, when a new calculation is performed, the output result from the event net 28 is written into each event net output storage section 30.

Above, when assuming that tS+Δ is the beginning of a word from tr,
The output from the event net 28 at the current time tb is obtained as described above.

Next, event net E in the interval [tll, tIr+Δ]
, ■ The beginnings of words determined by selecting the maximum value of the search range are expressed as f 1 , f2' . . . , fpL.

However, p is a value that satisfies the condition of p<Δ, and is usually 2 to 3.
It is.

The word net output storage unit 32 stores the event net E,1.
writes the output from the word net 31 when the above word initial fj' (j=1.2, . . . , p) is selected.

Then, the largest value among the values stored in the word net output storage section 32 is selected and output to the supernet 33.

The basic operations of the event net 28, word net 31, and supernet 33 will be explained below.

In FIG. 4, among the feature vector series, a frame series in a range corresponding to the input layer of the event net 28 is input to each event net 28.

In the event net 28, for a specific recognition target word, there are N pieces (where N is a positive integer) of feature vector sequences input to the input layer shifted in the time axis direction, and in this embodiment, N= It is 5.

Note that N may be set to a different value depending on the word.

For normal words of 3 to 4 syllables or less, N = 5, and for long words of 5 or more syllables, N = [m/2 + 3.5] (however, m
is the number of syllables, and [x] is the largest integer not exceeding X).

Next, a method of shifting the feature vector sequence in the time axis direction during recognition will be described.

Let El be the name of the i-th event net that recognizes the i-th word to be recognized, and the output layer of event+1 tonet E has two units C40 and I.
I
There is 11C.

” Event Net E9. When the partial phoneme sequence of word j (corresponding to the i-th word) that is in charge of recognition (corresponding to around j/N from the beginning of the word if the duration of the word is set to 1) is input, The event net E is trained so that , (C,,,C,,)=(1,O)III.

Conversely, if a partial phoneme sequence other than the above is input, the event net E is trained so that (C,,,C,,) = (0,1) j II. . That is, unit C is event net E.

11 N has a high value for a specific point in the word it is responsible for.

The shift interval in the time axis direction is one frame of the compressed feature vector series. Note that this may be set to two frames if it is desired to reduce the amount of calculation.

Assuming that the amount of shift range in the time axis direction (the same amount as the number of frames in the search range) is n, the value of n is equal to the value of event net E4. +1 In Figure 4, for event net Ei1, n
= 5, and n = 7 for the event net Ei2.

In addition, the event net E is indicated by Eijl'J E ..., El, in order from the front, and the output is C+
i2′ +1n , C1 . . . , C1, represented in the − membrane.

jl +i2' +1n Figure 4 shows E, , E, as part of it.

1 +12 E, , E , E, , C, and C112 are shown +13 +21 +22 dl.

As inputs to the word net 31, these n C,
, C, ..., C1, each jif +
Select for the value of j2 Bn.

Note that the search range of the event net E1 is set to be a fixed amount, for example, three frames each, around the assumed beginning of the word. Alternatively, the standard deviation of the duration of the entire word may be multiplied by a constant based on the statistics of many speakers.

In the figure, event net E1. The search range is indicated by a horizontal bar arrow, and the position selected as the maximum value for each unit C1 (j+17 = 1.2., .5) is represented by a thick solid line. ing.

For example, in Eventnet E, E, E.

il 112ゝ 12 In each case, E125 is selected.

Next, we convert event net Ei i-1 to event net E
, (j〉1) (For example, the event net before event net E14 is E, that is, El3. Hereinafter, 714-1ゝ Inus (-) symbol is all ).

The search range for event nets E, , (j>1) is j for event nets E1, . Based on the mean (m) and standard deviation (σ, ) of the temporal difference between and event net Ei i-1,
It is calculated as follows. Note that m is constant regardless of j.

Maximum +-1 from output C,,C,...,C1"
By selecting +1-2' +1-n, the position of event net E1j-1 is determined.

The search range of event net E is from m −Kaj to m + based on the maximum position of Cj−1]
It is within the range of Kaj. Here, K is a constant and is about 2 to 3. However, if Cj is smaller than the maximum position of C11-1 by m-, the former is adopted.

That is, if the search range is set as (L,,R,), then J j max (maximum position of m-Kaj,C,-1), R・-
σ to m+.

J It is expressed as

As an example, when j-2, using the above relationship, output c, , c, ,...''+27 to output C+2
1 +22 25 is chosen as the maximum value (see Figures 4 and 5).

In addition, when selecting the maximum value, simply max (C1j
, ), the following modifications may be considered depending on the properties of the event net and the amount of calculation.

First, all outputs C,j, (f=1.2°...
·, n) is a small value, the center l−m of the search range is selected without selecting the maximum value. As a result, event net E1. It is possible to avoid unnecessary matching for input words other than those for which the user is responsible, and improve the rejection ability.

Second, all outputs C1j, (J=L 2...
・When n) is a large value, J
= m. This makes it possible to avoid unnatural matching when similar feature vectors, such as those found in long vowels, continue for a long time.

Third, all outputs C,,,(j!=1. 2....
When m) is a small value, the search range is expanded by a certain amount α, and m=m+α, A=m+1°m+2.・・・
・Output C1j for 9m1α,! Find the maximum value selection. This is particularly effective for samples with slow speaking speeds.

Next, learning of the above event net 28, word net 31, and supernet 33 will be explained.

The event net 28, word net 31, and supernet 33 are basically trained using the error backpropagation method in a multilayer perceptron type neural network.

However, the event net 28, word net 31, and super net 33 perform learning not only on voice samples but also on silent samples, that is, noise sections. Then, (C,,,C,,)=(0,1)+1 1] is given. That is, it is assumed that the noise section is not a partial phoneme sequence handled by the event net.

Here, if the event net is in charge of a long silent section such as a consonant, samples of the above-mentioned noise section are not provided.

Regarding whether or not to provide a noise sample, it is decided to search for a sample that maintains a large error during the learning process, and if it is a noise sample, to exclude it from subsequent learning.

When a noise sample is input to a word net, a teacher signal of (C,,C,)-(0,1) is given, assuming that it is not a word for which the word net is responsible.

In the supernet, learning is performed by assigning 1 to units corresponding to rejects for such wordnet outputs.

During actual speech recognition operation, tb is set to the current time and tb +1. Increment one frame at a time, such as tb +2. Along with this, word-initial t1
is also incremented by one frame.

When incrementing all word beginnings tS uniformly by one frame, the word beginning tI has the same value regardless of the word r.

The section [
With reference to the calculation result of event net Er1 at
Frames with low values may be skipped for more efficient calculation.

The threshold value is θ (usually 0.1 to 0.2), and t11+
If C21<θl in i (1≦i≦Δ), then the increment amount is i+1, that is, the next word-initial assumption frame is t'r r+t + 1.

According to the method described above, at the current time tb, a plurality of word initial candidates exist for each word r.

However, only the maximum output among the plurality of outputs is selected as the output of each word net.

The output of the word net selected above is input to the supernet, and the output of the supernet is calculated at each current time tb.

The calculated supernet output is sent to the result determination section 34. The result determination unit 34 outputs the recognition result to the result display unit 35 by threshold determination as described below.

First, let C9 be the value of the supernet output corresponding to the first word, and let n be the number of recognized vocabulary. Further C
is the value of the output of the supernet that does not respond to rejection and is n+, and θ.

θ1 is a threshold value, and in this example, θa=0.6°θ,
=0.1.

Then, recognition is performed according to the following rules: max (
If C, ) < θ8, reject (Rule 1).

max (C,) - max (C,) < θd (where,
If I is ■ that satisfies m a x (C-) = C1), reject (Rule 2).

C〉θ n+1 a If so, reject it (Rule 3).

In cases other than rules 1 to 3 above, max (C,) = C1 The recognition result is I that satisfies (Rule 4).

The above recognition results are input to the result display section 35 and displayed.

It should be noted that speech other than recognition word tools may be handled as a learning target for the event net 28, word net 31, and super net 33. In this case, the learning method is the same as that for noise samples.

As the number of learning samples increases, the time required for learning to converge increases, but it improves the ability to reject input other than the target vocabulary and the ability to find target word nests from continuously uttered speech. You can also take it out.

Therefore, it works effectively even against relatively stationary noise.

In addition, when training the event net 28, by using speech samples to which several levels of stationary noise have been added as learning targets, the neural network's muddying ability can be used to improve accuracy for various levels of stationary noise. Voice recognition can be performed.

[Effects of the invention] An event net that inputs the feature values obtained by acoustically analyzing the input voice for each frame, and an input of the output from the event net to identify words to be recognized for the input voice. The event net is equipped with a word net that outputs a value corresponding to the similarity with the word of , based on time interval information obtained by analyzing voice samples of multiple speakers, temporally shift the features within a predetermined range with an arbitrary time as the beginning of a word, and then It is configured to select the position where the output value is maximum and output a value corresponding to the similarity of the partial phoneme sequence in a specific word.
Word spotting can be performed efficiently, and only the word nests to be recognized can be automatically extracted from continuously uttered speech without malfunctioning for sounds other than the words to be recognized. As a result, speech recognition under noise such as noise is improved.

[Brief explanation of drawings]

FIG. 1 is a block diagram showing the configuration of an embodiment of the speech recognition device of the present invention, FIG. 2 is a diagram showing the configuration of the feature vector storage section in FIG. 1, and FIG. 3 is a block diagram showing the configuration of the feature vector storage section in FIG. FIG. 4 is a diagram showing the configuration of the neural network in the speech recognition section of FIG. 1, and FIG. 5 is a diagram illustrating the selection of the maximum value of the output of the event net. 21...Microphone, 22...Amplifier, 23...
- A/D converter, 24... Acoustic analysis unit, 25... Band pass filter, 26... Compression unit, 27... Feature vector storage unit, 28 River event net, 29... Event net Group, 30...Event net output storage unit,
3I...WordNet, 32...WordNet output storage section, 33...Supernet, 34...Result determination section, 35...Result display section.

Claims (1)

[Claims] An event net that inputs the feature values obtained by acoustically analyzing input speech for each frame, and an event net that inputs the output from the event net and identifies specific words from the vocabulary to be recognized for the input speech. The event net includes a word net that outputs a value corresponding to the degree of similarity, and a supernet that inputs the output from the word net and outputs a value according to the recognized word to which the input speech belongs, and the event net includes: Based on time interval information obtained by analyzing voice samples of multiple speakers, the feature amount is temporally shifted within a predetermined range with an arbitrary time as the beginning of a word, and the feature amount shifted in time is calculated. A speech recognition device characterized in that the speech recognition device is configured to select a position where the output value is maximum and output a value corresponding to the degree of similarity of partial phoneme sequences in the specific word.