JP3467556B2 - Voice recognition device - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a voice recognition device using a neural network.

[0002]

2. Description of the Related Art Techniques that are practically used in conventional speech recognition apparatuses are roughly classified into a DP matching method and a hidden Markov model (HMM) method. These techniques are
For example, it is described in detail in "Speech Recognition by Stochastic Model" by Seiichi Nakagawa.

To summarize this, the DP matching method is
Assuming that the input data and the standard data correspond to the start end and end, the inside of the input data is transformed using various time normalization functions. Then, the deformation that minimizes the difference and the distance between the patterns at that time are defined as the points of the standard pattern. Then, of the plurality of standard patterns, the pattern having the smallest score is used as the matching result.

On the other hand, the speech recognition method using the HMM method is
It tries to perform voice recognition by a probabilistic method. In this method, an HMM model corresponding to the standard pattern in the DP method is set. One HMM model is composed of a plurality of states and a plurality of transitions.
The existence probability is given to each state, and the transition probability and the output probability are given to each transition. As a result, a certain HMM model can calculate the probability of generating a certain time series pattern.

[0005]

By the way, the characteristics of voice data greatly differ depending on the speaker. Especially men and women,
If the sexes and age groups are different, such as adults and children, even if the same sentence (or word) is pronounced, the voice data will have voice patterns with completely different characteristics. Therefore, in the conventional voice recognition device constructed by using the voice data of a specific speaker as the learning data, most of the voice data of a third party whose voice pattern is greatly different from the characteristics of the speaker for learning are recognized. could not.

An object of the present invention is to provide a voice recognition device capable of accurately recognizing voice data having different voice patterns.

[0007]

[0008]

In order to achieve the above object, the voice recognition device of the present invention is designed such that input voice data is learned in advance with voice patterns having different characteristics so as to recognize predetermined voice data. Speech recognition including a plurality of neural network units for speech recognition, which performs a speech recognition operation to determine whether or not the speech data to be recognized matches, and outputs a fitness determination data indicating the fitness of speech recognition. A processing means, a selection means for selecting the speech recognition neural network section having the highest degree of fitness of speech recognition based on the fitness determination data output from each of the speech recognition neural network sections, and the selection means. Output control means for outputting the voice recognition result from the generated voice recognition neural network unit.

Here, the voice recognition device includes a feature extraction means for extracting input voice data in frame units, converting the feature data into feature vectors and sequentially outputting the feature vectors, and each of the voice recognition neural network units includes the feature. It is preferable that the feature vector output from the extraction means is input as voice data.

Furthermore, each of the neural network units for speech recognition is constructed by mutually connecting a plurality of neurons having an internal state value X set therein, and each of the neurons has an internal state value X of the corresponding neuron. Is formed as a dynamic neuron that changes with time to a value satisfying a function X = G (X, Zj) represented by using input data Zj (j = 0 to n: n is a natural number) and internal state value X given to , Each of the dynamic neurons is preferably formed so that the internal state value X is converted into a value satisfying the function F (X) and then output.

Here, the function X = G (X, Zj
) Is

[0012]

[Equation 5]

Can be formed as represented by

Further, the function X = G (X, Zj) is such that the coupling strength Wij for coupling the output of the jth neuron to the input of the ith neuron, the external input value Di, and the bias value θi.
Using,

[0015]

[Equation 6]

It can also be expressed as:

The function X = G (X, Zj) is obtained by using the sigmoid function S,

[0018]

[Equation 7]

It can also be expressed as:

The function X = G (X, Zj) is a sigmoid function S, a coupling strength Wij for coupling the output of the jth neuron to the input of the ith neuron, an external input value Di, and a bias value θi. make use of,

[0021]

[Equation 8]

It can also be expressed as

Each of the speech recognition neural network units has an input neuron to which voice data is input, a recognition result output neuron that outputs a recognition result of voice data, and a fitness output neuron that outputs fitness determination data. The fitness output neuron is configured to estimate the voice data input to the input neuron and output the estimated data as the fitness determination data, and the selection unit is configured to output the fitness data to the actual voice data. The correct answer rate of the estimated data may be calculated as the goodness of fit of the voice recognition.

The function F (X) can be a sigmoid function.

Further, the function F (X) may be a threshold function.

Each of the dynamic neurons can be formed to include, as the input data Zj, data obtained by multiplying the output of its own neuron by a weight and feeding it back.

Further, each of the dynamic neurons can be formed so as to include, as the input data Zj, data obtained by multiplying the output of another neuron by a weight.

Further, each of the dynamic neurons can be formed so as to include desired data given from the outside as the input data Zj.

According to the voice recognition device of the present invention, the input voice data is given to the plurality of neural network units for voice recognition provided in the voice recognition means. And
In each voice recognition neural network unit, the recognition process of the input voice data and the calculation of the fitness determination data of the voice recognition between the input voice data and the voice data used for learning are performed.

Since each of the neural network units for voice recognition is previously learned so as to recognize voice data with different voice patterns, the recognition suitability also has a different value for each neural network unit.

The fitness determination data of each voice recognition neural network unit is given to the selecting means, and the voice recognition neural network unit having the highest recognition fitness is selected here. This selection result is given to the output control means, and the speech recognition result from the selected speech recognition neural network unit is output.

In this way, the voice data having different voice patterns can be accurately recognized.

Here, it is preferable that each of the neural network units for voice recognition is constructed by mutually connecting a plurality of neurons in which the internal state value X is set. The internal state value X of each neuron is the input data Zj
(J = 0 to n: n is a natural number) and the internal state value X is preferably used as a dynamic neuron that changes with time to a value that satisfies the function X = G (X, Zj) expressed using the internal state value X.

As a result, the data processing of the entire neural network section can be simplified and the voice recognition accuracy can be improved.

Further, the other speech recognition apparatus, cut out the audio data inputted in units of frames, a feature extraction means for sequentially outputs the converted feature vector, the recognition target speaker inputted from said feature extracting means The feature vector of the input recognition target speaker based on the feature vector of the speaker,
A plurality of voices that are pre-learned to be output as the fitness determination data representing the fitness of voice recognition and are formed to output the fitness determination data based on the feature vector actually input from the feature extraction means. Correct answer of voice recognition processing means including a recognition neural network section, fitness determination data output from each of the voice recognition neural network sections, and an actual speaker feature vector input from the feature extraction means. And a speaker recognizing means for recognizing the speaker of the input voice by calculating the ratio for each voice recognition neural network unit.

With the above configuration, a plurality of speakers can be accurately recognized from the input voice data.

Here, each of the neural network units for speech recognition is configured by mutually connecting a plurality of neurons in which the internal state value X is set, and in each of the neurons, the internal state value X is A function X = G (X, Z) represented by using input data Zj (j = 0 to n: n is a natural number) and internal state value X given to the neuron.
j) is formed as a time-varying dynamic neuron that satisfies
It is preferable to convert the internal state value X into a value that satisfies the function F (X) and output the value.

Each of the neural network units for speech recognition includes an input neuron to which the feature vector is input, and a fitness output neuron that outputs fitness determination data. The fitness output neuron is input. The feature vector may be estimated and the estimated data may be output as the fitness determination data.

[0039]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Next, preferred embodiments of the present invention will be described in detail with reference to the drawings.

FIG. 1 shows a preferred embodiment of the voice recognition apparatus of the present invention.

Description of Entire Voice Recognition Device The voice recognition device of the embodiment includes a feature extraction unit 10, a voice recognition processing unit 20, a selection unit 30, and an output control unit 40.

As shown in FIG. 2, the feature extracting unit 10
The input analog voice data 100 is cut out in frame units, converted into a feature vector 100, and output to the voice recognition processing unit 20. The feature vector 100 is obtained as follows. That is, as shown in FIG. 2 (A), the analog audio data 100 is converted into a predetermined frame 10
Cut out in units of 2. As shown in FIG. 2B, the features of the voice data 100 cut out in units of frames are extracted by linear prediction analysis, a filter bank, or the like, and sequentially output to the voice recognition processing unit 200 as a sequence of feature vectors 110. It

The voice recognition processing section 20 includes a plurality of neural network sections 200-1, 200-2, ... 200-.
Including k. The feature vector 110 output from the feature extraction unit 10 is input to each neural network unit.

Each of the neural network units 200-
1, 200-2 ... 200-k are learned with voice patterns having different characteristics so as to recognize predetermined voice data. Then, each of the neural network units 200-1, 200-2, ..., 200-k performs a voice recognition operation as to whether or not the voice data input as the feature vector 110 matches the voice data to be recognized, Further, it is configured to perform an operation of outputting the fitness determination data representing the fitness of the recognition.

For example, when the voice data "beer" is to be recognized, even if the same voice data "beer" is pronounced, the characteristics of the voice pattern greatly differ depending on the speaker, as described above. is there. Therefore, for example, the neural network units 200-1 and 20
0-2 is trained to recognize voice data “beer” with male voices having different voice patterns, and the neural network unit 200
-K is trained to recognize voice data "beer" by a female voice. By doing so, the neural network units 200-1, 200-
2 ... 200-k performs a voice recognition operation as to whether the input voice data matches the voice data “beer” used for learning, and outputs the recognition result 120 to the output control unit 40. . At this time, each of the neural network units 200-1, 200-2, ... 200-k calculates data for determining the fitness of the voice recognition, and the data is used as the fitness determination data 130 for the selection unit 30. Output to. The selection unit 30 controls each neural network 20.
Based on the fitness determination data 130 output from 0-1, 200-2 ... 200-k, the neural network having the highest recognition fitness is identified, and the selection data 140 is output to the output control unit 40.

The recognition conformity determination process is the conformance degree 1 between the input voice data and the voice data used for learning.
This is a process of determining 30. This determination process is a process of learning the neural network based on the input voice data so that the voice data that is previous (past) in time with respect to the input data can be estimated, and obtains the estimated correct answer rate as the recognition suitability. Say. For example, in FIG. 2, when the feature vector 110 is input to the neural network 200, the neural network 200 is trained and estimated so that the feature vector 110a that is input immediately before the input data 110 in time can be predicted. The selected feature vector is output as the fitness determination data 130 to the selection unit 30. In other words, the temporal relationship of the input data reflects the individuality of the speaker, and the voice data of the speaker, which is easy to estimate, is a phoneme or feature whose personality is similar to the voice data used in the learning of the neural network. It has a phoneme that is.

Therefore, the selection unit 30 uses the fitness determination data 130 (estimated previous feature vector) output from the neural networks 200-1, 200-2 ... 200-k as the feature extraction unit. The accuracy rate is calculated for each neural network by collating with the immediately preceding feature vector 110 actually output from 10. Since it can be said that the output of the speech recognition result of the neural network having the highest correct answer rate (recognition compatibility) is the most correct answer, the output may be adopted as the recognition result of the speech recognition device. And
The selection data 140 of the neural network having the highest recognition suitability is output to the output control unit 40.

Then, the output control section 40 recognizes the recognition data 120 of the neural network section 200 having the best compatibility specified by the selection data 140 thus input.
Is selected and output as the recognition result data 150.

As described above, according to the voice recognition device of the present invention, the voice data 100 input from a speaker having different voice pattern characteristics, such as male or female, or adult or child, is converted into the voice pattern. It can be accurately recognized without being affected by the difference of.

The above neural networks 2
00-1, 200-2 ... 200-k are the feature vectors 110 input from the feature extraction unit 10 as shown in FIG.
On the basis of the above, the feature vector 110 itself or any feature vector 100b input (future) after this feature vector 110 is predicted, and the predicted feature vector is sent to the selection unit 30 as the fitness determination data 130. You may form so that it may output toward.

Also in this case, the selection unit 30 causes the neural networks 200-1, 200-2, ... 200-
The feature vector predicted by k may be collated with the actual feature vector 110 input from the feature extraction unit 10 as a prediction target, and the accuracy rate may be calculated as the recognition suitability for each neural network. .

The neural network unit 200 used in the present invention may be a conventional static neural network represented by, for example, a hierarchical model or a Markov model, but it performs a better recognition operation with a simple structure. For this purpose, it is preferable to use a dynamic neural network as described in detail below.

Structure of Neural Network for Speech Recognition FIG. 3 shows the neural network 20 for speech recognition.
A simplified representation of an example of a dynamic neural network used as 0 is shown. The neural network 200 of the embodiment includes a plurality of neurons 200-1, 200-2, ...
00-6 are connected to each other. A weight whose size is variable is provided in the coupling portion of each neuron 200. By changing this weight to a predetermined value by learning, accurate voice recognition processing is performed.

The feature vector 2 of the voice data 100
10 is given to the neurons 210-2 and 210-3, and the recognition result data 150 of the voice recognition processing is output from the neurons 210-5 and 210-6. The neuron 210-5 outputs a negative output 158-B and the neuron 210-6 outputs a positive output 158-A. Furthermore, the neuron 210-4
Is configured to output the fitness determination data 130.

Structure of Neuron FIG. 4 schematically shows the structure of the neuron 210. The neuron 210 stores an internal state value storage means 220 for storing a predetermined internal state value X, and the internal state value X of the internal state storage means 220 with the internal state value X and an external input value Zj described below as inputs. The internal state value updating means 240 for updating and the output value generating means 260 for converting the internal state value X into the external output Y are included.

As described above, in the neural network 200 used in the embodiment, the value of the internal state value X of the neuron 210 is sequentially updated based on the value X itself. Therefore, the past history of the data input to the neuron 210 is converted and stored as the internal state value X. That is, the time history of the input is stored as the internal state value X and reflected in the output Y. In this sense, the operation of the neuron 210 of the embodiment can be said to be dynamic. Therefore, unlike the network using static neurons, the neural network 200 of the embodiment does not depend on the structure of the neural network,
The time series data can be processed, and the overall circuit scale can be reduced.

FIG. 5 shows a concrete example of the neuron 210. The internal state storage means 220 includes a memory 222 that stores an internal state value X. The internal state value updating means 240 includes an integrating means 242 for the input Zj, and a computing section 244 for computing a new internal state value X by performing the computation shown in the following equation and updating the contents of the memory 222.

[0058]

[Equation 9]

The output value generating means 260 includes an arithmetic unit 26.
Including 2. The calculation unit 262 is configured to convert and output the internal state value X stored in the memory 222 into an output value Y whose range is limited by using a sigmoid (logistic) function or the like.

At each time change of the internal state value X and the output value Y, the current internal state value is Xcurr, the updated internal state value is Xnext, and the external input value at the time of the updating operation is Zj (j Is 0 to n, and n is the number of external inputs to the neuron 210. At this time, when the operation of the internal state updating means 240 is formally expressed by a function G, it can be expressed as Xnext = G (Xcurr, Z1, ---, Zi, ---, Zn). Although various concrete forms of this expression can be considered, for example, the equation 9 using the first-order differential equation is used.
Can be shown as Where τ is a constant.

Further, as a modified form of the equation (9), the expression of the following equation (10) is also possible.

[0062]

[Equation 10]

Among them, Wij represents the coupling strength for coupling the output of the j-th neuron to the input of the i-th neuron. Further, Di represents an external input value. Further, θi represents a bias value. This bias value can be considered in Wij as a combination with a fixed value.

The internal state of the neuron 210 at a certain moment thus determined is X, and the output generating means 260 is used.
The output Y of the neuron 210 can be expressed as Y = F (X) when the operation of is expressed formally by a function F. As a concrete form of F,
A positive / negative symmetrical output sigmoid (logistic) function as shown in FIG.

[0065]

[Equation 11]

However, this function type is not indispensable, and simpler linear conversion, a threshold function, or the like may be considered.

Using such an arithmetic expression, the time series of the output Y of the dynamic neuron 320 of the embodiment is calculated by the processing shown in FIG. In FIG.
For simplicity, the neuron is simply described as a node.

The input Zj to the neuron 210 is
Is the output of the neuron itself multiplied by a certain weight, the output of another neuron multiplied by the connection weight, or an external input from other than the neural network.

In the embodiment, as shown in FIG. 3, each of the neurons 210-2 and 210-3 has a weighted output of itself, an output of another weighted neuron, and an output of the feature extraction unit 10. 110 is given. Further, the weighted output of itself and the output of another weighted neuron are given to the neuron 210-1. Furthermore, the neurons 210-4, 2
10-5 and 510-6 are given the weighted own output and the weighted output from other neurons. The output of the neuron 210-4 is given to the selection unit 30. Neurons 210-5, 210-6
Is output to the output control unit 40.

Initial Value Setting of Internal State Quantity Further, each neuron 210 of the embodiment sequentially uses the internal state quantity X stored in the internal state storage means 220 to sequentially use the internal state value updating means 240 as described above. It is configured to update. Therefore, in the neural network 200 configured by using such a neuron 210, it is necessary to set the initial value in advance before the operation.

Therefore, as shown in FIG. 1, the internal state initial value setting unit 60 is provided in the voice recognition apparatus of the embodiment. Then, the internal state initial value setting unit 60 is formed so as to give a predetermined initial value to all neurons before the neural network 200 operates. That is, the neural network 200
Prior to the operation of, the neuron 210 is set to the appropriately selected initial internal state value X, and the corresponding output Y is set. By setting the initial values in this way, the neural network will start quickly.

Learning of Neural Network Next, a learning method of the speech recognition processing of the neural network 200 will be described.

FIG. 7 shows a neural network 200.
The configuration of a learning device 300 for learning is shown. This learning device 300 is formed so as to learn each of the neural networks 200-1, 200-2, ... 200-k shown in FIG. 1 with a voice pattern having a different characteristic.

This learning device 300 is configured to learn by input data storage unit 310 in which input voice data for learning is stored, output data storage unit 312 in which model output data corresponding to the input voice data is stored. An input data selection unit 314 that selects desired input data, an output data selection unit 316 that selects output data, and a learning control unit 318 that controls learning of the neural network 200 are included.

When carrying out the learning method by the learning device 300, first, all the neurons 210 constituting the neural network 200 to be learned are to be learned.
The initial state value X is set to. Next, the voice data to be learned is selected by the input data selection unit 310,
It is input to the learning control unit 318. At this time, the learning output data corresponding to the selected learning input data is selected by the output data selection unit 316 and input to the learning control unit 318. The selected input voice data for learning is input to the voice extraction unit 10, and the feature vector 110 extracted here is input to the neural network 200 as an external input. The sum of the inputs Zj is calculated for all the neurons 210, and the internal state quantity X is updated. Then, the output Y of the neuron 210 is obtained from the updated X.

In the initial state, the neural network 2
A random value is given to the connection strength between the neurons of 00. Therefore, each neuron 210 in FIG.
-5, 210-6 output recognition results 120B, 1
20A is a random value. The weights between the neurons are slightly changed so that these outputs have the correct values.

Neural network 2 to be learned
When the voice data to be recognized is input, a signal 00 outputs a high level signal as a positive output 120A from the neuron 210-6 and a low level output from the neuron 210-5 as a negative output 120B, as shown in FIG. Learn so that the signal of is output. Thus, two types of recognition result data 120A, 1 for positive output and negative output
The reason why 20B is output is to improve the accuracy of voice recognition processing.

Then, the voice data 100 to be recognized is repeatedly input many times, and the weight between the neurons is changed little by little. As a result, the neurons 210-5, 2
The correct value will be output from 10-6. If the input voice data is learned as data that should not be recognized, the weight between the neurons is changed so that the positive output 120A becomes the low level and the negative output becomes the high level.

The number of repeated learnings until the output of the neural network 200 converges is about several thousand.

As an application of the learning method, there is a method in which two voice data are continuously input and learned. The reason is that in learning using voice data one by one, the positive output that once became high level cannot be lowered to low level, and the negative output that once became low level cannot be raised to high level. Because. That is, in the learning using the voice data one by one, as shown in FIG. 9A, the voice data to be recognized (hereinafter referred to as true data) is given to increase the positive output to a high level (in this case, learning). Negative output holds a low level), or as shown in FIG. 9 (B), learning (in this case, raising the negative output to a high level by giving data that is not desired to be recognized (hereinafter referred to as false data)) The positive output holds the low level). In this learning, there arises a problem that both the positive output and the negative output do not become the low level after the output value once rises to the high level.

Therefore, when a plurality of audio data in which true data and false data are mixed are continuously given, an affirmative output which once rises to a high level at the input of the true data has a false data input thereafter. Does not go down to low level. This also applies to the negative output.

Therefore, in this embodiment, FIG.
As shown in (D), a method is adopted in which two audio data are continuously given to perform learning of both rising and falling of the output. In FIG. 10 (A), true data and false data are continuously input, and this is repeated for learning. Through this learning, you can learn the rise of positive output and the rise and fall of negative output. In FIG. 10 (B), the false data and the true data are continuously input, and the learning is repeated. By this learning, you can learn the rise and fall of positive output and the rise of negative output. In FIG. 10C, false data is continuously input, and the learning is repeated. This learning is
This is for preventing the neural network 200 from erroneously recognizing that the data next to the false data is true data by the learning shown in FIG. Similarly, in FIG. 10D, two true data are continuously input, and the true data is repeatedly learned. This learning is also to prevent the neural network 200 from erroneously recognizing that the data next to the true data is false data by the learning shown in FIG.

Such learning is performed for the neural networks 200-1, 200-2, ... 200-k shown in FIG. 1 by using voice patterns having different characteristics. For example, each neural network 200-1, 200-2 ...
When it is desired to recognize voice data "beer" at 200-k, the voice data "beer" having voice patterns having different characteristics is used as learning voice data, and each neural network 200-1, 200- is used.
2 ... Let the learning of 200-k be performed as described above. As a result of such learning, the optimum input voice pattern for recognition is set for each neural network. Therefore, even if the same voice data 100 of "beer" is given, the recognition rate becomes different for each neural network. For example, in a voice recognition device in which the neural network 200-1 is learned by a male voice and the neural network 200-2 is learned by a female voice, when input data is given by another male voice,
The neural network 200-1 can recognize it with a high probability, but the neural network 200-2 hardly recognizes it. On the contrary, when the input data is given by the voice of another woman, the recognition rate of the neural network 200-2 is high and the recognition rate of the neural network 200-1 is low.

As described above, in this embodiment, the neural networks 200-1, 200-2, ..., 200-k are trained by the human voices having different characteristics. Is given to each neural network 200-1, 200-2 ... 200-k, the speech recognition result 120 is different for each neural network.

Each neural network 200-1, 2
00-2 ... In order to adopt the recognition result with the highest recognition rate out of the plurality of voice recognition results 120 output from 200-k, in the present embodiment, the data for recognition compatibility determination with the voice data is used. 130 is each neural network 200-
1, 200-2 ... 200-k are designed so that they are output respectively.

As described above, the recognition compatibility determination processing refers to processing for determining the compatibility 130 between the input voice data and the voice data used for learning. This determination process is a process of learning the neural network based on the input voice data so that the voice data preceding the input data in time can be estimated, and obtaining the estimated correct answer rate as the recognition suitability.

For example, in FIG. 2, when the feature vector 110 is input to the neural network 200, the neural network 200 can predict the feature vector 110a input immediately before (past) the input data 110 in time. 200 is learned, and the estimated feature vector is output to the selection unit 30 as the fitness determination data 130. In other words, the temporal relationship of input data is
The voice data of the speaker, which reflects the individuality of the speaker and is easy to estimate, has a phoneme that is a personality or a phoneme that is a feature similar to the voice data used in the learning of the neural network.

Therefore, the selection unit 30 uses the fitness determination data 130 (estimated previous feature vector) output from the neural networks 200-1, 200-2 ... 200-k as the feature extraction unit 10. Is compared with the previous one of the feature vectors 110 actually output from, and the accuracy rate is calculated for each neural network. Since it can be said that the output of the speech recognition result of the neural network having the highest correct answer rate (recognition compatibility) is the most correct answer, that output is adopted as the recognition result of the speech recognition device.

The learning of the determination process of the recognition suitability is performed at the same time as the learning of the voice recognition process described above. That is, the fitness output neuron 210-4, which is one of the elements configuring the neural network 200, is
The neutral network 200 is trained by using the speech data for learning so as to estimate the past feature vector input from 10-2 and 210-3 and output this as the fitness determination data 130. Good.

In addition to the prediction of the previous data, the recognition suitability determination process is performed by using the estimated data of the input feature vector 110 itself or the future features to be input next, as shown in FIG. It may be performed based on the prediction data of the vector 110b. However, according to the experiment, it was possible to perform the recognition operation with higher accuracy by predicting the past feature vector.

Speech Recognition Processing Operation Next, the neural network 2 configured as above will be described.
The voice recognition processing operation of 00 will be briefly described with reference to the flowchart of FIG.

First, when the voice recognition process is started, all the neurons 210-1, 210-2, ... 210-6.
Is set to an appropriately selected initial internal state value X,
The corresponding output Y is set (step 10).
1).

Next, the sum of the above-mentioned input data Zj is obtained for all neurons (step 104,
103).

Next, for each of all neurons, the sum of Zj obtained in step 103 and the internal state value X
The value of X is updated by (step 105). Then, the output value of each neuron is calculated based on the updated value of X (step 106). After this calculation, the process is returned to step 102, and if there is a command to end the process, the process ends.

The recognition result of the neural network 200 is given as the output of the neurons 210-5 and 210-6. Further, the output 130 for determining the fitness is given as the output of the neuron 210-4.

12, FIG. 13, and FIG. 14 show experimental data when a voice recognition operation is actually performed using the voice recognition device of the embodiment. In this experiment, the neural networks 200-1 and 200-2 configured as neural networks each having 20 input neurons, 2 output neurons, and 32 other neurons are used. Then, from the feature extraction unit 10 to 2
The 0-dimensional LPC cepstrum was given to each neural network 200-1, 200-2, and the data output from the neural networks 200-1, 200-2 at this time were measured.

12 (A), FIG. 13 (A), and FIG.
(A) shows the positive output 410 and the negative output 412 of the neural network 200-1. In addition, FIG.
(B), FIG. 13 (B), and FIG. 14 (B), a positive output 420 and a negative output 42 of the neural network 200-2.
2 and. Further, FIG. 12 (C), FIG. 13 (C), and FIG.
4 (C), the fitness 430 of the input voice data and the neural network 200-1, and the fitness 43 of the input voice data and the neural network 200-2.
2 and.

In this experiment, two speakers A and B having different phoneme groups are prepared, and the neural network 200
-1 is the voice of the speaker A and the neural network 200
-2 was learned by the voice of speaker B. Each of the neural networks 200-1 and 200-2 gives “for the time being” as a positive recognition target, and “end point”, “skill”, “rejection”, “transcendence”, as a negative recognition target.
Eight words were given: "classification", "rocker", "mountain range", and "hidden puritan". When a positive recognition target is given, each of the neural networks 200-1 and 200-2 changes the positive output and the negative output at the time when half of the target is recognized, so that the speaker A and the talker respectively talk. Person B's voice is used for learning. In the figure, the vertical axis represents the output value of the output neuron, and the horizontal axis represents the time flow from left to right.

Here, the experimental data shown in FIG. 12 is the result when the speech recognition apparatus thus learned is made to recognize the speech data of the speaker A. As is clear from FIG. 12A, in the neural network 200-1 learned by the voice of the speaker A, the positive output 410 changes to a large value with respect to the input of the word “for the time being”.
The negative output 412 has changed to a small value.
On the other hand, as shown in FIG. 12B, the positive output 420 and the negative output 422 of the other neural network 200-2 learned by the voice of another speaker are different from the input of the word “for the time being”. It has not changed significantly. From this, it can be seen that the neural network 200-1 correctly identifies the word “for the time being”, but the neural network 200-2 cannot. This is clear from the graph showing the determination result of the recognition suitability in FIG. Neural network 2
This is because the value of the goodness of fit 430 of 00-1 always shows a larger value than the value of the goodness of fit 432 of the other neural network 200-2.

From the above results, if the voice recognition result of the neural network 200-1 is adopted based on the determination result of the recognition suitability, a positive output and a negative output for correctly recognizing the word "for the time being" can be obtained. Be understood.

On the other hand, FIG. 13 shows data obtained when the voice recognition apparatus of the embodiment recognizes voice data input by the speaker B in the same manner.

As shown in FIG. 13A, the neural network 200-1 learned by another speaker A is a speaker B.
I can't recognize the word I entered for the time being.
On the other hand, the other neural network 200-2 that uses the voice of the speaker B for learning can accurately recognize the word “temporarily” input by the speaker B. This is
It is clear from the graph shown in (C) showing the determination result of the recognition suitability.

Also in this example, it can be seen that if the recognition result of the neural network 200-2 is adopted based on the recognition result of the recognition adaptability in the selection unit 30, a correctly recognized output can be obtained.

FIG. 14 shows data obtained when the same processing as that shown in FIGS. 12 and 13 is performed using voice data of another speaker C having different sound quality.

As is clear from FIGS. 14A and 14B, the neural network 200-1 can correctly recognize the word "for the time being" with respect to the voice data input by the speaker C. On the other hand, in the neural network 200-2, although the word “for the time being” is correctly recognized, another word “rejection” is replaced with the word “for the time being”.
I mistakenly recognize that. This is clear from the graph showing the determination result of the recognition suitability in FIG. Also in this example, based on the determination result of the recognition suitability in the selection unit 30,
It will be understood that if the recognition result of the neural network 200-1 is adopted, a correctly recognized output can be obtained.

FIG. 15 is a hardware configuration diagram of the voice recognition device of the embodiment. The voice recognition device according to the embodiment includes an analog-digital converter 7 that functions as the feature extraction unit 10.
0, a data memory 72 storing data such as the internal state value X of the neural network 200, and a CPU 76.
And a recognition processing program memory 74 in which a processing program for causing the CPU 76 to function as the selection unit 30 or the output control unit 40 is stored.

Other Embodiments The present invention is not limited to the above embodiments,
Various modifications can be made within the scope of the present invention.

Other Neuron Embodiments For example, in the above embodiment, the neural network 20 is used.
The case where the neuron 210 forming 0 is formed as the neuron having the configuration shown in FIG. 5 has been described as an example, but the present invention can use various neurons other than this.

FIG. 16 shows another dynamic neuron 2 used in the neural network 200 of the present invention.
Ten specific examples are shown.

In the dynamic neuron 210 of the embodiment, the internal state updating means 240 has an integrating section 250,
The function conversion unit 252 and the operation unit 254 are used to perform an operation based on the following equation to update the internal state quantity X of the memory 222.

[0111]

[Equation 12]

That is, the integrating section 250 integrates the input Zj, and the function section 252 is configured to convert the integrated value using the sigmoid (logistic) function S. Then, the calculation unit 254 performs the calculation of Expression 12 based on the function-converted value and the internal state quantity X of the memory 222, obtains a new internal state quantity X, and updates the value of the memory 222. Is formed in.

As a more specific operation, the operation shown in the following equation may be executed.

[0114]

[Equation 13]

In the above, Wij represents the coupling strength for coupling the output of the j-th neuron to the input of the i-th neuron. Di indicates an external input value. Further, θi indicates a bias value. This bias value can be considered to be included in Wij as a combination with a fixed value. Further, as a specific form of the range limiting function S, a sigmoid function having positive and negative symmetrical outputs may be used.

The output generation means 260 is formed as a mapping function calculation section 264 for converting the internal state value X into an output value Y which is a constant multiple.

In each of the above embodiments, the case of recognizing a word or the like as voice data has been described as an example. However, the present invention is not limited to this, and various types of phonemes and syllables may be recognized. It is possible.

[0118]

Embodiment of Speaker Recognition Type Voice Recognition Device FIG. 17 shows a preferred embodiment of the speaker recognition type voice recognition device. The members corresponding to those in the above-described embodiment are designated by the same reference numerals, and the description thereof will be omitted.

Here, the voice recognition processing section 20 includes a plurality of neural networks 200-1, 200-2 ... 200-k which recognize different speakers. Each neural network 200 predicts the speaker vector 100 of the input recognition target person based on the feature vector 110 of the recognition target speaker, and outputs it in advance as the goodness of fit determination data 130 representing the goodness of speech recognition. It has been learned (details of learning are the same as in the above-mentioned embodiment). The speaker feature amount used here is an 8th-order PARCOR coefficient. In addition to the PARCOR coefficient,
Various ones can be used. But PA
The RCOR coefficient has features such that its value is in principle a value of -1 to 1 and has a relatively high proportion depending on the speaker, and is an effective feature amount in speaker recognition.

Then, the speaker recognition unit 90 causes the neural networks 200-1, 200-2, ... 200-
The fitness determination data 130 input from k and the feature vector 10 of the actual speaker input from the feature extraction unit 10.
The correct answer rate with 0 is calculated for each neural network,
The neural network 200 with the highest correct answer rate is selected. Then, when the correct answer rate of the selected neural network is equal to or higher than the predetermined reference level, the input voice data 100 is converted into the selected neural network 2
It is determined that the speaker is a speaker used for learning 00, and this is output as a recognition result 150. For example, when the neural network 200-1 that recognizes the speaker A is selected, the input voice data 100 recognizes that the speaker is the speaker A, and outputs this as the recognition result 150. It will be.

If the correct answer rate of the selected neural network 200 is less than or equal to the predetermined standard, all the neural networks 200-1, 200-2, ...
00-k is not the recognition target speaker, and the recognition result 1
Output 50.

In addition to the speaker recognition operation, the speaker recognition section 90 may also be formed so as to recognize voice data as in the embodiment shown in FIG. In this case, the speaker recognition unit 90 includes the selection unit 30 and the output control unit 40.
Is configured to include.

Then, the selection unit 30 causes the neural networks 200-1, 200-2, ... 200-
The correct answer rate is calculated for each k and output to the output control unit 40.

The output control section 40 recognizes the speaker of each input voice data 100 on the basis of the correct rate of each input neural network. Furthermore, when there is a speaker to be recognized, the voice recognition data 120 output from the selected neural network 200.
Is output as the recognition result 150.

By doing so, not only the speaker recognition but also the voice data of the recognized speaker can be recognized at the same time, and the application field as a voice recognition device can be further expanded.

Next, the details of the actual voice recognition operation using the voice recognition device of FIG. 17 will be described. In this example, nine words were used as standard data for training the neural network: "end point", "skill", "rejection", "transcendence", "for the time being", "classification", "rocker", "mountain range", and "hidden puritan". Moreover, as the voice data, the one recorded in the Japanese voice database for research of the ATR person was used.

18 and 19 show the results of the speaker recognition experiment by the neural network 200 trained in this way. In this experiment, speaker recognition is performed by using the error between the feature vector predicted by the neural network and the actual feature vector instead of the correct answer rate.

The solid line in the figure shows the time change of the output error of the neural network learned to recognize the voice of the speaker MAU. Further, the broken line shows the time change of the output error of the neural network learned to recognize the voice of the speaker MXM. The error shown here is a value obtained by averaging the absolute value of the length of the error vector generated by comparison with the 8th-order input vector data and the output vector for 32 frames before and after the frame at that time. It is a thing. The input speaker in FIG. 18 is MAU.
The input speaker in FIG. 19 is MXM.

As is clear from the figure, in the case of FIG. 18, the data restoration error by the neural network trained by the voice of MAU is small, and the restoration error by the neural network trained by MXM is large. This is M
It is shown that the data restoration using the AU utterance feature enables more accurate restoration. That is, it indicates that the input voice is from the MAU.

Further, in the case of FIG. 19, contrary to the case of FIG. 18, the data restoration error due to the neural network trained by the MXM voice is small. That is, this indicates that the input voice is due to MXM.

As is apparent from FIGS. 18 and 19, according to the speaker recognition method of the present invention, continuous speaker recognition results can be obtained.

Table 1 below shows the average value of the error when a total of 11 voices including 9 speakers other than the training speaker are input to the two neural networks in the above example. The input is the 9 words themselves used for training. The average was performed for all the speech sections. As is clear from Table 1, in each neural network,
The error with respect to the training speaker is the smallest with respect to the voice input of 11 people, and it is shown that the training speaker is correctly recognized from the 11 people.

[0134]

[Table 1]

Table 2 below shows the same result as that of Table 1, but unlike the above case, it shows the result when a word voice having a different content from the word voice used for training is input. The words used here are "calendar,""welcome," and "extreme."
These are "parking", "program", "recording", "purchase", and "typuter".

[0136]

[Table 2]

As is clear from the above table, the speaker recognition method of the present invention can accurately recognize the training speaker even if the utterance contents of the input voice are different.

Although the above description has been made with respect to the case of being discrete in time, it is also applicable to continuous time processing by performing analog processing, for example.

[0139]

As described above, according to the inventions of claims 1 to 13, even if a plurality of voice data having different voice patterns are inputted, the neural network unit for voice recognition having the highest matching degree. Since the recognition processing is performed in step 1, there is an effect that it is possible to obtain a voice recognition device in which the recognition rate is not influenced by the voice pattern of voice data, such as sound quality and phoneme.

Particularly, by using a dynamic neuron whose internal state quantity changes with time as a neuron forming the neural network unit for speech recognition, the entire structure of the neural network unit is simplified and its recognition accuracy is improved. There is an effect that can increase.

[0141]

[Brief description of drawings]

FIG. 1 is a block diagram showing an embodiment of a voice recognition device of the present invention.

FIG. 2 is an explanatory diagram showing a conversion process in a feature extraction unit shown in FIG.

FIG. 3 is a conceptual diagram showing a configuration of a neural network unit of the embodiment.

FIG. 4 is an explanatory diagram of neurons forming a neural network unit of the embodiment.

5 is an explanatory diagram showing a specific configuration of the neuron shown in FIG.

FIG. 6 is a flowchart showing the operation of the neuron of the embodiment.

FIG. 7 is an explanatory diagram of a learning device used for learning the neural network unit of the embodiment.

FIG. 8 is an explanatory diagram showing an example of a learning method.

FIG. 9 is an explanatory diagram showing an example of a learning method.

FIG. 10 is an explanatory diagram showing an example of a learning method.

FIG. 11 is a flowchart showing a voice recognition processing operation.

FIG. 12 is an explanatory diagram illustrating an output example of voice recognition processing.

FIG. 13 is an explanatory diagram illustrating an output example of voice recognition processing.

FIG. 14 is an explanatory diagram illustrating an output example of voice recognition processing.

FIG. 15 is a hardware configuration diagram of the present embodiment.

FIG. 16 is an explanatory diagram of another specific example of the dynamic neuron used in this embodiment.

FIG. 17 is a block diagram of a voice recognition device used for speaker recognition.

FIG. 18 is a diagram showing a speaker recognition result using the voice recognition device in the example.

FIG. 19 is a diagram showing a speaker recognition result using the voice recognition device in the example.

[Explanation of symbols]

10 Feature extraction unit 20 Speech Recognition Management Department 30 Selector 40 Output control unit 100 voice data 110 feature vector 120 recognition data 130 Goodness-of-fit judgment data 140 selection data 150 recognition output 200 Neutral Network 210 neurons 220 internal state value storage means 240 internal state value storage updating means 260 output value generation means

─────────────────────────────────────────────────── ─── Continued Front Page (56) References JP-A-4-328799 (JP, A) JP-A-6-309464 (JP, A) JP-A-6-110500 (JP, A) Nakano Kaoru, “Neuro Fundamentals of Computers ", Japan, Corona Co., Ltd., April 5, 1990, first edition, pp. 44-49, 115-122, ISBN: 4-339-02276-4, Masaaki Sato, et al., "Learning of Voice Fluctuation by Recurrent Net", IEICE Technical Report, Japan, The Institute of Electronics, Information and Communication Engineers, March 19, 1991, Vol. 90, No. 484 (NC90-112 to 141), pp. 169-174 Takao Watanabe, "Application of Neural Networks to Speech Processing," System / Control / Information, Japan, Society of System Control Information, January 15, 1991, Vol. 35, No. 1, pp. 19-25, JST Material No .: G 0902A (58) Fields surveyed (Int.Cl. ⁷ , DB name) G06N 1/00-7/08 G10L 3/00 G10L 3/02 G10L 5/06 G10L 7/08 G10L 9/00-9/02 G10L 9/06-9/20 JST file (JOIS) CSDB (Japan Patent Office)

Claims (13)

(57) [Claims] 1. A voice recognition operation for preliminarily learning voice patterns of different characteristics so as to recognize predetermined voice data, and performing a voice recognition operation as to whether or not the input voice data matches the voice data to be recognized, A speech recognition processing unit including a plurality of neural network units for speech recognition for performing an operation of outputting the fitness determination data representing the degree of fitness of speech recognition, and the fitness determination unit output from each neural network unit for speech recognition. Selecting means for selecting a neural network portion for speech recognition having the highest degree of conformity for speech recognition based on the data; and output control means for outputting a speech recognition result from the neural network portion for speech recognition selected by the selecting means. A voice recognition device comprising: 2. The feature extraction means according to claim 1, further comprising: feature extraction means for extracting the input voice data in frame units, converting the feature data into feature vectors and sequentially outputting the feature vectors, wherein each voice recognition neural network unit includes the feature extraction means. A voice recognition device, characterized in that the feature vector output from is input as voice data. 3. The neural network unit for speech recognition according to claim 1, wherein each of the neural networks for speech recognition is configured by mutually connecting a plurality of neurons having an internal state value X set therein. Of the internal state value X satisfies the function X = G (X, Zj) represented by using the input data Zj (j = 0 to n: n is a natural number) and the internal state value X given to the neuron. Is formed as a time-varying dynamic neuron, and each dynamic neuron is formed so as to be output by converting its internal state value X into a value that satisfies the function F (X). Speech recognizer. 4. The function X = G (X, Zj) according to claim 3, wherein: A speech recognition device characterized by being formed as follows. 5. The coupling strength W according to claim 3, wherein the function X = G (X, Zj) couples the output of the j-th neuron to the input of the i-th neuron.
Using ij, external input value Di, and bias value θi, A speech recognition device characterized by being formed as follows. 6. The function according to claim 3, wherein the function X = G (X, Zj) is calculated by using a sigmoid function S: A speech recognition device characterized by being formed as follows. 7. The function according to claim 3, wherein the function X = G (X, Zj) is a sigmoid function S, j.
The coupling strength Wij for coupling the output of the n-th neuron to the input of the i-th neuron, the external input value Di, and the bias value θ
Using i, A speech recognition device characterized by being formed as follows. 8. The neural network unit for speech recognition according to claim 3, wherein each of the speech recognition neural network units includes: an input neuron into which speech data is input; a recognition result output neuron that outputs a recognition result of the speech data; And a fitness output neuron for outputting the fitness determination data, wherein the fitness output neuron estimates speech data input to the input neuron,
The voice recognition device is formed so as to output the estimated data as fitness determination data, and the selecting unit calculates a correct answer rate of the estimated data with respect to actual voice data as a fitness of voice recognition. 9. The voice recognition device according to claim 3, wherein the function F (X) of each of the dynamic neurons is a sigmoid function. 10. The voice recognition device according to claim 3, wherein the function F (X) of each of the dynamic neurons is a threshold function. 11. The speech according to claim 3, wherein each of the dynamic neurons includes, as the input data Zj, data obtained by multiplying an output of its own neuron by a weight and feeding it back. Recognition device. 12. The speech recognition device according to claim 3, wherein each of the dynamic neurons includes, as the input data Zj, data obtained by multiplying an output of another neuron by a weight. 13. The voice recognition device according to claim 3, wherein each of the dynamic neurons includes desired data given from the outside as the input data Zj.