JP3521844B2 - Recognition device using neural network - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION The present invention, which relates to the recognition apparatus using a two-menu neural network, in processing the time-series data such as voice data, giving start-end of the prior input data like the Instead of processing all possible combinations of start and end points, the neuron device itself is configured to retain the past history of input data, so that time series data such as voice data can be stored. The present invention relates to a technique capable of performing highly accurate processing with a simple hardware configuration.

Further, the present invention relates to a learning method of a neural network for causing the neural network to perform such processing.

[0003]

2. Description of the Related Art In the prior art, data recognizing means, particularly means practically used as means for recognizing categories of time-series data by learning, are dynamic programming (DP) methods, There are a Hidden Markov Model (HMM) method and a method using a back propagation learning method and a multilayer perceptron type neural network (MLP method). Details of these are described in, for example, Seiichi Nakagawa "Speech Recognition by Probabilistic Model" (IEICE), and Nakagawa, Kano, and Higashikura "Speech / Hearing and Neural Network Model" (Ohmsha). There is.

A problem common to the DP method and the HMM method is that the data to be a teacher and the data to be recognized require a start end and an end. In order to perform processing that does not seem to depend on the starting end, there is a method of performing processing for all possible starting ends and finding the starting end that gives the best result by trial and error. However, considering the case of detecting a portion of data belonging to a certain category from a pattern of length N, for example,
There is a possibility of the order of N at the beginning and a possibility of the order of N at the end. In other words, there is a possibility that there will be N surplus orders as a combination of the start end and the end. Therefore, in this case, the recognition process must be performed for all of these very large combinations. Then, the processing takes a huge amount of time.

Also, before the quantitative problem of the number of combinations, there is a more fundamental problem in the assumption itself of the existence of the beginning and end. The beginning and end are trivial if the input data contains only one category of data, but if one or more categories of data are continuous, such a boundary is not trivial. Absent. In particular, in time-series information such as voice, such a boundary does not exist clearly, and two consecutive categories of data change from one to the other through a transition region where the information overlaps. Therefore, assuming the beginning and end of data has a very big problem in its accuracy.

In the case of the MLP method which is another method of the conventional method, it is not necessary to particularly assume the beginning and end of such data. However, instead, a new start / end problem occurs in the sense of the input data range. That is, ML
The P method is basically a method for recognizing static data, and in order to make it recognize time series data, data in a certain time range is input as one input data, and the time information is equivalently obtained. There is a problem of having to handle. This time range must be fixed due to the structure of the MLP.

On the other hand, the length of the time-series data varies greatly depending on its category and within the same category. For example, in the case of phonemes in speech, the average lengths of vowels, which are long phonemes, and plosives, which are short phonemes, differ by a factor of 10 or more. In addition, even within the same phoneme, the length in the actual voice fluctuates more than twice. Therefore, assuming that the input range of data is set to an average length, when recognizing a short phoneme, the input data will include a large amount of data other than the recognition target, and a long phoneme will be included. When recognizing, only a part of the data to be recognized is included in the input data. All of these are factors that reduce cognitive ability. Even if a different input length is set for each phoneme, the length of the phoneme itself changes, and the problem is the same. Moreover, such a thing is generally seen in time series information.

The conventional DP method and HMM method require the start and end of data to be handled, and the MLP method requires the start and end of the input range during learning. However, in time-series information, this cannot be clarified in principle, and forcibly assuming the beginning and the end will reduce the cognitive ability.

Further, in order to apparently alleviate this, it is necessary to perform processing for all combinations of the start end and end, and a huge amount of processing is required.

[0010]

According to the present invention, the input data is
A recognition unit that performs recognition processing to determine whether or not it matches the recognition target.
Ural network and the neural network
The internal state value storage means of the neuron element that constitutes
An internal state value initialization means for giving a set initial value, and
Input background noise to neural network
From the sound input means and the output of the neural network
The equilibrium state is detected and based on the detection of the equilibrium state.
Internal state initial value
Equilibrium state detecting means for outputting a signal for changing the period value,
Each neural cell element forming the neural network includes the internal state value storage means for storing a current internal state value, the internal state value stored in the internal state value storage means and the neural cell element thereof. At least entered in
An internal state value updating means for updating the internal state value based on one weighted input value; and an output value generating means for converting the output of the internal state value storage means into an external output value. Characterize.

[0011]

[0012]

[0013]

[0014]

[0015]

[0016]

[0017]

[0018]

[0019]

[0020]

[0021]

[0022]

[0023]

[0024]

[0025]

[0026]

[0027]

[0028]

[0029]

BEST MODE FOR CARRYING OUT THE INVENTION A recognition device using a neural network according to the present embodiment is as follows: 1) Each neural cell element forming the neural network is stored in an internal state value storage means and an internal state value storage means. Internal state value updating means for updating the internal state value with the internal state value and the input value input to the neuron element, and output value generating means for converting the output of the internal state value storage means into an external output value. 2) The internal state value updating means comprises a weighted integrating means for weighting and integrating the input value and the internal state value, and the internal state value storing means integrates the values integrated by the weighted integrating means. The output value generating means includes an output value limiting means for converting the value obtained by the integrating means into a value between a preset upper limit value and a lower limit value. 3) The above 1) or ), Xi is the internal state value of the i-th neural cell element constituting the neural network, τi is a time constant, and the weighted input value to the neural cell element is Zj (j is 0 to n, n is n 0 or a natural number), the internal state value updating means

[0030]

[Equation 2]

The internal state value is updated to a value satisfying 4). 4) In the above 1) to 3), the weighted input value Zj to the i-th nerve cell element is the i-th nerve cell element itself. 5) In the above 1) to 4), the weighted input value Zj to the i-th neural cell element is the output of another neural cell element that constitutes the neural network. 6) In 1) to 5) above, the weighted input value Zj to the i-th neuron element includes data given from the outside of the neural network, 7) In 1) to 6) above, the weighted input value Zj to the i-th neuron element includes a value obtained by adding weight to a fixed value. 8) Output in 1) to 7) above The value generation means
It has a positive / negative symmetrical output range. 9) In the above 1) to 8), the neural network has at least two outputs, a positive output and a negative output. 10) In the above 1) to 9), the recognition device wants the recognition device to recognize. Speech feature extraction means for performing input feature extraction and inputting the feature extracted value to the neural network, recognition result output means for converting the output value of the neural network into a recognition result, and a neural cell element forming the neural network. 11) The internal state value initialization means for giving a preset initial value to the internal state value storage means of 11). 11) In the recognition device according to 10), background noise input means for inputting background noise to a neural network; The equilibrium state is detected from the output of the network, and the internal state is initialized based on the detection result. By providing the equilibrium state detecting means for outputting a signal to change the internal state value setting means and.

Further, the learning method of the recognition device using the neural network of this embodiment is as follows: 12) The recognition device of 10) or 11) has a learning unit for learning the neural network, and the learning unit is Input data storage means for storing learning input data,
Input data selection means for selecting learning input data from the input data storage means, output data storage means for storing learning output data, and output data selection for selecting learning output data by the selected input data and its chain. Means and learning control means for controlling the learning of the neural network while inputting the selected learning input data to the feature extraction unit, and the learning control means provides the output of the neural network and the output of the output data selecting means. 13) In 12), the input data storage means has a plurality of categories, and the output data storage means corresponds to each category of the input data storage means. The input data selection means selects a plurality of data to be learned from the category of the input data storage means, and outputs the output data. The selecting means selects the learning output data corresponding to the learning input data selected by the input data selecting means, and the learning control section connects the plurality of data selected by the input data selecting means into one input data connection. Means and output data connecting means for connecting the learning output data selected by the output data selecting means into one, and the learning section inputs the connected one learning input data to the speech feature extracting means, and the neural network. Changing the weighting of the connection of the neural cell elements based on the output of the network and the output of the output connection means, 14) the number of categories in 13) is 2, 15) learning in 12) to 14) The unit has noise data storage means for storing noise data and noise superposition means for superposing the noise selected from the noise data storage means on the selected learning data. , Learning a neural network by using input data on which noise is superimposed by noise superimposing means, 16) in 15), the position of superimposing background noise is shifted and repeated learning is performed, 17) in 15), the background is first described. It is characterized in that after learning is performed with input data on which noise is not superimposed, background noise is superimposed on the same input data for learning.

As described above, according to the recognition device and the learning method using the neural network of the present embodiment, 1) the conventional example requires a processing time proportional to the surplus of the voice input length N. However, in this embodiment, the data can be given only once, and extremely high-speed processing can be performed. 2) The memory for storing the input data may be very small. 3) It is necessary to normalize the result. 4) Easy continuous processing is possible 5) Sufficient accuracy is obtained even with integer type data representation 6) Very high accuracy recognition result is obtained by combining positive / negative output ) It is possible to output arbitrary information with more outputs, 8) It is possible to easily improve noise resistance, etc. 9) It is possible to self-organize by responding to phenomena of various time scales by learning. 10) The NN's associative ability and the information compression / expansion ability can be easily arranged optimally according to the purpose. 11) The learning is extremely easy, and the trial-and-error part for that is very small. There is.

Specific embodiments will be described in detail below.

FIG. 1 schematically shows the function of a nerve cell element (hereinafter referred to as "node") which constitutes the NN in the present invention. In the figure, 104 represents one entire node,
Reference numeral 101 denotes internal state value storage means, and 102 denotes internal state value updating means for updating the internal state value based on the internal state value stored in 101 and the input value input to the node.
Reference numeral 3 denotes an output value generating means for converting an internal state value into an external output.

FIG. 2 shows the function of the node shown in FIG. 1 more concretely. In the figure, 201 is a data input means, 202 is a weighted integrating means for weighting and integrating the data input values obtained by 201, 203 is an integrating means for integrating the integrated data values, and 204 is a result of integration. The output value limiting means for converting the obtained value into a preset value within a certain range is schematically shown.

FIG. 3 shows an example in which the configuration of FIG. 2 is an electronic circuit. In the figure, 301 indicates the data input means and weighted integrating means of FIG. 2, 302 indicates integrating means, and 303 indicates output value limiting means.

On the other hand, FIG. 28 shows the NN according to the conventional MLP method.
3 schematically shows the functions of the nodes forming the.
In the figure, reference numeral 2803 denotes one entire node, 2801 denotes an internal state value calculating means for calculating an internal state value, and 2802 denotes 28.
The output value generation means for converting the internal state value calculated by 01 into the external output is shown.

Similarly, FIG. 29 specifically shows the function of the conventional node shown in FIG. 28. In the figure, 2901 is a data input means, and 2902 is a data input value obtained by 2901. Weighted integrating means for adding and integrating
Reference numeral 2903 denotes an output value limiting means for converting the value of the integrated data into a value within a preset range.

FIG. 30 shows an example in which the configuration of FIG. 29 is replaced by an electronic circuit. In the figure, reference numeral 3001 indicates the data input means and weighted integration means of FIG. 29, and reference numeral 3002 indicates the output value limiting means.

As is apparent from FIGS. 1 to 3 and FIGS. 28 to 30, the node of the present invention has an integrating means which is not provided in the conventional node. Therefore, in the conventional node, the output is static in the sense that it is determined only by the input at that time, whereas the node of the present invention is a past one of the data input to the node. It can be said that the history is dynamic in the sense that the history is converted and held as its integrated value, and the output is determined by it.

That is, the NN using the conventional static node
In order to process time-series data with, it was necessary to capture the time structure of the data as the structure of the network, whereas the NN using the dynamic node of the present invention is
The time series data can be processed by the node itself regardless of the structure of N and the like.

More specifically, when a conventional NN attempts to process time-series data, a method of expanding the time information into spatial information, for example, inputting data input at a plurality of timings It is necessary to have a method such as collecting data. This requires hardware and processing to store and manage this summarized data. Alternatively, a special context element is needed to store the time dependent information as described above. In addition, the hardware and processing that manages this context is also required.

On the other hand, according to the NN of the present invention, since the context information and the like are stored as the integrated value inside each element, it is not necessary to set a special structure in the NN.
Therefore, for the input data, the simplest input method of inputting the data of each timing at each timing is sufficient, and no special hardware or processing for processing the time information is required.

Next, the actual operation of the node of the present invention and the NN constructed by the node will be described.
The internal state value of the node is X, the output value is Y, and the current internal state value is Xcurr, the updated internal state value is Xnext, and the input that is input to the node at the time of the update operation in the time change of X and Y. The value is Zi (i is 0 to n,
n is the number of inputs to the node). When the operation of the internal state value updating means is formally expressed as a function G, the updated internal state value Xnext is: Xnext = G (Xcurr, Z0, ..., Zi, ..., Zn) (1) Can be expressed. Although various concrete forms of the equation (1) are conceivable, for example, the following equation (2) using a first-order differential equation is also possible.

[0046]

[Equation 3]

Here, τ _i is a certain time constant.

Now, defining the input value Zj in a little more detail, the output of the node itself multiplied by a certain connection weight, the output of another node multiplied by a certain connection weight,
A fixed output value multiplied by a connection weight for biasing the internal state updating means equivalently, NN at that node
An external input, etc., which is input from the outside, can be considered. Therefore, consider updating the internal state value of the i-th node for such an input value Zj. The internal state value is Xi, the output of any node is Yj, the coupling strength that couples the output of the jth node to the input of the ith node is Wij, the bias value is θi, and the external input value to the ith node is Di is
Equation (2) can be written more concretely as follows.

[0049]

[Equation 4]

When the internal state of the node at a certain moment thus determined is X and the operation of the output value generating means is formally represented by a function F, the output Y of the node is Y = F (X) ( 4) can be expressed as As a concrete form of F, the following formula (5) is used.
A positive / negative symmetrical output sigmoid (logistic) function as shown in FIG.

[0051]

[Equation 5]

However, this function type is not indispensable, and simpler linear conversion, a threshold function, etc. may be considered.

As described above, the time series of the output Y of the NN in the present invention is calculated according to the formula.

FIG. 4 shows an example of a speech recognition apparatus using an NN constructed by the nodes of the present invention.
In the figure, reference numeral 401 is a voice feature extraction means, 402 is an NN formed by the nodes of the present invention, and 403 is a recognition result output means. The output extracted by the voice feature extraction means is input to the two nodes. Then, this NN is a fully connected NN in which an arbitrary node is connected to all other nodes. Then, two outputs from the NN are output to the recognition result output means. In the NN of the present invention, the number of outputs can be set arbitrarily. Therefore, in the case of word recognition, it is possible to improve the recognition accuracy by providing two outputs, a positive output and a negative output, and comprehensively determining the recognition result from these outputs. Of course, the number of inputs and the number of outputs to the NN are not limited to two as shown in FIG. 4, but may be any number.

5 to 9 show other examples of the structure of the NN composed of the nodes of the present invention.

First, FIG. 5 shows an example in which only the configuration of the NN 402 in FIG. 4 is changed. Here, the NN 402 is composed of an input layer 501, a hidden layer 502, and an output layer 503. This configuration looks similar to the prior art MLP method. However, the NN constructed by the node of the present invention
As in the prior art, first, the value of the input layer is determined, then the value of the hidden layer with that value as the input is determined, and subsequently, the values of each layer up to the output layer are sequentially determined. It is not a feedforward type network.

The NN using the node of the present invention recognizes time-series data without requiring a context layer as in the prior art because the node itself can hold the internal state value, and is equivalent to the prior art having a context layer. You can get the result of. Further, since the outputs of all layers are determined at the same time, more efficient parallel processing is possible as compared with the conventional MLP method.

Further, the NN using the node of the present invention is
It also has high noise resistance. FIG. 10 a) shows a conventional simple M
It shows the correspondence between the input and the output of the node in the LP method. As is apparent from the figure, when a signal in which spike-like noise is superimposed on a square wave input is given as an input, almost the same waveform appears at the output. In this way, the node of the MLP method simply reflects its input on the output and is therefore directly affected by noise.

However, the node of the present invention stores the time history as the internal state value, and the next internal state value and the output value are determined as a function of the internal state value and the input. Therefore, even if spike-like noise similar to that in a) is superimposed on the input, the spike-like waveform is blunted as shown in FIG. 10B and its influence is reduced, and good noise resistance can be obtained.

Although such noise resistance can be obtained to some extent even in the conventional technique having the context layer, the history information of a part of the nodes forming the NN is stored in an external node having a special structure. The noise resistance is inferior to that in the case of using the node in the present invention in which all the nodes retain their own history information as internal state values.

The next example is an example in which the hourglass network is configured by making the configuration of the NN of FIG. 5 more multilayered, and is shown in FIG. In the figure, 601 is a feature extraction (or information compression) network, 602 is a transmission network, and 603 is a recognition (or information decompression) network. The configuration of the NN shown in FIG. 6 is seemingly similar to the conventional MLP method. However, the operation is completely different as described above. By adopting such a configuration, feature extraction (or information compression) NN incorporating a time series effect without impairing the effects of the present invention, and a recognition network (or information decompression) incorporating a time series effect. It is also possible to configure a voice recognition method in which functions such as a network are modularized.

Next, the transmission network 602 shown in FIG.
The information transmission function 702 and the information reception function 70 shown in FIG.
This is an example divided into three. The wavy line between 702 and 703 is
It shows that these may be separated spatially and temporally.
If this wavy line indicates a spatial distance such as a transmission line,
FIG. 7 shows an audio compression transmission device, and if this wavy line indicates a temporal distance, FIG. 7 shows an audio compression recording device, for example. Of course, the object to be compressed here is not limited to voice, and more general information may be used. It goes without saying that the recognition processing is information compression processing in a broad sense.

Also in FIG. 7, the effects of the present invention described so far are not impaired. For example, due to the noise resistance described with reference to FIG. 10, the transmission error and noise are mixed in on the transmission line,
Alternatively, it shows a good posture against defects and deterioration of the recording medium.

Next, the configuration of FIG. 4 is simplified. The NN in FIG. 8 has an autoregressive loop,
It is possible to handle a phenomenon with a wider time variation range. That is, assuming that the coupling strength of the portion of the autoregressive loop in the input value Z is W, considering this autoregressive loop is equivalent to replacing the time constant τ of the system with the following equation. . τ / (1−W) (6) Since this W is a value corrected by the learning described below, the time scale of the system response can be optimized according to the learning data. In the conventional method using the context layer, such a thing cannot be self-organized by learning, and it is necessary for a human to set a network according to a time scale.

FIG. 11 is a diagram conceptually showing this effect. If there is a continuous input of square wave as shown in 11a) of the figure, if the response time constant of the system is larger than the period of this square wave, the response of the system becomes the previous output like the output of a). The next output is added, and the correct recognition result cannot be obtained.

On the other hand, in a system having an autoregressive loop as shown in FIG. 8, the time constant of the system is optimized by learning, so that its response can be modified as shown in b) of FIG. 11, for example. , You can get a good recognition rate.

By combining the time constant learning function of such a system with an appropriate learning method, the noise resistance and the like of the system of FIGS. 6 and 7 can be further improved.

As a final configuration example of the NN, an example in which the NN of FIG. 8 is a random connection NN is shown in FIG. The random connection NN 902 is composed of two sub-networks, an input network 904 and an output network 905.
In this example, the input network is a fully-coupled subnetwork, the output network is a randomly-coupled subnetwork, and two subnetworks are unidirectionally connected.

With such a configuration, in addition to the effects described above, a function of compensating for input defects by using the associative ability of the fully-coupled NN, or improving noise resistance, and one direction It is possible to obtain the effect that functions such as heuristically processing the flow of information by using the combination of (3) and compressing and decompressing information can be optimally performed as a design of the overall configuration.

The above is another example of the configuration of the NN shown in FIG. 4. Next, another example of the configuration of the speech recognition apparatus itself will be examined.

FIG. 12 is the same as FIG. 4 except that internal state initial value setting means 1204 is added to the voice recognition apparatus of FIG. As shown in the equation (2), the operation of the NN of the present invention is described by the first-order differential equation. Therefore, an initial value is required to determine the operation. The internal state initial value setting means gives a predetermined initial value to all nodes in order for the NN to operate. The operation procedure of the speech recognition apparatus will be described with reference to FIG. The internal state initial value setting means sets an appropriately selected initial internal state value X to all nodes and sets an output Y corresponding thereto. 2. If the processing is completed, it ends. 3. The sum of the input values Z is calculated at all nodes.
The input value Z is as described above, and the voice feature amount extracted by the voice feature extraction means is calculated as a part of this Z as an external input value. 4. For each node, the value of X is updated by the sum of Z obtained in 3 and the value of the internal state value X itself. 5. The output value Y is calculated from the updated value of X. 6. Return to processing 2. It becomes the procedure. The recognition result is given to the recognition result output means as an output of the node assigned to the output.

The above is the basic operating principle of the speech recognition apparatus by the NN using the node of the present invention and its configuration. It is necessary to learn the NN in order to allow such an NN to perform desired processing. Becomes Therefore, a learning method of NN will be described next.

FIG. 14 is a block diagram showing the learning method of the speech recognition apparatus of the present invention. In the figure, 1410 is the NN1402
The learning part for learning is shown. Reference numeral 1411 denotes an input data storage means in which predetermined learning input data is stored, 14
Reference numeral 13 is an output data storage means in which model output data corresponding to each learning input data is stored, 1412 is input data selection means for selecting input data to be learned from the input data storage means, and similarly 1414 is output data. Is an output data selecting means, and 1415 is a learning control means for controlling the learning of the NN.

Next, the learning method of the speech recognition apparatus by this learning unit will be described with reference to FIGS. 13 and 14. First, the initial state value X preset in all nodes
Set. Next, the input data for learning to be learned is selected by the input data selection means. The selected input data is sent to the learning control means. At this time, the output data for learning corresponding to the selected input data for learning is selected by the output data selecting means. The selected output data is also sent to the learning control means. The selected learning input data is input to the voice feature extraction means 1401, and the feature vector feature extracted here is input to the NN as an external input. The sum of the inputs Z is calculated for all the nodes, and the internal state value X is updated according to the equation (2). Then, the output Y is obtained from the updated X.

At the initial stage, a random value is given to the bond strength between the NN units. Therefore,
The output value Y output from the NN is a random value.

The above contents are repeated until the end of the input data time series. With respect to the time series of the output Y thus obtained, the learning evaluation value C is obtained by the equation shown in the following equation (7).

[0077]

[Equation 6]

Here, C is a certain learning evaluation value, and E is a certain error evaluation value. According to the equation (7), the time series of C is calculated by the processing as shown in FIG.

As a concrete example of this processing, let T be the learning output data corresponding to the selected learning input data,
If the output value corresponding to the learning input data is Y, for example, as an error evaluation function, kullback expressed by the following equation (8) is used.
Using the -leibler distance, E is

[0080]

[Equation 7]

Can be written as The use of the kullback-leibler distance has the advantage of speeding up learning due to various factors.

Further, although substantially the same as the equation (8),
When the output value generated by the output value generating means is a symmetrical output, the equation (8) is expressed by the following equation (9).

[0083]

[Equation 8]

By using these, the following equation (10) is obtained as a more specific example of the equation (7).

[0085]

[Equation 9]

By giving the above, the correction rule of the coupling strength W is given by the following equation (11).

[0087]

[Equation 10]

Here, α is a small positive constant. Accordingly, the strength of the coupling between the units is changed so that the output has the target value. By repeatedly inputting the voice data to be recognized and gradually changing the coupling strength between the units, the correct value can be output from the network. The number of repetitions until the output converges is about several thousand.

It is obvious that this learning rule can be applied not only to the fully connected neural network illustrated, but also to a more general random connected neural network including layered connections as a special example.

Next, a method for successively inputting and learning two pieces of learning input data will be described by taking the case where the NN has two outputs, a positive output and a negative output, as an example.

In the learning using the input data one by one, the positive output which once becomes high level cannot be lowered to low level. On the contrary, the negative output that has once become low level cannot be raised to high level. That is, in the learning using the input data one by one, the input data to be recognized as shown in FIG.
16) to increase the positive output to the high level (the negative output remains at the low level), or FIG.
The data that is not desired to be recognized, as shown in (b) (hereinafter,
Learning to give the negative output to the high level by giving "negative data" (the positive output remains at the low level) is performed. However, in this learning, both the positive output and the negative output do not decrease the output value that once rises to the high level.

Therefore, when a plurality of voice data in which positive data and negative data are mixed are continuously given, the positive output that has once been raised to a high level by the output of the positive data is followed by the input of the negative data. Does not go down to low level. This also applies to the negative output.

Therefore, in the present embodiment, FIG.
As shown in (d), two voice data are continuously given, and a method of learning both rising and falling of the output is used.
In FIG. 17A, the negative data and the positive data are continuously input, and the rise of the positive output and the rise and fall of the negative output are learned. In FIG. 17 (b), positive data and negative data are continuously input to let the positive output rise and fall and the negative output rise. In FIG. 17C, two negative data are continuously input so that the NN does not have an erroneous recognition that the positive data follows the negative data in the learning of FIG. 17A. Similarly, in FIG. 17D, the positive data is 2
In this case, the NN is prevented from erroneously recognizing that the positive data is followed by the negative data in the learning of FIG. 17B.

In other words, this is a problem of the initial value dependence of the operation of the NN. That is, in learning using only one input data, the learning is started from only a specific initial value, and therefore only a learning result showing the expected ability is obtained only in the initial value. In order to be able to adapt this to the more general case, it has to be trained so that an exact reaction also occurs with different initial values. However, it is not necessary to give all examples as such various initial values. At the time of actual recognition, possible combinations of initial values are limited due to various restrictions on the recognition target. The use of a chain of two or more data for learning approximately gives such a possible combination of initial values, and for this purpose even a series of two data is good enough. The result is obtained. Of course, three or more continuous data may be used.

FIG. 18 is a block diagram of a voice recognition device for making the NN learn these two continuous inputs. Here, the input data storage means described in FIG. 14 is composed of two categories of positive data and negative data.
In the figure, 1801 is an affirmative data storage unit that is a data group of words to be recognized collected under various conditions, and 1802 is a negative data storage unit that is another category other than the words to be recognized, 1803. , 1804 are output data storage means for storing the learning output data for each category. Here, it is assumed that there are three data in each category. Reference numeral 1805 denotes input data selection means, 1806
Indicates output data selecting means, 1807 indicates input data connecting means, 1808 indicates output data connecting means, 1809 indicates learning control means, and 1810 indicates NN.

The input data selection means selects two input data for learning from the positive data storage means and the negative data storage means. The combination is as described in FIG. The selected two pieces of input data become one continuous data by the input data connecting means. And
This continuous data is feature-extracted by the voice feature extraction means NN
Is input to. In the NN, the output values are calculated in time series according to the processing of FIG. The output of the NN is sent to the learning control means, the error with respect to the preselected learning output data is calculated, and the weight of the connection of each node is corrected, so that the NN repeats the learning. In FIG. 18, the output of the NN is two, that is, a positive output node and a negative output node, and
The solid line in 3 and 1804 is the learning output of the positive output node corresponding to the positive data, and the broken line is the learning output of the negative output node corresponding to the negative data.

Therefore, the recognition result of the speech recognition apparatus composed of the NN constituted by the nodes having such characteristics is
The case where learning is performed by the learning method described with reference to FIG. 18 is shown below as an example. Actually, as the output of the voice feature extraction means, 2
Assuming a 0th-order LPC cepstrum, the number of inputs is 20, the number of outputs is 2, and the others are 10, and a total of 32 nodes make N
Configured N.

First, in learning, "temporary" is used as a word to be recognized (affirmative data), and "end point", "skill" is used as other reference words (negative data).
The eight words "rejection", "transcendence", "classification", "rocker", "mountain range", and "hidden Puritan" were given. As the output of NN, an affirmative output corresponding to the above affirmative data,
We considered two types of negative output corresponding to negative data. As the learning output, the four cases described in FIG. 17 are assumed. The curve portion of this learning output has an origin at the midpoint in time of the data, and the starting end of the data is -10,
Set the sigmoid function of equation (5) with the terminal end to 10 to 0.
What was deformed in the range of up to 0.9, or what was inverted to that was used. The speakers for learning were MAU and FSU in the Japanese speech database for research of ATR Automatic Translation Telephone Research Institute.

Regarding the correspondence between the input and the output, the input for one frame (in this case, the 20th order LPC spectrum) is input and a set of positive output and negative output is obtained. Therefore, it is not necessary to input a plurality of frames of data as in the conventional case.

In the "BP model with feedback coupling" type NN, which is a modification of the MLP method of the conventional example, it is difficult to converge learning, and the learning output must be created by trial and error. However, the NN of the speech recognition method of the present invention is designed to generate a desired output by learning several hundred to several thousand times by learning by the above method. Further, the learning output can be uniquely determined without any trial-and-error part.

FIG. 25 shows the NN which has been subjected to such learning.
This is the result of verifying the ability by giving data including unknown words that were not used for learning. The total number of word types is 216 words, 9 of which are used for learning. Data of various combinations of two-word chains were created from these 216 words and used for verification. The total number of words that appear during verification is 1290 words per speaker.
The recognition result is determined by a combination of positive output and negative output. If the positive output is 0.75 or more and the negative output is 0.25 or less, it is detected, and the positive output is 0.25 or less and the negative output is 0. If it is 75 or more, it is not detected. Under this judgment condition, the insertion error is a case where a detection output is obtained at a position where there is no word to be detected, and the missing error is a case where a non-detection output is obtained at a position where a word to be detected is present.

Further, FIG. 26 below shows the same experiment as that of FIG. 25 performed for nine unknown speakers other than the speaker used for learning.

As is clear from FIGS. 25 and 26,
According to the voice recognition method of the present invention, a very good recognition rate can be obtained by learning a small amount of data.

FIG. 19 shows an example in which a word to be recognized is detected from three or more consecutive words. In the figure, the solid line shows the positive output, and the broken line shows the negative output. As is clear from the figure, it is understood that the word "temporarily" is recognized without giving a start end and an end like the conventional example.

Further, FIG. 20 shows an example in which the recognition target word "for the time being" is recognized from the unknown words. Similar to FIG. 19, the solid line shows the positive output and the broken line shows the negative output. Thus, it can be seen that the recognition method of the present invention has sufficient generalization ability.

Comparing these with the conventional example, the total length of the data given in FIG. 19 is 1049, so in the case where the conventional start and end are given and recognized, it is simply 1
It is necessary to examine the combination of 049 surplus pieces of order. However, according to the present invention, since it is only necessary to give 1049 pieces of data as input once, it is possible to process the data in several hundredth of the time compared with the conventional processing method. Further, since it is only necessary to input the data only once, it is not necessary to store the data in the range that can be the start end and the end as in the conventional case, a small amount of data memory is required, and the amount of calculation is reduced.

Further, the output does not monotonically increase or monotonically decrease as in the conventional DP method and HMM method, but has a peak value at a necessary position. Therefore, the output value is set to the length of the input data. No need to normalize. That is, the output is always in a certain range (between -1 and 1 in this example), and the weight of the value is the same everywhere in the recognition section.
This means that the dynamic range of the value to be processed is narrow, and it means that integer type data can provide sufficient performance without using floating point data or logarithmic data during processing.

Since the recognition is made by the comprehensive judgment of the two outputs of the positive output and the negative output, for example,
Even if an affirmative output starts rising at "Purchase" in FIG. 20, the negative output does not decrease, so that there is no erroneous recognition, and the accuracy of the voice recognition processing can be improved. Of course, the number of outputs is not limited to two, and any number may be provided as needed. For example, the accuracy of the recognition result can be further improved by adding an output indicating how similar the currently input data is to the data used for learning. Furthermore, by using a plurality of them, it is possible to select the NN that gives the optimum result.

The unit to be recognized may be not only a word as illustrated but also a syllable or a phoneme.
In this case, a relatively small number of NNs makes it possible to recognize the entire language speech. This allows, for example, a dictation system. Further, the recognition unit may be an abstract unit that does not consider the correspondence with the above language. The use of such a recognition unit is particularly effective when the recognition device is used for information compression.

FIG. 21 shows another embodiment.
For the voice recognition device shown in FIG.
105 and the equilibrium state detection means 2106 are added. Others are the same as that of FIG.

FIG. 22 shows a processing flow of how to determine the internal state initial value in the configuration of FIG. The portion related to the background noise data creation in the figure may not be provided as an appropriate initial value setting means, an appropriate steady input creating means, or a unit corresponding to no input. FIG. 27 shows the results obtained by learning this apparatus by the learning method shown in FIG. 18 and recognizing it, and summarizes the results corresponding to Table 1 and Table 2 of the first embodiment. This is to store the internal state value of the NN in the equilibrium state due to the background noise input for about 3 seconds as the initial value, and use that value as the initial value of the differential equation of equation (2) during the recognition process. It is a thing.

As is apparent from FIG. 27, in many cases of this embodiment, the word missing error is improved as compared with the result of the first embodiment.

In actual higher-performance voice recognition devices, linguistic processing is often used in addition to the simple voice recognition function. At this time, it is relatively easy to correct and delete an insertion error by such a linguistic constraint, but it is difficult to infer and add a missing error by such a linguistic constraint. is there. Therefore, the improvement of the loss error rate as shown in this embodiment is an important matter for realizing a higher performance speech recognition apparatus.

FIG. 23 shows an example in which noise data storage means and noise data superposition means are added to the learning section of FIG. The basic learning method is as described in FIG. The feature of this embodiment is that the data on which the noise component is superimposed in advance is used as the learning data. In the recognition processing of the learning data, the weight between the units of the NN is adjusted by the learning control unit so that the data in which the noise component included in the learning data is removed is recognized. That is, the NN is trained so that the noise component included in the learning data can be clearly identified.

Then, depending on how the noise component is superimposed on the learning data, the noise component is superimposed on the learning data at a plurality of points as shown in FIG. 240 in the figure
Reference numeral 1 indicates learning data, and reference numerals 2402 and 2403 indicate noise components. 24B is an example in which the noise component 2402 is superimposed on the front part of the learning data of FIG.
(C) shows a noise component 2403 in the latter part of the learning data.
It is an example in which is superimposed. In this way, by using the superimposition data in which the noise component is superimposed on a plurality of points of the learning data and recognizing the data in which the noise component superimposed on the learning data is recognized, the NN is Only the noise component can be clearly identified.

As a result, the NN can correctly recognize the noise portion of the voice data on which the non-stationary noise is superimposed.

As described above, the speech recognition apparatus and learning method of the present invention are very effective not only for continuous speech recognition but also for isolated speech recognition.

Further, the present invention is not limited to speech recognition and is widely effective in processing time series information, and any kind of time series information can be processed as long as input data and output data can be associated with each other. Is. Possible uses include information compression, decompression, waveform equalization, and the like.

[Brief description of drawings]

FIG. 1 is a diagram showing a nerve cell element constituting a neural network of the present invention.

FIG. 2 is a diagram in which the nerve cell element of FIG. 1 is replaced with a specific function.

FIG. 3 is an example in which the configuration of FIG. 2 is replaced with an electric circuit.

FIG. 4 is a diagram showing a recognition device using a neural network configured by using the nerve cell element of the present invention.

5 is a diagram showing the neural network of FIG. 4 in three layers.

FIG. 6 is a diagram in which the neural network of FIG. 5 is further multilayered.

FIG. 7 is a diagram showing a division of the transmission network of FIG.

FIG. 8 is a diagram showing a neural network having an autoregressive loop.

FIG. 9 is a diagram showing a random connection neural network.

FIG. 10 is a diagram for explaining noise resistance of the recognition device of the present invention.

FIG. 11 is a diagram for explaining a learning term effect of a time scale of the recognition device of the present invention.

FIG. 12 is a diagram showing the configuration of another recognition device using the nerve cell element of the present invention.

FIG. 13 is a diagram illustrating an operation procedure of the recognition device in FIG. 12.

FIG. 14 is a diagram showing a learning method of a recognition device using the neural network of the present invention.

FIG. 15 is a diagram showing a learning procedure of the learning method of the present invention.

FIG. 16 is a diagram showing connection of learning data according to the present invention.

FIG. 17 is a diagram showing a structure of learning data according to the present invention.

FIG. 18 is another diagram showing the learning method of the recognition device using the neural network of the present invention.

FIG. 19 is a diagram showing a speech word detection output by the recognition device of the present invention.

FIG. 20 is a diagram showing another speech word detection output by the recognition device of the present invention.

FIG. 21 is a diagram showing another configuration of the recognition device of the present invention.

22 is a diagram showing an operation procedure of the recognition device in FIG. 21.

FIG. 23 is a diagram showing a learning method of a recognition device having background noise superimposing means.

FIG. 24 is a diagram showing how to superimpose a noise component on learning data.

FIG. 25 is a diagram showing a recognition result when an unknown word is given to the neural network trained by the learning method of the present invention.

FIG. 26 is a diagram showing a recognition result when the same process as in FIG. 25 is performed on an unknown speaker.

FIG. 27 is a diagram showing a recognition result when the same processing as in FIG. 26 is performed by giving background noise.

FIG. 28 is a diagram showing a nerve cell device of the background art.

FIG. 29 is a diagram in which the nerve cell element of FIG. 28 is replaced with a specific function.

FIG. 30 is a diagram in which the configuration of FIG. 29 is replaced with an electric circuit.

[Explanation of symbols]

101 internal state value storage means 102 internal state value updating means 103 output value generation means 104 whole nodes 401 voice feature extraction means 402 Neural network 403 Recognition result output means

─────────────────────────────────────────────────── ─── Continuation of front page (31) Priority claim number Japanese Patent Application No. 4-159422 (32) Priority date June 18, 1992 (June 18, 1992) (33) Priority claim country Japan (JP) (31) Priority claim number Japanese Patent Application No. 4-159441 (32) Priority date June 18, 1992 (June 18, 1992) (33) Country of priority claim Japan (JP) (31) Priority claim number Japanese Patent Application No. 4-161075 (32) Priority date June 19, 1992 (June 19, 1992) (33) Priority claiming country Japan (JP) Preliminary examination (56) Reference JP-A-3-201161 (JP, A) JP-A-2-238495 (JP, A) JP-A-1-238696 (JP, A) JP-A-3-265077 (JP, A) JP-A-1-311386 (JP, A) Kaihei 4-149661 (JP, A) JP-A 5-342189 (JP, A) JP-A 3-248259 (JP, A) Shunichi Amari, "Mathematical Mathematical Neural Networks , Japan, Sangyo Tosho Co., Ltd., April 27, 1978, first edition, pp. 11-28, ISBN: 4-7828-5255-X Matsumoto, et al., "Brain and Computer 1 Neurocomputing", Japan, Baifukan Co., Ltd., January 15, 1992, first edition, pp. 1-9, ISBN: 4-563-01421-4 Kiyotoshi Matsuoka, "Excitation pattern of nerve field and generation of vibration", Computeroll, Nihon, Corona Co., Ltd., October 10, 1988, no. 24, pp. 15-21, ISB N: 4-339-02043-5 Nakano Kaoru, "Foundation of Neurocomputer", Japan, Corona Co., Ltd., April 5, 1990, first edition, pp. 44-49, 115-122, ISBN: 4-339-02276-4 Shunichi Amari, "PDP Model", Japan, Sangyo Tosho Co., Ltd., February 27, 1989, first edition, pp. 325-334, ISBN: 4-7828-5125-1 Nakagawa Seiichi et al., "Voice recognition using sequential neural network", The Institute of Electronics, Information and Communication Engineers, Japan, The Institute of Electronics, Information and Communication Engineers, 1991. September 25, Vol. J74-D-II, No. 9, pp. 1174-1183 Kazuyuki Aihara, "Neural Computer", Japan, Tokyo Denki University Press, April 30, 1988, first edition, pp. 98-105, ISBN: 4-501-51320-9 Noboru Kindera et al., "Phonological segmentation of continuous speech by neural network", IEICE Transactions, Japan, The Institute of Electronics, Information and Communication Engineers, 1990. January 25, Vol. J73-D-II, No. 1, pp. 72-79 Watanabe Tatsumi et al., “Study on each learning rule of recurrent neural network and shape of learning surface”, IEICE Transactions, Japan, The Institute of Electronics, Information and Communication Engineers, December 25, 1991, Vol. J74-D-II, No. 12, pp. 1776-1787 Yoshihiro Futami et al., “Self-organizing neural circuit model for analyzing and integrating vowel patterns”, IEICE Technical Report, Japan, 1986, Vol. 85, No. 331, pp. 261-266, JST Material No .: S 0532B Yoshihiro Futami et al., "Time Series Processing Capability of Mutually Coupled Neural Networks", Technical Report of IEICE, Japan, The Institute of Electronics, Information and Communication Engineers of Japan, March 19, 1991, Vol. 90, No. 484 (NC90-112 to 141), pp. 31-36 Kazuyuki Aihara, "Small Feature: Neural Computing I. General", The Institute of Electrical Engineers of Japan, Japan, The Institute of Electrical Engineers of Japan, June 20, 1989, Vol. 109, No. 6, pp. 427-433, ISSN: 0020-2878 Yuki Kage et al., "Associative Memory Model Using Fatigue Effect", IEICE Technical Report, Japan, The Institute of Electronics, Information and Communication Engineers, January 18, 1992, Vol. 91, No. 414 (NC91-82 to 97), pp.79-86 Naoki Mitsuya, "Improvement of recall ability in mutual connection type neural network that memorizes arbitrary patterns", IEICE technical report, Japan, incorporated association IEICE, March 18, 1991, Vol. 90, No. 483 (NC90-68 to 111), pp. 125-130 (58) Fields investigated (Int.Cl. ⁷ , DB name) G06N 1/00-7/08 G10L 3/00-9/20 JST file (JOIS ) CSDB (Japan Patent Office)

Claims (10)

(57) [Claims] 1. A recognition neural network for recognizing whether or not input data matches a recognition target, and an initial value preset in an internal state value storage means of a neuron element forming the neural network. An internal state value initialization means for giving a background noise input means for inputting background noise to the neural network; an equilibrium state is detected from the output of the neural network; and an internal state initial value setting based on the detection of the equilibrium state. A balanced state detecting means for outputting a signal for changing an initial value of an internal state preset in the means; each of the nerve cell elements constituting the neural network stores the current internal state value; A value storage means, an internal state value stored in the internal state value storage means and its nerve cell element. Internal state value updating means for updating the internal state value based on at least one weighted input value, and output value generating means for converting the output of the internal state value storage means into an external output value. A recognition device using a neural network characterized by the above. 2. The internal state value updating means comprises a weighted integrating means for weighting and integrating the input value and the internal state value, and the internal state value storage means is a value integrated by the weighted integrating means. And an output value limiting means for converting the value obtained by the integrating means into a value between a preset upper limit value and a lower limit value. A recognition device using the neural network according to claim 1. 3. The internal state value of the i-th element for nerve cells constituting the neural network is Xi,
Letting τi be a time constant, the weighted input value to the neuron element is Zj (j is 0 to n, n is 0 or a natural number).
Then, the internal state value updating means is The internal state value is updated to a value that satisfies the following condition. 3. The recognition device using the neural network according to claim 1 or 2. 4. The weighted input value Zj to the i-th nerve cell element includes a value obtained by adding a weight to the output of the i-th nerve cell element itself. A recognition device using the neural network according to claim 3. 5. The weighted input value Zj to the i-th neural cell element includes a value obtained by adding weights to the outputs of other neural cell elements constituting the neural network. A recognition device using the neural network according to any one of claims 1 to 4. 6. The weighted input value Zj to the i-th element for nerve cells includes data given from outside the neural network, according to claim 1. A recognition device using the described neural network. 7. The weighted input value Zj to the i-th neuron cell element includes a value obtained by adding weight to a fixed value, according to any one of claims 1 to 6. A recognition device using the neural network described in 1. 8. The recognition device using a neural network according to claim 1, wherein the output value generation means has a positive / negative symmetrical output range. 9. The speech recognition neural network carries out a recognition process as to whether the input data is positive data to be recognized or other negative data, and when positive data is input, the output is in the first state, When the negative data is continuously input, the positive output whose output changes from the first state to the second state, and when the negative data is input, the output becomes the third state, and the positive data is continuously input. The recognition result is output as a combination of at least two categories of a negative output in which the output changes from the third state to the fourth state when output is performed. A recognition device using the neural network according to claim 1. 10. The recognition device is a voice recognition device, which extracts a feature of an input to be recognized and inputs a feature-extracted value to the neural network, and an output value of the neural network. 10. A speech recognition device using a neural network according to claim 1, further comprising a recognition result output means for converting into a recognition result.