JPH0798694A - Neural network simulator - Google Patents

Abstract

PURPOSE:To clarify a potential function for guaranteing the convergency of learning, to obtain a sufficient learning speed on practice, to smoothly approximate an object function, and to obtain a structure suitable to the approximation of the object function, in a neural network simulator used for pattern recognition or the like. CONSTITUTION:This device is provided with a divider 12 which specifies the output signals of plural local base output calculating circuits 10 having a value for an output which is not O for a limited input according to the total sum, adder 14 which outputs the total sum of the products of the specified local base output signals and a coupling summation as an output signal, coupling summation correcting circuit 15 which corrects the coupling summation from a teaching signal or the like, local base correcting circuit 16 which corrects the inside parameter of the local base output calculating circuit 13, coupling summation setting circuit 17 which sets the coupling summation, and local base setting circuit 18 which sets the inside parameter of the local base output calculating circuit 10.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a neural network simulating device used for pattern recognition such as voice and image, and adaptive control.

[0002]

2. Description of the Related Art In recent years, neural network simulation devices to which neural networks have been applied have come to be widely applied to voice recognition and the like. The neural network simulator that applies the neural network is
There are roughly two types. One is the literature (PD
P model, DE Ramelhart and 2 others, translated by Shunichi Amari, 1
989), which is referred to as a multi-layer perceptron (MLP), and the other is a self-organization and associative memory (Self-Organization and Associative Memories).
mory, T. Kohonen, Springer-Verlag, 1987)
This is called the learning vector quantization (LVQ) described in. Most of the purpose of such a neural network simulator is to acquire (learn) an input / output relationship from the input presented and the teacher signal (ideal output signal). Since the neural network simulation device has this function, it can be used even when the input / output relationship is not easy to describe by the conventional rule, that is, even when pattern classification or control value estimation is difficult. It is an elemental technology that can realize an adaptive control device.

A more mathematical expression of the purpose of such a neural network simulator is described in the literature (Competitive learning type neural circuit model reflecting the structure of the output signal, Yutaka Sakaguchi, Noboru Murata, IEICE Technical Committee Report NC89. -54, 1989)
, "A system that has a neural network structure is used to realize an arbitrary continuous nonlinear function by sequential learning." From this point of view, the MLP is caught by the minimum value of the error function in the course of learning, and the learning does not necessarily converge, as summarized in the same document. Since the LVQ forms an intermediate layer element using unsupervised learning, the neural network obtained as a result of learning does not necessarily have a structure suitable for realizing a target function (objective function). In the same document, Sakaguchi et al. Propose the following learning method that forms an intermediate element that reflects the properties of the objective function and better approximates the objective function.

FIG. 9 is a block diagram showing the configuration of a conventional neural network simulation device proposed by Sakaguchi et al. In FIG. 9, 1-i (i = 1 to 4) is a distance calculation circuit that receives the input signal x, calculates the distance to the reference vector Wi held therein, and outputs the distance signal Di, and 2 is the distance. Upon receiving the distance signals Di (i = 1 to 4) from the calculation circuits 1-1 to 4, the minimum distance number signal IDmin indicating the number of the distance calculation circuit that gives the smallest distance signal and the second smallest distance signal are given. Second distance number signal I indicating the number of the distance calculation circuit
Competing circuit outputting Dsec, 3 is stored in the address designated by the minimum distance signal IDmin from the memory allocated in accordance with the number of the distance calculation circuit held internally upon receiving the minimum distance number signal IDmin from the competing circuit 2. An output calculation circuit that reads out the calculated value (or vector) and outputs it as an output signal y. Reference numeral 4 denotes a difference signal E = between the teacher signal z given from the teacher signal input terminal and the output signal y from the output calculation circuit 4. z-y and the minimum distance number signal IDmin from the competing circuit 2 are combined weight correction circuits for correcting the memory of the output calculation circuit 3, and 5 is a difference signal E, a minimum distance number signal IDmin and a second distance number signal. A reference vector correction circuit that receives IDsec and the input signal x, and corrects the reference vector of the distance calculation circuits 1-1 to 4.

The operation of the neural network simulation device configured as described above will be described below. First, the distance calculation circuit 1-i (i = 1 to 4) receives the input signal x from the input terminal, calculates the distance signal Di indicating the distance to the reference vector Wi held therein by the following equation. ,Output.

[0006]

[Equation 1]

The competition circuit 2 is a distance calculation circuit 1-
Minimum distance number signal indicating the number of the distance calculation circuit that receives the distance signal Di from i and gives the minimum distance signal

[0008]

[Equation 2]

And a second distance number signal IDsec indicating the number of the distance calculation circuit which gives the second smallest distance signal. The output calculation circuit 3 receives the minimum distance number signal IDmin from the competing circuit 2, and specifies the minimum distance number signal IDmin from the value v (i) stored in the internal storage corresponding to each distance calculation circuit 1-i. Value v (IDmi
n) is read and output as an output signal y.

The learning operation is performed as follows. The connection weight correction circuit 4 receives the difference signal E between the output signal y and the teacher signal z and the minimum distance number signal IDmin, and receives the minimum distance number signal ID stored in the internal storage of the output calculation circuit 3.
The value v (IDmin) specified by min is modified to v (IDmin) + c2 · E. Here, c2 is a parameter that determines the learning speed. The reference vector correction circuit 5 receives the difference signal E between the output signal y and the teacher signal z, the minimum distance number signal IDmin and the second distance number signal IDsec, and receives the distance calculation circuit 1
The reference vector WIDmin stored inside IDmin and the reference vector WIDsec stored inside the distance calculation circuit 1-IDsec.

[0011]

[Equation 3]

Modify as follows. Here, c1 is a parameter that determines the learning speed. As described above, by learning not only the reference vector of the minimum distance but also the reference vector of the adjacent area specified by the second distance number signal, the minimum distance number signal can be changed as shown in FIG. The designated area does not move much by learning, and the size of the area becomes smaller instead. At this time, the greater the degree to which the reference vector moves, the shorter the distance between the two reference vectors, and the smaller the area. Therefore, if the reference vector is moved in a size proportional to the absolute value of the error, the regions become smaller as the function slopes steeper due to the pulling of the reference vectors, and the reference vectors are densely distributed. A structure suitable for function approximation is realized.

[0013]

However, although the above-mentioned conventional structure provides a neural network simulating device that can obtain a structure suitable for approximating the objective function, as Sakaguchi et al. Point out in the literature, Since the potential function related to the learning rule of Sakaguchi et al. Is not clear, the guarantee of the convergence of learning and the property of the learning equilibrium solution cannot be generally discussed. Further, the learning speed cannot be said to be superior to MLP, and a practically sufficient learning speed cannot be obtained. In addition, since the approximation of the function is based on the step function, there is a problem that it is difficult to use it for control.

The present invention solves the above-mentioned conventional problems. It has a clear potential function that guarantees the convergence of learning and the like, a practically sufficient learning speed is obtained, and the objective function can be smoothly approximated. It is intended to realize a neural network simulation device that can obtain a structure suitable for function approximation.

[0015]

In order to achieve this object, the neural network simulator of the present invention uses a function (local basis function) that locally has a non-zero value in the input signal space. A local basis output calculation circuit that receives and calculates the respective local basis output signals by using a plurality of local basis functions (functions that locally have a non-zero value), and a total sum of the received local basis output signals. An adder that outputs a total local base output signal, a divider that outputs a standardized local base output signal using the total local base output signal for each of the local base output signals, and a standardized local from the divider A local output calculation circuit that receives a base output signal and outputs a product of the combination weight internally held as a local output signal and a total sum of the plurality of local output signals corresponding to each local base output signal are output as an output signal. An adder, a combined weight correction circuit that corrects the combined weight of the corresponding local output calculation circuit by receiving the difference signal between the teacher signal and the output signal, and the normalized local basis output signal, and the input signal, the output signal, and the difference signal And a local basis correction circuit that receives the normalized local basis output signal from the divider and corrects the internal parameter of the corresponding local basis output calculation circuit, the output signal, the teacher signal, and the total local basis output signal from the adder. In response to this, a connection weight setting circuit that sets the connection weight of the local output calculation circuit, and a local that sets the internal parameter of the local basis output calculation circuit based on the input signal, the normalized local output signal, and the internal parameter of the local output calculation circuit It has a configuration including a base setting circuit.

[0016]

With this configuration, the output signal y is generated by the combination of the elementary function using the input signal x and some parameters, and the potential function C for the learning rule of the neural network simulator is

[0017]

[Equation 4]

Can be written as By using this, the convergence of learning can be easily analyzed. In addition, a structure suitable for the objective function can be obtained by learning about tens to hundreds of times. In addition, as is apparent from the configuration of the present invention, normally, a plurality of local output calculation circuits are zero for one input signal.
However, since the output signal y is given as the total sum, the smooth function approximation can be performed, unlike the neural network simulator of LVQ and Sakaguchi et al. Further, when the error is large, the teacher signal z provided with the nerve cell simulating element that works only near the input signal x is output as the output signal y while keeping the information stored in the parameters accumulated in the past as it is. By activating, speeding up of learning can be realized.

[0019]

【Example】

(First Embodiment) A first embodiment of the present invention will be described below with reference to the drawings.

In FIG. 1, reference numeral 10 is a local basis output calculation circuit that receives an input signal x and calculates and outputs a local basis output signal S, and 11 is a plurality of local basis output calculation circuits 10.
Is an adder that receives the local basis output signal S from
Is a divider that receives the local basis output signal S and the total local basis output signal Stotal from the local basis output calculation circuit 10 and the adder 11, and outputs a normalized local basis output signal b, 1
Reference numeral 3 denotes a local output calculation circuit that receives a normalized local base output signal from the divider 12 and outputs a product of the combined weight v internally held as a local output signal o, and 14 represents a plurality of local output calculation circuits 13. Is an adder for outputting the total sum of the local output signals o of the output signals y as the output signal y, and 15 receives the difference signal E between the teacher signal z and the output signal y and the normalized local base output signal b from the divider 12 and responds. Local output calculation circuit 13
Is a combination weight correction circuit for correcting the combination weight v of
Is a local basis correction circuit that receives the input signal x, the output signal y, the difference signal E, and the normalized local basis output signal b from the divider 12 and corrects the internal parameter of the corresponding local basis output calculation circuit 10. Is a combination weight setting circuit that receives the output signal y, the teacher signal z, the difference signal E, and the total local base output signal Stotal, and sets the combination weight of the local output calculation circuit 13, and 18 is the difference signal E and the input signal x. It is a local basis setting circuit that sets internal parameters of the local basis output calculation circuit 10 based on the normalized local basis output signal b and the internal parameters of the local output calculation circuit 10.

The operation of the neural network simulation device configured as described above will be described. Note that, here, a case will be described in which a Gaussian function is used as a local basis function that characterizes the property of the local basis output calculation circuit 10.

First, the input signal x enters the local basis output calculation circuit 10. In the i-th local basis output calculation circuit,
The local basis output signal S (i) is

[0023]

[Equation 5]

Is calculated and output. Where a (i)
Is an activation parameter with a value of 0 or 1 and μ (i)
Is the reference vector and σ ² (i) is the variance. Adder 1
1 is the sum of all the local basis output signals S (i) output from the local basis output calculation circuit 10.

[0025]

[Equation 6]

Is calculated to output the total local basis output signal Stotal. The i-th divider 12 receives the local basis output signal S (i) from the i-th local basis output calculation circuit 10 and the adder 11.
And the total local basis output signal Stotal, the normalized local basis output signal b (i)

[0027]

[Equation 7]

Is calculated and output. The i-th local output calculation circuit 13 calculates the local output signal o (i) from the normalized local basis output signal b (i) from the divider 12 and the joint weight v (i) held therein by the following equation. To calculate and output.

[0029]

[Equation 8]

The adder 14 receives the local output signal o from the local output calculation circuit 13, calculates the output signal y according to the following equation, and outputs it.

[0031]

[Equation 9]

Next, the learning operation will be described. When the teacher signal z is given to the input signal x, first, the difference between the output signal y calculated from the input signal x and the teacher signal z is calculated, and the difference signal E is generated. The difference signal E is input to the combined weight correction circuit 15, the local basis correction circuit 16, the combined weight setting circuit 17, and the local basis setting circuit 18.

When the absolute value of the difference signal E is smaller than the threshold value θ E or the number Na of active parameters whose value is 1 is equal to the number N of the local basis output calculation circuit 10, the combination weight correction circuit 15 and the local weight correction circuit 15 are used. The base correction circuit 16 operates as follows.

The i-th (i = 1 to Na) joint weight correction circuit 15 also receives the teacher signal z and the normalized local basis output signal b (i) from the i-th divider, and calculates the i-th local output. The connection weight v (i) of the circuit 13 is modified as follows.

[0035]

[Equation 10]

On the other hand, the i-th local basis correction circuit 16 is
The input signal x and the normalized local basis output signal b (i) and the output signal y are received from the i-th divider, the combined weight v (i) is read from the local basis output calculation circuit 13, and the i-th local basis is read. The internal parameters of the output calculation circuit 10 are modified as follows.

[0037]

[Equation 11]

Next, when the absolute value of the difference signal E is larger than the threshold value θ E and the number Na of active parameters whose value is 1 is smaller than the number N of the local basis output calculation circuit 10, the joint weight correction circuit 15 The local basis correction circuit 16 does nothing.

In the combination weight setting circuit 17 and the local basis setting circuit 18, whether the absolute value of the difference signal E is smaller than the threshold value θE,
Alternatively, when the number Na of activation parameters whose value is 1 is equal to the number N of local basis output calculation circuits 10, nothing is done.

Next, when the absolute value of the difference signal E is larger than the threshold value θE and the number Na of active parameters whose value is 1 is smaller than the number N of the local basis output calculation circuits 10, the combination weight setting circuit 17 The local base setting circuit 18 operates as follows.

The joint weight setting circuit 17 sets the joint weight v (Na + 1) of the Na + 1-th local output calculation circuit 13 as follows.

[0042]

[Equation 12]

On the other hand, the local basis setting circuit 18 also receives the input signal x and the normalized local basis output signal b from the divider,
The parameters of the local basis output calculation circuit 10 are read out and N
The internal parameters of the (a + 1) th local basis output calculation circuit 10 are set as follows.

[0044]

[Equation 13]

C2 is a parameter having a positive value. However, in the initial state where Na is 0, σ (Na + 1) = σ0 and a fixed value σ0 are given.

The value of c2 is preferably set to a value of about 0.1 to 1.0, but this parameter has considerable arbitrariness and is not limited to this range. Regarding the initial settings, the first two data (x1, z1),
From (x2, z2),

[0047]

[Equation 14]

The above method may be used. According to the neural network simulator of this embodiment, the output signal y according to this embodiment is generated by the combination of the elementary functions using the input signal x and some parameters, and the potential function C is

[0049]

[Equation 15]

Can be written as By using this, the convergence of learning can be easily analyzed. Further, as shown in FIG. 2, a structure suitable for the objective function can be obtained by learning several tens to several hundreds. According to this embodiment, the reference vector is distributed in a large area in which the first derivative of the objective function is large. In addition, as is apparent from the configuration of the present invention, normally, a plurality of nerve cell simulating elements are set to 0 for one input signal x.
However, since the output signal y is given as the total sum, the smooth function approximation can be performed, unlike the neural network simulator of LVQ and Sakaguchi et al. Further, when the error is large, the teacher signal z given a new neuron-mimicking element that works only near the input signal x without changing the information stored in the parameters accumulated in the past is used as the output signal y. The speeding up of learning is realized by this.

Although a Gaussian function is used as the basis function here, D is well known in the field of neurophysiology as long as this function has a non-zero value locally.
A function obtained by multiplying a Hermitian polynomial known as an OG function, a Gabor function, or a harmonic oscillator wave function in quantum mechanics by a Gaussian function, a wavelet transform basis function, or the like can be used. Even when such a function is used, in the present invention, since the potential function for the learning rule is given, the parameter correction amount can be easily calculated by taking the derivative with respect to the function parameter. This is the same in the following embodiments, and will not be repeated in the following embodiments.

(Second Embodiment) A second embodiment of the present invention will be described below with reference to the drawings.

FIG. 3 shows another configuration of the neural network simulation device having the same function as that of the first embodiment. As mentioned in the first embodiment, the output signal y of the neural network simulator of the present invention is expressed by a combination of elementary functions. Therefore,
Some freedom remains with respect to changing the order of operations. This embodiment shows this.

In FIG. 3, reference numeral 10 is a local basis output calculation circuit that receives the input signal x and calculates and outputs the local basis output signal S, and 11 is a plurality of local basis output calculation circuits 10.
Is an adder that receives the local basis output signal S from
Is a local output calculation circuit that receives the normalized local base output signal from the local base output calculation circuit 11 and outputs the product of the internally weighted combined weight v as a local output signal o, and 14 is a plurality of local output calculation circuits. Reference numeral 12 denotes an adder that outputs the total sum of the local output signals o from 13 as a non-standardized output signal ny, and 12 represents the non-standardized output signal ny and the total local base output signal Stotal from the adder 14 and the adder 11. A reference numeral 19 denotes a divider which receives and outputs the output signal y, and 19 receives the local basis output signal S and the total local basis output signal Stotal from the local basis output calculation circuit 10 and the adder 11, and outputs the normalized local basis output signal b. Is a divider for outputting the difference signal E between the teacher signal z and the output signal y and the normalized local basis output signal b from the divider 19 and corresponding local output calculation circuit 1
A combination weight correction circuit for correcting the combination weight v of 3
6 is an input signal x, an output signal y, a difference signal E, and a divider 19
Is a local basis correction circuit that corrects the internal parameters of the corresponding local basis output calculation circuit 10 by receiving the standardized local basis output signal b from Reference numeral 18 denotes a combination weight setting circuit for setting, which is an input signal x and a normalized local basis output signal b.
And a local basis setting circuit for setting internal parameters of the local basis output calculating circuit 10 based on the internal parameters of the local output calculating circuit 10.

The operation of the neural network simulation device constructed as described above will be described. Note that, here, a case will be described in which a Gaussian function is used as a local basis function that characterizes the property of the local basis output calculation circuit 10.

First, the input signal x enters the local basis output calculation circuit 10. In the i-th local basis output calculation circuit,
The local basis output signal S (i) is calculated and output as follows.

[0057]

[Equation 16]

Here, a (i) is an activation parameter having a value of 0 or 1, μ (i) is a reference vector, and σ ²
(i) is the variance. The adder 11 calculates the total sum of all the local basis output signals S output from the local basis output calculation circuit 10 according to the following equation and outputs a total local basis output signal Stotal.

[0059]

[Equation 17]

The i-th local output calculation circuit 13 calculates the local output signal o (i) from the normalized local base output signal b (i) from the divider 12 and the joint weight v (i) held inside. Calculate and output as the following formula.

[0061]

[Equation 18]

The adder 14 receives the local output signal o from the local output calculation circuit 13, calculates the non-standardized output signal ny according to the following equation, and outputs it.

[0063]

[Formula 19]

The divider 12 includes the denormalized output signal ny and the total local base output signal Stotal from the adder 14 and the adder 11.
In response, the output signal y is calculated according to the following equation and output.

[0065]

[Equation 20]

Next, the learning operation will be described. When the teacher signal z is given to the input signal x, first, the difference between the output signal y calculated from the input signal x and the teacher signal z is calculated, and the difference signal E is generated. The difference signal E is input to the combined weight correction circuit 15, the local basis correction circuit 16, the combined weight setting circuit 17, and the local basis setting circuit 18. The divider 19 receives the total local basis output signal Stotal and the local basis output signal S from the adder 14 and the local basis output circuit 10, and outputs the normalized local signal b. The following operation is the same as that of the first embodiment, and thus the description is omitted. The same applies to the effects and the like.

(Third Embodiment) A third embodiment of the present invention will be described below with reference to the drawings. Prior to the description of the configuration of this embodiment, the difference between this embodiment and Embodiments 1 and 2 will be described.

In the first and second embodiments, the function of the local output calculation circuit 13 is to multiply the standardized local base output signal b or the local output signal S by the coupling weight v which does not depend on the input signal x. In this embodiment, this is expanded to include a term depending on x. That is, the local output signal o (i)
To

[0069]

[Equation 21]

It is extended with Along with this expansion, the weighted coupling setting circuit 17 is the only component that requires a major function change. In view of this, in FIG. 4, which is a block diagram showing the configuration of the present embodiment, other components are collectively represented by 30 in order to avoid complication.

In FIG. 4, reference numeral 30 is a block including all of the neural network simulating apparatus of the present invention except the connection weight setting circuit 17 which receives the input signal x and outputs the output signal y and the normalized local basis output signal b. , 20 is a combination weight setting control unit for controlling the combination weight setting based on the difference signal E.
Reference numeral 1 is a reference vector signal generator that reads the reference vector μ from the block 30 and outputs it as an input signal to an input line, and 22 is an input signal x (or μ), a normalized local basis output signal b, and a teacher signal z. A signal storage unit that stores the output signal y, and 23 is a combination weight setting unit that reads out data from the signal storage unit 23 as necessary to calculate and set a connection weight.

The operation of the neural network simulating device configured as described above will be described. When the input signal x is input, signal processing is performed as in the case of the first or second embodiment, and the output signal y is output. However, i-th (i = 1
~ Na) in the local output calculation circuit 13, the local output signal o
(i) is calculated as follows.

[0073]

[Equation 22]

And the local output signal o (i) is calculated. A difference signal E between the output signal y and the teacher signal z is calculated and sent to the combination weight setting control unit 20 of the combination weight setting circuit 17.
The connection weight setting control unit 20 determines the number N of active parameters whose absolute value of the difference signal E is larger than the threshold value θE and whose value is 1.
When a is equal to the number N of local basis output calculation circuits 10,
The connection weight setting control signal C is turned on and sent to the reference vector signal generator 21. The reference vector signal generator 21 determines from the block 30 that the activation parameter a is 1 when the connection weight setting control signal C is ON.
The reference vector μ is read one by one and sent as an input signal to the input line, and when the processing for all reference vectors is completed, a signal generation end signal is sent to the combination weight setting unit 23. The block 30 outputs the output signal y (μ) and the standardized output signal b (i, μ). When the coupling weight setting control signal C is ON, the signal storage circuit 22 receives the input signal x (or μ), the output signal y (or y (μ)), the normalized local basis output signal b (i, μ), and the total local. Output signal Stotal (or Stotal
(μ)) The teacher signal z is additionally recorded.

Upon receiving the signal generation end signal from the reference vector signal generation unit 21, the connection weight setting unit 23 receives the signals stored in the signal storage unit 22, x, y, z, μ, y (μ), S.
The combined weight of the Na + 1-th local output calculation circuit 13 is calculated from total, Stotal (μ), b (i, μ) as follows.
Although the case where the input signal is two-dimensional is described here, this can be similarly calculated regardless of the number of input dimensions. The connection weight setting unit 23 calculates the connection weights v (Na + 1) and w (Na + 1) of the Na + 1th local output calculation circuit 13.

[0076]

[Equation 23]

Set as follows. Finally, the signal storage unit 22 is reset. The operation of the local base setting circuit 18 is as described in the first embodiment.
Same as 2.

When the number Na of active parameters whose absolute value of the difference signal E is larger than the threshold value θ E and whose value is 1 is equal to the number N of the local basis output calculation circuits 10, the connection weight setting controller 20 determines the connection weight. The setting control signal C is turned off and sent to the reference vector signal generation unit 21 and the signal storage unit 22. The reference vector signal generator 21 receives the combined weight setting control signal C.
If is OFF, do nothing. The signal storage unit 22 resets the stored contents when the connection weight setting control signal C is OFF.

When the absolute value of the difference signal E is smaller than the threshold value θ E or the number Na of active parameters whose value is 1 is equal to the number N of the local basis output calculation circuit 10, the connection weight correction circuit 15 outputs the connection weight. v (i) and w (i)

[0080]

[Equation 24]

Modify as follows. Other operations are the same as those in the first and second embodiments.
Is the same as. According to the neural network simulation apparatus of the present embodiment, the output signal y of the present embodiment is generated by the combination of the elementary function using the input signal x and some parameters, and the potential function C is also the same.

[0082]

[Equation 25]

Can be written as By using this, the convergence of learning can be easily analyzed. In addition, a structure suitable for the objective function can be obtained by learning about tens to hundreds of times. In particular, according to this embodiment, the reference vector is distributed in a large area in which the value of the second derivative of the objective function is large. in addition,
As is apparent from the configuration of the present invention, a plurality of nerve cell simulating elements normally output a non-zero value for one input signal x, and the output signal y is given by the sum thereof, so that the LV
Unlike Q and Sakaguchi's neural network simulator, smooth function approximation is possible. Further, when the error is large, the teacher signal z given a new neuron-mimicking element that works only near the input signal x without changing the information stored in the parameters accumulated in the past is used as the output signal y. The speeding up of learning is realized by this.

In principle, it is easy to extend the coupling weight to not only the first order of the input signal but also the second order and the third order. It is also possible in principle to use an appropriate orthogonal function such as a Fourier series for the connection weight.

(Embodiment 4) In the above embodiment, each local basis correction circuit 16, each joint weight correction circuit 15, each local base setting circuit 18, and each joint weight setting circuit 17 independently determine the learning process. Was going on. This method is effective considering the robustness of the device, but has the aspect of increasing the circuit scale. In the present embodiment, a case will be described in which a learning method switching circuit for collectively controlling the learning methods is provided in order to realize the reduction of the circuit scale.

The fourth embodiment of the present invention will be described below with reference to the drawings. In FIG. 5, 10 is the local basis output calculation circuit, 11 is the adder,
12 is the divider, 13 is the local output calculation circuit, 14 is the adder, 44 is the difference signal obtained by taking the difference between the teacher signal and the output signal, and the internal of the local basis output calculation circuit Reference numeral 46 is a learning method switching circuit for switching the learning method from parameters, and 46 is a local basis output calculation circuit that receives an input signal, an output signal, a normalized local basis output signal from the divider, and a modified difference signal from the learning method switching circuit. 10 is a simple local basis correction circuit that reads internal parameters from the local output calculation circuit 13 and the internal parameters of the local basis output circuit 10, and 45 is a normalized local basis output signal from the divider and a learning method switching circuit. The internal parameter of the local output calculation circuit 13 is read in response to the modified difference signal from and the internal parameter of the local output calculation circuit 13 is modified. Reference numeral 48 denotes a joint weight correction circuit, which receives an input signal, a standardized local basis output from the divider 12 and a setting signal from the learning method switching circuit 44, reads internal parameters of the local basis output calculation circuit 10, and outputs a local basis. Reference numeral 47 is a simple local basis setting circuit for setting internal parameters of the output calculation circuit 10, and 47 receives a total local basis output signal from the adder 11, an output signal, a teacher signal, and a setting signal from the learning method switching circuit 44. Local output calculation circuit 13
It is a simple coupling weight setting circuit for setting internal parameters of.

The operation of the neural network simulation device configured as described above will be described. Note that, here, a case will be described in which a Gaussian function is used as a local basis function that characterizes the property of the local basis output calculation circuit 10.

First, the input signal x enters the local basis output calculation circuit 10. In the i-th local basis output calculation circuit,
The local basis output signal S (i) is calculated as (Equation 5) and is output. Where a (i) is the activation parameter with a value of 0 or 1, μ (i) is the reference vector and σ ² (i) is the variance. The adder 11 calculates the total sum (Equation 6) of all the local basis output signals S output from the local basis output calculation circuit 10 and outputs the total local basis output signal Stotal. The i-th divider 12 receives the local base output signal S (i) and the total local base output signal Stotal from the i-th local base output calculation circuit 10 and the adder 11, and outputs the normalized local base output signal b (i). Is calculated and output. The i-th local output calculation circuit 13 calculates the local output signal o (i) from the normalized local basis output signal b (i) from the divider 12 and the internally-stored combination weight v (i) (Equation 8). ) Is calculated and output. The adder 14 receives the local output signal o from the local output calculation circuit 13, calculates the output signal y as in (Equation 9), and outputs it.

Next, the learning operation will be described. When the teacher signal z is given to the input signal x, first, the output signal y corresponding to the input signal x obtained by the above operation is output.
And the teacher signal z, the difference signal E is generated. The difference signal E is input to the learning method switching circuit 44. Upon receiving the difference signal E, the learning method switching circuit 44 reads out the internal parameter of the local basis calculation circuit 10. The learning method switching circuit 44 calculates the number Nα of local basis output calculation circuits in the active state from the read parameters, and when | E | <θE or Nα = (total number of local basis output calculation circuits), Modified difference signal E1 = E setting control signal E2 = NO | E | ≧ θE, and when Nα <(total number of local basis output calculation circuits), modified difference signal E1 = 0 setting control signal E2 = Nα + 1 is output.

The simple combined weight correction circuit 45 and the simple local basis correction circuit 46 operate as follows.

The i-th (i = 1 to Na) simple coupling weight correction circuit 45 also receives the teacher signal z and the normalized local basis output signal b (i) from the i-th divider 12, and receives the i-th local The connection weight v (i) of the output calculation circuit 13 is modified as follows.

[0092]

[Equation 26]

The i-th simple local basis correction circuit 16 also receives the input signal x, the normalized local basis output signal b (i) and the output signal y from the i-th divider 12, and receives the i-th local The connection weight v (i) of the output calculation circuit 13 is read out, and the internal parameters of the i-th local basis output calculation circuit 10 are modified as follows.

[0094]

[Equation 27]

After the above parameters have been corrected, the simple local basis correction circuit 46 and the simple combination weighting correction circuit 45
Returns the modification end signal.

On the other hand, the simple combination weight setting circuit 47 and the simple local basis setting circuit 48 do nothing when the setting control signal is NO. When the setting control signal is (Nα + 1), the operation is as follows.

The simple combination weight setting circuit 47 outputs the output signal y
, The teacher signal z, and the total local basis output signal Sto from the adder 11
Na designated by the setting control signal in response to tal
The connection weight v (Na + 1) of the + 1st local output calculation circuit 13 is set as in (Equation 12).

The simple local basis setting circuit 48 also receives the input signal x and the normalized local basis output signal b from the divider, reads the internal parameters of the local basis output calculation circuit 10,
The internal parameter of the Na + 1th local basis output calculation circuit 10 designated by the setting control signal is set as shown in (Equation 13). After the above parameters are set, the simple local basis setting circuit 48 and the simple combination weight setting circuit 47
Returns a setting end signal.

As in the above embodiment, c2 is a parameter having a positive value. However, in the initial state where Na is 0, σ (Na + 1) = σ0 and a fixed value σ0 are given. The value of c2 is preferably set to a value of about 0.1 to 1.0, but this parameter has considerable arbitrariness and is not limited to this range. Further, for the initial setting, a method of setting the first two data (x1, z1) and (x2, z2) to (Equation 14) may be used.

According to the neural network simulation apparatus of this embodiment, the output signal y of this embodiment is generated by the combination of the elementary functions using the input signal x and some parameters, and the potential function C is You can write (Equation 15).
By using this, the convergence of learning can be easily analyzed.

Further, as shown in FIG. 2, a structure suitable for the objective function can be obtained by learning several tens to several hundreds. According to this embodiment, the reference vector is 1 of the objective function.
Many are distributed in the area where the differential is large. In addition, as is apparent from the configuration of the present invention, a plurality of nerve cell simulating elements normally output a non-zero value for one input signal x, and the output signal y is given by the sum thereof, so that LVQ and Unlike the neural network simulator of Sakaguchi et al., Smooth function approximation is possible. Furthermore, when the error is large, the information stored in the parameters accumulated in the past can be directly input to the input signal x
By activating a new neural cell simulating element that works only near the so that the given teacher signal z becomes the output signal y, high-speed learning is realized.

Further, by providing the learning method switching circuit 44, the learning method selection judgment is performed instead of the local basis correction circuit 16, the joint weight correction circuit 15, the local basis setting circuit 18, and the joint weight setting circuit 17. No simple local basis correction circuit 46, simple combination weight correction circuit 45, simple local basis setting circuit 48, and simple combination weight setting circuit 4
7 can be used.

(Embodiment 5) In the neural network simulating device of the present invention, when the error is large, the local basis output calculation circuit 10 and the like are newly set to speed up the learning. However, with this method alone, the substantially unnecessary local basis output calculation circuit 10 may survive depending on the order of the learning data. FIG. 6 shows the contour map of the objective function and the distribution of the reference vector of the experimental result of the present invention. The reference vector A in the figure is activated at the initial stage of learning, but after the learning is advanced in this way, it can be replaced by another reference vector, that is, the output changes even if the reference vector A disappears. It's gone. Such a phenomenon is effective in terms of robustness of the neural network simulator, but is disadvantageous in terms of memory and circuit scale. In order to solve this problem and efficiently use the local basis output calculation circuit and the like, an example in the case of forgetting is described.

The fifth embodiment of the present invention will be described below with reference to the drawings. 7 is different from FIG. 5 in that instead of the learning method switching circuit 44, 49 selects a learning method from the difference signal obtained by taking the difference between the teacher signal and the output signal and the internal parameter of the local basis output calculation circuit. A learning method selection circuit 50 is a learning method selection circuit 4
9 is a point provided with a forgetting processing circuit for receiving the forgetting processing start signal from 9 and reading the internal parameter of the local basis output calculation circuit 10 to perform the forgetting processing. The operation of the neural network simulation apparatus configured as described above will be described. Note that, here, a case will be described in which a Gaussian function is used as a local basis function that characterizes the property of the local basis output calculation circuit 10.

First, the input signal x enters the local basis output calculation circuit 10. In the i-th local basis output calculation circuit,
The local basis output signal S (i) is calculated as in (Equation 5),
Is output. Here, a (i) is an activation parameter having a value of 0 or 1, μ (i) is a reference vector, and σ ²
(i) is the variance. The adder 11 calculates the total sum (Equation 6) of all the local basis output signals S output from the local basis output calculation circuit 10 and outputs the total local basis output signal Stotal. The i-th divider 12 receives the local base output signal S (i) and the total local base output signal Stotal from the i-th local base output calculation circuit 10 and the adder 11, and outputs the normalized local base output signal b (i). Is calculated and output. The i-th local output calculation circuit 13 calculates, from the normalized local basis output signal b (i) from the divider 12 and the joint weight v (i) held therein,
The local output signal o (i) is calculated as in (Equation 8) and output. The adder 14 receives the local output signal o from the local output calculation circuit 13, calculates the output signal y as in (Equation 9), and outputs it.

Next, the learning operation will be described. When the teacher signal z is given to the input signal x, first, the output signal y corresponding to the input signal x obtained by the above operation is output.
And the teacher signal z, the difference signal E is generated. The difference signal E is input to the learning method selection circuit 49. When the learning method selection circuit 49 receives the difference signal E,
The internal parameters of the local basis calculation circuit 10 are read out. The learning method selection circuit 49 calculates the number Nα of the local basis output calculation circuits in the active state from the read parameters, and the index INDEX of the local basis output calculation circuit in the inactivated state.
When | E | <θE or Nα = (total number of local basis output calculation circuits), modified difference signal E1 = E setting control signal E2 = NO | E | ≧ θE, and Nα <(local In the case of the total number of base output calculation circuits), a modified difference signal E1 = 0 setting control signal E2 = INDEX is output.

The simple combined weight correction circuit 45 and the simple local basis correction circuit 46 operate as follows.

The i-th (i = 1 to Na) simple coupling weight correction circuit 45 also receives the teacher signal z and the normalized local basis output signal b (i) from the i-th divider 12, and receives the i-th local The connection weight v (i) of the output calculation circuit 13 is modified as in the following (Equation 26).

The i-th simple local basis correction circuit 16 also receives the input signal x, the normalized local basis output signal b (i) and the output signal y from the i-th divider 12, and receives the i-th local The connection weight v (i) of the output calculation circuit 13 is read out, and the internal parameter of the i-th local basis output calculation circuit 10 is corrected as shown in (Equation 27).

After the above parameters have been corrected, the simple local basis correction circuit 46 and the simple combination weighting correction circuit 45
Returns the modification end signal.

On the other hand, the simple combination weight setting circuit 47 and the simple local basis setting circuit 48 do nothing when the setting control signal is NO. When the setting control signal is INDEX, it operates as follows.

The simple combination weight setting circuit 47 outputs the output signal y
, The teacher signal z, and the total local basis output signal Sto from the adder 11
IN specified by the setting control signal in response to tal
The connection weight v (INDE of the DEX-th local output calculation circuit 13
Set X) as follows.

[0113]

[Equation 28]

The simple local basis setting circuit 48 also receives the input signal x and the normalized local basis output signal b from the divider, reads out the parameters of the local basis output calculation circuit 10, and sets the INDEX-th position designated by the setting control signal. The internal parameters of the local basis output calculation circuit 10 are set as follows.

[0115]

[Equation 29]

After the above parameters are set, the simple local basis setting circuit 48 and the simple combination weight setting circuit 47
Returns the setting end signal. Upon receiving the setting end signal, the learning method selection circuit 49 outputs the forgetting process start signal to the forgetting process circuit 5.
Send to 0. A forgetting processing circuit 50 which receives a forgetting processing start signal
The operation will be described with reference to the flowchart shown in FIG.

The forgetting processing circuit 50 reads internal parameters from the local basis output calculating circuit 10. Then, 1 is entered in the index i that specifies the local basis output circuit 10. When the read active parameter of the i-th local basis output calculation circuit 10 is 1, the read reference vector μ (i) of the i-th local basis output calculation circuit 10 is output to the input terminal, and the adder 14 outputs it. Of the reference vector μ of the i-th local basis output calculation circuit 10 is received, the i-th local basis output calculation circuit 10 is deactivated, and the i-th local basis output calculation circuit 10 is deactivated. (i) is output to the input terminal and its output signal y '(μ
(i), i) is compared with the held output signal y (μ (i)), and if | y ′ (μ (i), i) −y (μ (i)) | <θ, the process is terminated. , I-th local basis output calculation circuit is kept inactive state | y '(μ (i), i) −y (μ (i)) | ≧ θ If i-th local basis output calculation circuit Activate i Perform the process of incrementing i.

When the read active parameter of the i-th local basis output calculation circuit 10 is 0, i is incremented.

If i = N + 1 is not satisfied, the process returns to the above.
If i = N + 1, the process ends.

In this way, the unnecessary redundant local basis output calculation circuit 10 is deactivated and ready for the next learning.

The threshold value θ is preferably about θ = θE / 2 with respect to the threshold value θE set for the difference signal E. However, the threshold value θ is not limited to this, and may be θ <θE.

Further, the threshold value θ may be a function θ = θE f (σ ² (i)) of the variance σ ² (i). If the function f is a function that takes a positive value that monotonically decreases, the local basis output calculation circuit 10 that covers a wide input signal region is less likely to be inactivated. As a result, it is possible to suppress a temporary increase in error due to inactivation of the element.

As means for suppressing the influence of the deactivation of the local basis output calculation circuit on the output signal y, the a-th local output calculation circuit 13 corresponding to the deactivated a-th local basis output calculation circuit 10 There is a method of transferring the information carried by the connection weight v (a) of (1) to the connection weight of another local output calculation circuit 13.

Specifically, using the output of the a-th local basis output calculation circuit 10 before inactivating the connection weight v (b) of the local-output calculation circuit 13 other than the b-th, v (b) = V (b) + (v (a) -v (b)) B (μ
(B)) Here, B (μ (b)) is the output of the a-th local basis output calculation circuit 10 before inactivation when the input signal is μ (b).

By performing this modification of the connection weighting simultaneously with the inactivation of the a-th local basis output calculation circuit 10, the influence of the inactivation of the local basis output calculation circuit 10 on the output signal y can be suppressed. it can.

The initial learning is the same as in the above embodiment. According to the neural network simulating device of this embodiment, in addition to the effects of the above embodiment, the internal parameters of the neural network simulating device can be set most efficiently, and even a device of a relatively small scale can be used. It is possible to realize an excellent neural network simulator that can approximate a function with high accuracy.

In this embodiment, the output signals y and y'from the adder 14 are compared when the i-th local basis output calculation circuit 10 is activated and inactivated. Although the reference vector μ (i) of the i-th local basis output calculation circuit 10 is used as an input for this, the input signal x given as the learning data is used instead of this reference vector μ (i). You can also In this case, CD (i, x) = | y '(x, i) -z (x is calculated for each activated local basis output calculation circuit 10 every time learning data is given. ) |-| Y (x)-
z (x) | is calculated and stored. Here, CD (i, x) represents the degree of contribution of the i-th local basis output calculation circuit 10 to the input signal x, and y ′ (x, i) deactivates the i-th local basis output calculation circuit 10. Represents an output signal from the adder 14 when x is an input signal, z (x) represents a teacher signal corresponding to the input signal x, and y (x) represents an i-th local basis output calculation circuit. The output signal from the adder 14 when x is an input signal without deactivation is shown. Further, for each activated local basis output calculation circuit, the average value ACD of the contributions CD (i, x) to the past learning data is calculated.
(i) is obtained, and if ACD (i) <θ ', deactivate the i-th local basis output calculation circuit. If ACD (i) ≧ θ', keep the i-th local basis output calculation circuit activated. Perform the process of setting. According to this method, even if the learning data includes noise, the effect of noise can be reduced and smooth function approximation can be performed. Further, in this method, instead of switching activation and inactivation of each local basis output calculation circuit 10 discontinuously with θ ′ as a threshold value, each local basis output calculation circuit 10 is proportional to ACD (i). The internal parameter a (i) of 10 can also be changed continuously. However, when ACD (i) has a negative value, the value of a (i) is set to 0.

(Embodiment 6) In the neural network simulating device of the present invention, a smooth function approximation can be performed at high speed by newly setting the local basis output calculating circuit 10 and the like. However, with this method alone, the substantially unnecessary local basis output calculation circuit 10 may survive depending on the order of the learning data. On the other hand, Example 5
According to this method, the unnecessary local basis output calculation circuit 10 can be eliminated, but the amount of calculation required for that purpose increases. In this embodiment, when there is information that is known in advance regarding the input / output relationship to be learned, this information is used to initialize the internal parameters of the neural network simulator of the present invention, thereby reducing the amount of calculation. Faster and fewer local basis output calculation circuit 1 without increasing
A method for enabling learning by 0 will be described.

The configuration of the sixth embodiment of the present invention is the same as that of the first embodiment shown in FIG. 1 and will not be repeated here.

Next, the operation of the sixth embodiment will be described. Prior to the correction of the internal parameters of the neural network simulator of the present invention based on the learning data, first, the internal parameters of the neural network simulator of the present invention are utilized by utilizing the information known in advance regarding the input / output relationship to be learned. Is initialized as follows.

Regarding input / output relations to be learned,
There are R pieces of known information, and the kth information is "input
If x belongs to the closed region a (k) on the input space, the output y
(x) is equal to the constant c (k) ".
It This information does not have to be completely correct.
Yes. The following are two examples of specific information. Here input x
Is a two-dimensional vector (x ₁, X₂) And the output y (x) is
Scalar.

"0.2 <x ₁ <0.5 and 0.4 <x ₂ <0.
If y is 8, then y (x) = 1.0 ”“ If x ₁ ² + x ₂ ² ≦ 4.0, y (x) = 0.5 ”Now, the input x is generally an N-dimensional vector (x ₁ , ... , X _N )
, And the J-th element of this vector will be denoted as x _J. Since a (k) is a closed region, each element x _J has the maximum value x _J max and the minimum value x _J min in the region a (k). Here, the local basis output calculation circuit 1
The internal parameters of 0 and the internal parameters of the local output calculation circuit 13 are initialized as follows. kth (k is 1 to R
The internal parameters a (k), μ (k), σ (k) of the local basis output calculation circuit 10 and the internal parameter v (k) of the kth local output calculation circuit 13 are

[0133]

[Equation 30]

The initial setting is as follows. Where the vector μ (k)
The J-th element of is written as μ _J (k). Further, c3 is a positive constant, and a value equal to the constant c2 used at the time of learning in the first embodiment is set. In addition, the internal parameters of the local basis output calculation circuit 10 other than the above are initialized to a (i) = 0.

After the above initialization, the learning operation is performed in the same procedure as the learning operation in the first embodiment.
The internal parameter a (i), μ of the local basis output calculation circuit 10
(i), σ (i) and the internal parameter v (i) of the local output calculation circuit 13 are sequentially modified. I knew in advance
If the individual information is not completely correct, or if the input / output relationship of the neural network simulation device created by the initial value setting of the internal parameters based on these R information is completely correct with these R information. Even if they do not match, it is possible to realize a good approximation of the objective function at high speed by sequentially correcting the internal parameters based on the learning data.

In the initial setting of the internal parameters μ (k) and σ (k), instead of the above setting method, μ (k) is initialized so that it coincides with the center of gravity of the closed region a (k). , Σ
(k) may be initialized to a value proportional to the (1 / N) th power of the volume of the closed region a (k).

Further, in the present embodiment, the method of initializing the internal parameters of the neural network simulation device having the same configuration as the first embodiment has been described, but the second embodiment, the third embodiment, It goes without saying that the same initial setting method as in the present embodiment can be applied to the neural network simulating device having the same configuration as that of either the fourth embodiment or the fifth embodiment. In particular, the information known in advance is that "if the input x belongs to the closed region a (k) on the input space, f (x) is a function of x, then the output y (x) is equal to f (x)." In the case of the expression (1), it is necessary to perform the initial setting of the parameters for the neural network simulator having the same configuration as that of the third embodiment by the same method as this embodiment. It is effective to realize a good approximation.

Further, the internal parameters a (i) and μ (i) of the local basis output calculation circuit 10 are applied to the neural network simulating apparatus having the same structure as that of the fifth embodiment by the same method as this embodiment. ,
σ (i) and the internal parameter v (i) of the local output calculation circuit 13
When the learning with forgetting is performed using the method of the fifth embodiment after initializing, the objective function is well approximated with only a relatively small number of local basis output calculation circuits 10 being activated. be able to. Therefore, in this case, the internal parameters a (i), μ (i), σ (i), v
By examining the value of (i), there is an advantage that the operation of the neural network simulator can be easily understood and predicted.

Further, in the present embodiment, there are R pieces of information that are known in advance regarding the input / output relationship to be learned,
Internal parameter in the case where the k-th information is expressed in the form "if the input x belongs to the closed region a (k) on the input space, the output y (x) is equal to the constant c (k)". , The input x is an N-dimensional vector (x ₁ , ..., X _N ), and each element x _{J of} this vector is described.
Is mixed with those that take only two kinds of values, 0 or 1, and those that can take any value continuously between 0 and 1,
The information known in advance regarding the input-output relationship to be learned is that "if the input x satisfies the logical expression L (k) (x),
Even if the output y (x) is expressed in the format of “the output y (x) is equal to the constant c (k)”, the internal parameters can be initialized as follows. Where the logical expression L (k) (x)
Is a conjunction of the elementary logical expressions, and the elementary logical expression is the element x _J
When x takes only two kinds of values 0 or 1, it is in the form of x _J = 0 or x _J = 1 and both the prime logical formula x _J = 0 and the prime logical formula x _J = 1 exist at the same time. Element x _J
When can take any value continuously between 0 and 1,
Let x _J min ≤ x _J ≤ x _J max.

The internal parameter a of the k-th (k represents each of 1 to R) local basis output calculation circuit 10
(k), μ (k), σ (k) and k-th local output calculation circuit 1
For the internal parameter v (k) of 3,

[0141]

[Equation 31]

The initial setting is as follows. Where the vector μ (k)
The J-th element of is written as μ _J (k). Further, c3 is a positive constant, and a value equal to the constant c2 used at the time of learning in the first embodiment is set. W (J) has the following values.

[0143]

[Equation 32]

A local basis output calculation circuit 1 other than the above
The internal parameters of 0 are initialized as a (k) = 0.

The information known in advance regarding the input-output relationship to be learned is that "if the input x satisfies the logical expression L (k) (x), the output y (x) is equal to the constant c (k)." The above initialization method for the case expressed in the form is
"The input x satisfies the logical expression L (k) (x)" is converted into "the input x belongs to the closed region a (k) on the input space" by the following procedure, and the converted information is obtained. Correspondingly, it is equivalent to performing the initial setting by the above method.

1) A prime logical expression x _J = 0 is a logical expression L (k) (x)
Is included in the prime logical expression −0.5 ≦ x _J ≦ 0.
Replace with 5.

2) The elementary logical expression x _J = 1 is the logical expression L (k) (x)
Is included in the logical formula 0.5 ≦ x _J ≦ 1.5
Replace with.

3) No replacement is performed for a prime logical expression of the form x _J min ≤x _J ≤x _J max.

4) If the prime logical expression for the element x _J is not included in the logical expression L (k) (x), then the prime logical expression 0 ≦ x _J ≦
1 is added to the logical expression L (k) (x).

(Embodiment 7) A second embodiment of the forgetting processing circuit will be described.

In FIG. 7, the forgetting processing circuit 50 reads internal parameters from the local basis output calculation circuit 10.

1 is put in the index j that specifies the local basis output circuit 10. The activation parameter of the jth local basis output calculation circuit 10 is determined.

When the activation parameter of the j-th local basis output calculation circuit 10 is 1, the following processing is performed.

1 is put in the index i for designating the local basis output circuit 10. When the read active parameter of the i-th local basis output calculation circuit 10 is 1, the read reference vector μ (i) of the i-th local basis output calculation circuit 10 is output to the input terminal, and the adder 14 outputs it. Output signal y (μ (i))
In response to this, it is held and i is incremented. By repeating this, all output signals y (μ (i)) of the reference vector μ (i) whose activation parameter is 1 are calculated.

Next, the jth local basis output calculation circuit 10
Are inactivated, the same processing is performed, and all output signals y ′ (μ (i)) when the j-th element is inactivated are calculated.

Further, using the calculated two kinds of output signals, the maximum error Emax = max {| y '(μ (i), i) -y (μ) with respect to the reference vector μ (i)
(i)) |} is calculated. Compare with this maximum error Emax, if Emax <θ, terminate the process and keep the jth local basis output calculation circuit in an inactive state If Emax ≧ θ, reactivate the jth local basis output calculation circuit Yes The process of incrementing j is performed.

When the read activation parameter of the j-th local basis output calculation circuit 10 is 0, j is incremented.

If j = N + 1 is not satisfied, the above processing is returned to.
If j = N + 1, the process ends.

In this way, the unnecessary unnecessary redundant basis output calculation circuit 10 is deactivated and ready for the next learning.

The threshold value θ is preferably about θ = θE / 2 with respect to the threshold value θE set for the difference signal E. However, the threshold value θ is not limited to this, and may be θ <θE.

As means for suppressing the influence of the inactivation of the local basis output calculation circuit on the output signal y, the a-th local output calculation circuit 13 corresponding to the inactivated a-th local base output calculation circuit 10 There is a method of transferring the information carried by the connection weight v (a) of (1) to the connection weight of another local output calculation circuit 13.

Specifically, using the output of the a-th local basis output calculation circuit 10 before inactivating the connection weight v (b) of the local-output calculation circuit 13 other than the b-th, v (b) = V (b) + (v (a) -v (b)) B (μ
(B)) Here, B (μ (b)) is the output of the a-th local basis output calculation circuit 10 before inactivation when the input signal is μ (b).

By performing this correction of the connection weighting at the same time as the inactivation of the a-th local basis output calculation circuit 10, the influence of the inactivation of the local basis output calculation circuit 10 on the output signal y can be suppressed. it can.

[0164]

As described above, the present invention calculates the local basis output signal S by receiving the input signal x and outputs it, and the local basis output signals from the plurality of local basis output calculation circuits. An adder that receives S to obtain the sum of them and outputs a total local basis output signal Stotal, and a local basis output signal S and a total local basis output signal Stotal from the local basis output calculation circuit and the adder A local output calculation circuit that outputs, as a local output signal o, a product of a divider that outputs a base output signal b, a standardized local base output signal from the divider, and a coupling weight v that is internally held, and a plurality of local output calculation circuits. The adder that outputs the sum of the local output signals o from the output calculation circuit as the output signal y, the difference signal E between the teacher signal z and the output signal y, and the standardized local base output signal b from the divider, are provided to correspond. Local output meter A connection weight correction circuit for correcting the connection weight v of the circuit, and a difference signal E from the output signal y and the standardized local base output signal b from the divider to correct the internal parameter of the corresponding local base output calculation circuit. Based on a local basis correction circuit, a combination weight setting circuit that receives a teacher signal z and sets a combination weight of the local output calculation circuit, and an internal parameter of the input signal x, the normalized local base output signal b, and the local output calculation circuit. By configuring with a local basis setting circuit that sets internal parameters of the local basis output calculation circuit, a structure having a potential function and suitable for the objective function can be obtained by learning about several tens to several hundreds of times, and smooth. It is possible to realize a neural network simulation device capable of performing various function approximations.

Furthermore, when there is information that is known in advance about the objective function, faster learning can be realized by initializing the internal parameters of the neural network simulation device based on that information.

[Brief description of drawings]

FIG. 1 is a block diagram of a neural network simulation device according to a first embodiment of the present invention.

FIG. 2A is a conceptual diagram showing a structure for approximating an objective function obtained by the same embodiment. FIG. 2B is a conceptual diagram showing how the same embodiment improves the approximation accuracy of the objective function by learning.

FIG. 3 is a block diagram of a neural network simulator according to a second embodiment of the present invention.

FIG. 4 is a block diagram of a neural network simulator according to a third embodiment of the present invention.

FIG. 5 is a block diagram of a neural network simulation device according to a fourth embodiment of the present invention.

FIG. 6 is a conceptual diagram showing that useless reference vectors are generated as learning progresses in the present invention having no forgetting processing circuit.

FIG. 7 is a block diagram of a neural network simulator according to a fifth embodiment of the present invention.

FIG. 8 is a flowchart illustrating the operation of the forgetting processing circuit.

FIG. 9 is a block diagram of a conventional neural network simulation device.

FIG. 10 is a conceptual diagram for explaining a learning state of a conventional neural network simulation device.

[Explanation of symbols]

 1-1 to 4 Distance calculation circuit 2 Competitive circuit 3 Output calculation circuit 4 Coupling weight correction circuit 5 Reference vector correction circuit 10 Local basis output calculation circuit 11 Adder 12 Divider 13 Local output calculation circuit 14 Adder 15 Coupling weight correction circuit 16 local basis correction circuit 17 coupling weight setting circuit 18 local basis setting circuit 19 adder 20 coupling weight setting control section 21 reference vector signal generating section 22 signal storage section 23 coupling weight setting section 44 learning method switching circuit 45 simple coupling weight correction circuit 46 Simple Local Basis Correction Circuit 47 Simple Coupling Weight Setting Circuit 48 Simple Local Basis Setting Circuit 49 Learning Method Selection Circuit 50 Forgetting Processing Circuit

Claims (12)

[Claims] 1. A neural network simulating device using a function (local basis function) that locally has a non-zero value, and a local basis setting circuit that newly sets at least a function that locally has a value. A neural network simulator having a connection weight setting circuit for setting connection weight. 2. A neural network simulating apparatus using a local basis function, wherein the coupling weight is a function of an input signal. 3. A neural network simulating device using a local basis function, comprising a local basis setting circuit and a connection weight setting circuit for newly setting a function having a value locally at least.
A neural network simulator in which the coupling weight is a function of the input signal. 4. The neural network simulating device according to claim 1, wherein the coupling weight is a linear expression of the input signal. 5. A neural network simulating device using a local basis function, wherein the output signal is standardized by the sum of the outputs of the local basis functions.
The neural network simulation device according to any one of 1. 6. A local basis output calculation circuit that receives an input signal and outputs a local basis output signal by each of a plurality of local basis functions, and a total local basis output signal that receives a sum of the plurality of local basis output signals. A divider for receiving the local base output signal and the total local base output signal to output a standardized local base output signal; and a combination for internally receiving the standardized local base output signal. A local output calculation circuit that outputs the product of the weights as a local output signal, an adder that outputs the sum of the local output signals as an output signal, a difference signal between the teacher signal and the output signal, and a normalized local from the divider. A combined weight correction circuit that receives the base output signal and corrects the combined weight of the corresponding local output calculation circuit, and receives the difference signal from the output signal and the normalized local base output signal from the divider A local basis correction circuit that corrects internal parameters of the local basis output calculation circuit; and a combination weight setting circuit that receives the difference signal, the output signal, the teacher signal, and the total local basis output signal and sets the combination weight of the local output calculation circuit. ,
A neural network simulation device comprising: a local basis setting circuit that sets an internal parameter of a local basis output calculation circuit based on an input signal, a normalized local output signal, and an internal parameter of the local output calculation circuit. 7. A local basis output calculation circuit that receives an input signal and calculates and outputs a local basis output signal, and a local basis output signal from a plurality of local basis output calculation circuits, and sums them to obtain a total local basis. A local output calculation circuit that outputs, as a local output signal, a product of an adder that outputs an output signal, a standardized local base output signal from the local base output calculation circuit, and a coupling weight that is held inside, and a plurality of local outputs An adder that outputs the sum of the local output signals from the calculation circuit as a non-standardized output signal, and a division that outputs the output signal by receiving the non-standardized output signal and the total local base output signal from the adder and the adder , A local basis output calculation circuit and an adder to receive the local basis output signal and the total local basis output signal, and to output a normalized local basis output signal, and a difference signal and the normalized local from the divider. Basal A joint weight correction circuit for receiving the force signal and correcting the joint weight of the corresponding local output calculation circuit, and a corresponding local basis output calculation circuit for receiving the difference signal from the output signal and the normalized local basis output signal from the divider Local basis correction circuit for correcting the internal parameters of the input signal, the combination weight setting circuit for receiving the difference signal, the teacher signal, the total local base output signal and the output signal and setting the connection weight of the local output calculation circuit, the input signal and the standard. And a local basis setting circuit for setting internal parameters of the local basis output calculation circuit based on the internal parameters of the generalized local basis output signal and the local output calculation circuit. 8. The combination weight setting circuit includes a combination weight setting control unit that controls the combination weight setting based on the difference signal, a reference vector signal generation unit that reads a reference vector and outputs it as an input signal to an input line, It is composed of a signal storage unit that stores an input signal, a normalized local basis output signal, a teacher signal, and an output signal, and a connection weight setting unit that reads out data from the signal storage unit as necessary to calculate and set a connection weight. The neural network simulation device according to any one of claims 1 to 7. 9. The local basis output calculation circuit, the adder for calculating a total local basis output signal, the divider, the local output calculation circuit, the adder for calculating an output signal, and a teacher signal. And a learning method switching circuit that switches a learning method from a difference signal obtained by taking a difference between the output signal and a local basis output calculation circuit, an input signal, an output signal, and a normalized local basis output signal from the divider. A simplified local basis correction circuit that receives a modified difference signal from the learning method switching circuit, reads internal parameters from the local basis output calculation circuit and the local output calculation circuit, and corrects internal parameters of the local basis output circuit; The internal parameter of the local output calculation circuit is read by receiving the normalized local basis output signal from the divider and the modified difference signal from the learning method switching circuit. However, a simple coupling weighted correction circuit for correcting internal parameters of the local output calculation circuit, the input signal, the normalized local base output from the divider, and the setting signal from the learning method switching circuit are received to calculate the local base output. A simple local basis setting circuit for reading internal parameters of the circuit and setting internal parameters of the local basis output calculation circuit, a total local basis output signal from the adder, an output signal, a teacher signal, and a learning method switching circuit. A neural network simulating device comprising: the simple coupling weight setting circuit that receives a setting signal and sets an internal parameter of the local output calculation circuit. 10. The neural network simulating device according to claim 1, further comprising a forgetting processing circuit. 11. The local basis output calculation circuit, the adder for calculating a total local basis output signal, the divider,
The local output calculation circuit, the adder for calculating an output signal, the simple combination weight correction circuit, the simple local basis correction circuit, the simple combination weight setting circuit, and the simple local basis setting circuit, A learning method selecting circuit for selecting a learning method from a difference signal obtained by taking a difference between a teacher signal and an output signal and an internal parameter of the local basis output calculating circuit, and a forgetting process start signal from the learning method selecting circuit A neural network simulation device comprising a forgetting processing circuit for reading out internal parameters of the local basis output calculating circuit and performing a forgetting process. 12. An internal parameter of the local basis output calculation circuit and a joint weight of the local output calculation circuit are initialized based on information known in advance regarding a functional relationship between the input signal and the teacher signal. The local basis correction circuit and the combined weight correction circuit, or the simple local basis correction circuit and the simple combined weight correction circuit, are used to combine the internal parameters of the local basis output calculation circuit and the combined output weight of the local output calculation circuit. 12. The neural network simulation device according to claim 6, wherein: and are modified.