JPH04305785A - Method for evaluating neural network - Google Patents

Abstract

PURPOSE:To obtain a method for evaluating cause and effect relation between the input and output of a neural network. CONSTITUTION:The I/O functions of units in the neural network constituted of the hierarchical connection of units using respective neurons as models are developed by series and the network is expressed by a non-linear regression expression based upon the developed expression to attain the evaluation. Thus the cause and effect relation between the I/O of the neural network can be cleared by expressing the network by the non-linear regression expression.

Description

[Detailed description of the invention]

0001

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a neural network evaluation method, and more particularly to a neural network evaluation method suitable for evaluating causal relationships between inputs and outputs of a hierarchical neural network.

0002

2. Description of the Related Art Hierarchical neural networks are constructed by hierarchical combinations of units modeled on neurons, and are capable of nonlinear transformation of input signals, and are characterized by the ability to construct the functional form of this nonlinear transformation through learning. For this reason,
Taking advantage of this feature, “neural computers” (
Tokyo Denki University Press, 1986-4), “Neural
As described in "Network Information Processing" (Sangyo Tosho, 1986-7), attempts have been made to apply it to various fields.

An example of the configuration of a hierarchical neural network and a unit configuration are shown in FIGS. 1 and 2, respectively. The function of the nonlinear transformation is the input/output function f(x) of the unit.
The function expressed by the following equation is generally used.

0004

[Math 1]

Here, x: Input θ: Threshold Regarding the nonlinearity of this input/output function f(x) and the nonlinear transformation function of the network, please refer to "About Realization of Approximation of Continuous Mapping by Neural Network" (Electronic Information and Communication Academic Technical Research Report, MBE88-9, 1988-4),
“About the capability of neural networks” (IEICE technical research report, MBE88
52, 1988-7), it has been studied from the perspective of mathematical theory and it has been proven that hierarchical neural networks have a certain degree of versatility as a mechanism for realizing continuous mapping. .

[0006]

[Problem to be Solved by the Invention] The above-mentioned prior art proves that hierarchical neural networks have a certain degree of versatility as a mechanism for realizing continuous mapping.
It does not provide a specific method for designing a hierarchical neural network that approximates a certain continuous mapping. Furthermore, no theoretical background has been provided for extracting the structure of input-output relationships from networks formed through learning. For this reason, networks have traditionally been treated as black boxes.

An object of the present invention is to provide a method for evaluating causal relationships between inputs and outputs of a neural network.

[0008]

[Means for Solving the Problems] In order to achieve the above object, the present invention provides a neural network application system configured by hierarchically connecting units modeled on neurons, in which the input/output functions of the units are expanded into a series. Then, using this expansion formula, the network was expressed by a nonlinear regression formula.

[0009]

[Operation] By expressing the neural network using a nonlinear regression equation, the causal relationship between the input and output of the network becomes clear.

0010

[Embodiment] An embodiment of the present invention will be described below with reference to FIG. This embodiment includes a learning system 2 that causes a neural network to learn the input/output characteristics of the target system 1, an estimation system 3 that estimates the input/output characteristics of the target system 1 using the neural network after learning, and a neural network after learning. The system consists of an evaluation system 4 that evaluates.

The target system 1 may be any type of system, such as a plant, as long as outputs can be associated with inputs.

The estimation system 3 estimates the output y (vector) of the target system 1 for the input x (vector) using a hierarchical neural network. This relationship is expressed by the following equation.

[0013]

[Math 2]

Here, g: nonlinear relationship (vector) y:
Estimated value (vector) of output y A hierarchical neural network, as shown in FIG. 1, is constructed by a hierarchical combination of units shown in FIG. 2. The input/output relationship of the units in each layer is expressed by the following equation.

[0015]

[Math 3]

[0016]

[Math 4]

Here, uj(k): sum of inputs to the j-th unit of the k-th layer vj(k): output wij(k-
1, k): Weighting coefficient for coupling from the i-th unit of the (k-1)th layer to the j-th unit of the k-th layer f: Function that provides the input/output function of each unit (input/output function) Each unit in the layer (input layer) outputs the same thing as the unit's input. The characteristics of the nonlinear function g in Equation 2 are the number of layers, the number of units in each layer, and the weighting coefficient wij (
k-1, k) changes. Therefore, by adjusting these, a nonlinear function g representing the input/output characteristics of the target system 1 can be obtained. In particular, this weighting factor w
Adjustment of ij (k-1, k) can be realized by learning.

The learning system 2 constructs a nonlinear function g representing the input/output characteristics of the target system 1 through learning. Next, this learning algorithm will be explained.

First, an input/output set (x
t, yt) is given, the sum of squared errors shown in the following equation is defined as the loss function r.

[0020]

[Math 5]

Here, w: A collection of all the weighting coefficients of connections in the neural network vj (m) (w, xt): The nth layer (output layer) obtained comprehensively from the input xt and the weight w. Output of j-th unit w
The correction amount Δw is the gradient (gr
adient) and is expressed by the following formula.

[0022]

[Math 6]

Each component of ▽r on the right side of equation 6 can be transformed as shown in the following equation.

[0024]

[Math 7]

By substituting the equation 4 into the equation 7 and sorting it out, the following equation is derived.

[0026]

[Math. 8]

When k≠m, ∂r/∂uj on the right side of equation 8
(k) is obtained by the following equation.

[0028]

[Math. 9]

By substituting Equation 3 and Equation 4 into Equation 9 and rearranging, the following equation is obtained.

[0030]

[Math. 10]

Here, f': derivative of f ∂r/∂uj(
k)=dj(k), Equations 6 and 10 are expressed by the following equations.

[0032]

[Math. 11]

[0033]

[Math. 12]

[0034] Also, when k=m, ∂r/∂uj(m)
=dj(m) is obtained from Equation 5 using the following equation.

[0035]

[Math. 13]

Using Equations 11, 12, and 13, the modification of the connection weighting coefficient wij(k-1,k) can be calculated recursively from k=m to k=2. In other words, the error between the ideal output and the actual output at the output layer is wil(k, k
+1) and propagates while calculating the weighted sum. This is the error backpropagation learning algorithm.

When the input/output function f is common to all units and is given by equation 1, f' is expressed by the following equation.

[0038]

[Math. 14]

From Equations 3 and 14, the following equation is derived.

[0040]

[Math. 15]

Note that in order to converge the learning smoothly and quickly, equation 11 can be modified as shown in the following equation.

[0042]

[Math. 16]

Here, α: a small positive constant (α=1−ε
) t: number of corrections (or time (discrete)) The evaluation system 4 evaluates the neural network after learning. That is, a neural network representing the input/output characteristics of the target system 1 is obtained through learning, and this neural network is evaluated. For this purpose, the input/output function of the unit is expanded into a series, the network is expressed by a nonlinear regression equation using this expansion, and the network is evaluated using this nonlinear regression equation. This will be explained in detail below. Note that the evaluation system 4 can be used not only for analysis and evaluation of neural networks after learning, but also for designing neural networks.

One method of series expansion of a function is Taylor expansion, and when this is used, the input/output function f(x
) is expressed by the following formula.

[0045]

[Math. 17]

The derivative of the input/output function f(x) shown in Equation 1 is expressed by the following equation, and as the order increases, the equation becomes rapidly complicated.

[0047]

[Math. 18]

x=x0 of the derivative expressed by this number 18
By substituting the value in Equation 17, the Taylor expansion of f(x) is obtained.

The basic formula of the above Taylor expansion of f(x) is the Taylor expansion of the input/output function f(x) at x0 = 0, that is, the Maclaurin expansion, and errors are evaluated for this. By substituting x0 = 0 into Equation 17, the following equation is obtained.

[0050]

[Math. 19]

Further, by substituting x=0 into Equation 18, the following equation is derived.

[0052]

[Math. 20]

Rearranging this number 20, the value of the derivative up to the sixth order of the input/output function f(x) at x=0 is expressed by the following equation.

[0054]

[Math. 21]

By substituting the threshold value θ=0 into Equation 21, the following equation is obtained.

[0056]

[Math. 22]

In addition, in Equation 21, the threshold value θ=0.
By substituting 5 and 1, the following formula is obtained.

[0058]

[Math. 23]

[0059]

[Math. 24]

In Equation 19, by omitting the terms of order 7 or higher and substituting Equation 22, Equation 23, and Equation 24, the following equation is derived.

[0061]

[Math. 25]

[0062]

[Math. 26]

[0063]

[Math. 27]

Here, fn(x): From the n-th approximation formula of f(x) (25), when the threshold value θ is zero, the input/output function f(
The Taylor expansion of x) at x=0 is expected to be expressed only by odd-order terms. On the other hand, the number 2
6. From Equation 27, it can be seen that when the threshold value θ is non-zero, this Taylor expansion equation is expressed by odd-order and even-order terms.

For the case where the threshold value θ=0, Table 1 and FIG. show. From this table and figure, it can be seen that x where the value of f(x) is approximately 0.1 to 0.9
range of,

[0066]

[Table 1]

It can be seen that when -2≦x≦2, the accuracy increases as the order of approximation increases. In other words, the first, third, and fifth approximations are ±12%, ±5%, respectively.
Approximate values within ±2% error have been obtained.

In addition, for the cases where the threshold value θ=0.5 and 1, the sixth-order approximation formula f6(x) of the input/output function f(x)
(Table 2, Figure 5, Table 3, and Figure 6 show the comparison results between the approximate values and true values obtained using Equations 26 and 27. From these tables and figures, the range of x is -2≦x≦2. , θ=0.5,
1, the error is ±50% and ±140%, respectively.
It can be seen that the larger the threshold value θ, the worse the accuracy.

[0069]

[Table 2]

[0070]

[Table 3]

Next, the characteristics of the Taylor expansion of the input/output function f(x) will be explained.

When the threshold value θ is zero, the input/output function f(x
) at x=0 is expressed by only odd-order terms, and when the threshold θ is non-zero, it is expressed by odd-order and even-order terms, First, this will be explained.

The function g(x) is defined by the following equation.

[0074]

[Math. 28]

By substituting -x into Equation 28, the following equation is derived.

[0076]

[Math. 29]

Further, by multiplying both sides of Equation 28 by -1, the following equation is obtained.

[0078]

[Math. 30]

From Equations 29 and 30, the following equation holds true.

[0080]

[Math. 31]

Equation 31 shows that the function g(x) is an odd function. Therefore, the function g(x) is expressed by the following equation.

[0082]

[Math. 32]

Here, a2n+1: Coefficient On the other hand, from Equation 1 and Equation 28, the following equation holds true.

[0084]

[Math. 33]

Here, h(x): Input/output function f(x), function when θ=0. Substituting equation 32 into equation 33, the following equation is obtained.

[0086]

[Math. 34]

From Equation 34, it can be seen that when the threshold value θ is zero, the Taylor expansion of the input/output function f(x) at x=0 is expressed by only odd-order terms.

Next, the following equation is derived from Equation 28.

[0089]

[Math. 35]

From equations 1 and 35, the following equation is obtained.

[0091]

[Math. 36]

By substituting the number 32 into the number 36, the following equation is derived.

[0093]

[Math. 37]

By transforming Equation 37, the following equation is obtained.

[0095]

[Math. 38]

Here, bi: coefficient (function of θ) number 38
From this, it can be seen that when the threshold value θ is non-zero, the Taylor expansion equation of the input/output function f(x) at x=0 is expressed by odd-order and even-order terms.

Furthermore, first, when the threshold value θ is zero, the Taylor expansion of the input/output function f(x) at x=0 is
It was found that if the following terms are used, a fairly accurate approximation value can be obtained, but if the threshold value θ is non-zero, the accuracy is poor. However, this problem can be easily solved. That is, the following equation is derived from Equation 33 and Equation 1.

[0098]

[Math. 39]

Equation 39 shows that when h(x) is translated by θ, it becomes f(x). Therefore, when the threshold θ is non-zero, in order to obtain good accuracy with the Taylor expansion of the input/output function f(x), f(x) when θ=0
It is sufficient to translate the Taylor expansion equation at x=0 by θ and use this.

Previously, it was shown that the nonlinearity of the input/output function f(x) can be studied using the Taylor expansion equation. Next, we show that this Taylor expansion can be used to study the nonlinear transformation function of hierarchical neural networks. Note that the basic type of hierarchical network is a three-layer network, and this will be explained below.

Among the three-layer networks, a relatively simple two-input one-output network will be considered. Figure 7
The structure is shown below. Note that in order to simplify the expansion, it is assumed that the same input/output function f(x) is used in all units.

First, the output vj(1) of the input layer unit
(j=1, 2) is given by the following equation from the network definition.

[0103]

[Math. 40]

Next, the sum of inputs to the units in the middle layer u
j(2) (j=1, 2, 3,..., N) is given by the following equation.

[0105]

[Math. 41]

[0106] Also, the output vj (2) of the unit in the middle layer
(j=1, 2, 3, . . . , N) is expressed by the following equation when used in Equation 38.

[0107]

[Math. 42]

The sum of inputs to the output layer unit uj(3
) (j=1) is given by the following equation.

[0109]

[Math. 43]

[0110] Also, the output vj (3) of the output layer unit
(j=1) is expressed by the following equation using Equation 38.

[0111]

[Math. 44]

By substituting equation 42 into equation 44, the following equation is obtained.

[0113]

[Math. 45]

[0114] Substituting the number 40 into this number 45, v1(
If 3) is expressed as y, the following equation is derived.

[0115]

[Math. 46]

By expanding and rearranging Equation 46, the following equation is obtained.

[0117]

[Math. 47]

Here, bmn:bi, wij(k, l
), the neural network shown in FIG. 7 has weighting coefficients w1i (1, 2), w2i (1, 2), wi1 (
2, 3) (i=1, 2, 3, . . . , N) increases, the degree of freedom of adjustment increases, and it is possible to approximate higher-order and more complex nonlinear functions.

Next, regarding the function of the threshold unit,
We will show that it can be studied using the Taylor expansion formula of f(x).

The threshold unit always outputs 1 and has a function of changing the threshold value of the input/output function of each unit of the hierarchical neural network for each unit. That is, this allows the amount of parallel movement of the input/output function to be changed for each unit. A two-input one-output network incorporating this threshold unit will be discussed below. Figure 8 shows its configuration. Note that here, the threshold unit of the input layer is excluded from the number of inputs. Also, to simplify the expansion, the input/output function f(x) is
The same function shall be used in all units.

First, the output vj(1) of the input layer unit
(j=0, 1, 2) is given by the following equation from the network definition.

[0122]

[Math. 48]

Next, the sum of inputs to the units in the middle layer u
j(2) (j=0, 1, 2, 3,..., N) is given by the following equation.

[0124]

[Math. 49]

[0125]

[Number 50]

At this time, the output vj(
2) (j=0, 1, 2, 3,..., N) can be expressed by the following equation using Equation 38.

[0127]

[Math. 51]

[0128] Also, the summation uj(3
) (j=1) is given by the following equation.

[0129]

[Math. 52]

At this time, the output vj(
3) (j=1) can be expressed by the following equation using Equation 38.

[0131]

[Math. 53]

By substituting equation 51 into equation 53, the following equation is obtained.

[0133]

[Math. 54]

[0134] While substituting the number 48 into this number 54, v
If 1(3) is expressed as y, the following equation is derived.

[0135]

[Math. 55]

When formula 55 is expanded and rearranged, an expression having the same form as formula 47 obtained by expanding formula 46 is derived. In this case, by introducing the threshold unit, number 47 is better than number 46 with w01 (2, 3), w0i (1, 2) (i
=1, 2, 3, . . . , N), the number of weighting coefficients increases, and the degree of freedom for translating the Taylor expansion of the input/output function increases. It can be seen that this allows a large degree of freedom in adjustment in the approximation of high-order, complex nonlinear functions, and improves the approximation accuracy.

Previously, it was shown that the nonlinearity of the input/output function f(x) and the nonlinear transformation function of the hierarchical neural network can be studied using the Taylor expansion equation of f(x). Next, the following items will be explained.

(1) A simple method for deriving the higher-order derivative value of the input/output function f(x) (2) Other methods for constructing a neural network (3) One method for determining the structure of a neural network First, input/output A simple method for deriving the higher-order derivative value of the function f(x) will be explained. First, derivatives up to the sixth order of the input/output function f(x) were derived, and the value of the derivative at x=0 was determined. However, there is a problem in that as the order of the derivative increases, the equation rapidly becomes more complex, and it takes a very long time to derive the equation and calculate the value. For information on how to resolve this issue,
This will be explained below.

First, in the input/output function f(x), θ=
The function h(x) when 0 is, as explained earlier, the number 33
It is expressed as The Maclaurin expansion of this function h(x) is given by the following equation.

[0140]

[Number 56]

[0141] Equation 56 can be transformed as shown in the following equation.

[0142]

[Math. 57]

[0143]

[Number 58]

[0144] Furthermore, Equation 33 can be transformed as shown in the following equation.

[0145]

[Math. 59]

[0146]

[Number 60]

By substituting this number 60 into number 59 and rearranging it, the following equation is derived.

[0148]

[Number 61]

If we rearrange the equations 57 and 61 as being equal, we obtain the following equation.

[0150]

[Number 62]

In Equation 62, in order for the n-th coefficients of x on both sides to match, the following equation must hold.

[0152]

[Number 63]

The generalized equation of number 63 is expressed by the following equation.

[0154]

[Number 64]

Furthermore, from equations 39 and 57, the input/output function f
(x) is expressed by the following formula.

[0156]

[Number 65]

[0157] Using Equation 64, cn (n=0, 1,...
, 14) as shown in Table 4, and it can be seen that the coefficient cn decreases rapidly as the order n increases.

[0158]

[Table 4]

[0159] Also, using these values, the input/output function f
In Maclaurin expansion formula number 57 of function h(x) when θ = 0 in 5 and FIG. 9.

[0160]

[Table 5]

[0161] From this table and figure, the value of f(x) is almost 0.
．． In the range of x from 1 to 0.9, -2≦x≦2,
±0 for 7th, 9th, 11th, and 13th approximations, respectively.
．． 78%, ±0.32%, ±0.13%, ±0.05%
It can be seen that an approximate value within the error of can be obtained. That is,
It can be seen that the larger the truncation order of approximation, the better the accuracy. However, when the range of x is -3≦x≦3, even the 7th, 9th, 11th, and 13th approximations are ±22
%, ±20%, ±18%, ±16.7%, and the error is considerably larger than in the case of -2≦x≦2. From this, it can be seen that the degree of nonlinearity of the input/output function f(x) increases rapidly as the range of x widens, and the error does not decrease even with a very high-order approximation formula.

Next, it will be explained that another method of constructing a neural network can be studied using the Taylor expansion formula of f(x).

[0163] Earlier, it was explained that the nonlinear transformation processing function can be studied using the three-layer neural network shown in Fig. 7 as a target.

[0164] This network consists of an input layer, an intermediate layer, and an output layer, and each layer performs linear, nonlinear, and nonlinear transformations. Therefore, this configuration will be referred to as a linear-nonlinear-nonlinear configuration. This configuration is the basic configuration of a neural network, but other configurations include (1) linear-nonlinear-linear configuration and (2) linear-linear-nonlinear configuration. It will be explained below that these configurations can also be studied by using the Taylor expansion of f(x).

First, in the case of a linear-nonlinear-linear configuration, FIG. 10 shows a neural network with this linear-nonlinear-linear configuration. Note that the number of inputs and outputs is 2 inputs and 1 output. Furthermore, in order to simplify the expansion, it is assumed that the same input/output function f(x) is used in all units.

[0166] Output vj (1) of input layer unit (j=
1, 2) is given by equation 40, similar to the linear-nonlinear-linear configuration shown in FIG. N) and output v
j(2) (j=1, 2, 3,..., N) is also the number 4
1 and number 42. Furthermore, the summation uj(3) (j=1) of the firepower to the units in the output layer is similarly expressed as Equation 43.
It is expressed as However, the output vj (3
) (j=1) is the sum of inputs uj(
3) It is obtained as a value that is output as is (j=1).

[0167]

[Number 66]

By substituting equation 42 into equation 66, the following equation is obtained.

[0169]

[Number 67]

[0170] While substituting the number 40 into the number 67, v1(
When 3) is expressed by y, the following equation is derived.

[0171]

[Number 68]

When formula 68 is expanded and rearranged, an expression having the same form as formula 47 obtained by expanding formula 46 is obtained. From this, Figure 10
The hierarchical network with the configuration shown in FIG.
Similar to a network with a nonlinear-nonlinear configuration, when the number of units in the hidden layer increases, the weighting coefficient w1i (1, 2
), w2i (1, 2), wi1 (2, 3) (i=1, 2
, 3, . However, Equation 46 can approximate a function with a higher degree of nonlinearity than Equation 68 because the nonlinear transformation is performed twice.

Next, in the case of a linear-linear-nonlinear configuration, FIG.
Show network. In this case as well, the number of inputs and outputs is 2 inputs and 1 output. Also, in order to simplify the expansion, it is assumed that the same input/output functions are used in all units.

Output vj (1) (j=
1, 2) is given by Equation 40, similar to the linear-nonlinear-nonlinear configuration shown in FIG. is given by equation 41. At this time, the output vj(2
) (j = 1, 2, 3, ..., N) is obtained as the value obtained by outputting the input summation uj (2) (j = 1, 2, 3, ..., N) as is, as shown in the following formula. .

[0175]

[Number 69]

Further, the summation uj(3) (j=1) of the input layer to the unit of the output layer is given by Equation 43 as in the linear-nonlinear-nonlinear configuration shown in FIG. Furthermore, the output vj(3) (j=1) of the unit in the output layer is similarly expressed by the equation 44.
It is expressed as By substituting the number 69 into the number 44, the following equation is obtained.

[0177]

[Number 70]

[0178] While substituting the number 40 into the number 70, v1(
If 3) is expressed as y, the following equation is derived.

[0179]

[Math. 71]

When Equation 71 is expanded and rearranged, an expression having the same form as Equation 47 obtained by expanding Equation 46 is obtained. however,
In this case, unlike Equation 46, increasing the number of units in the intermediate layer does not increase the degree of freedom in approximating the nonlinear function.
Approximating arbitrary high-order nonlinear functions is difficult. In other words, number 7
1 can be transformed as shown in the following equation.

[0181]

[Number 72]

[0182]

[Number 73]

There are actually two parameters, W1 and W2, in Equation 72, and even if the number of units in the intermediate layer is two or more, the degree of freedom is the same as in the case of one unit.

Next, it will be explained that one method for determining the structure of a neural network can be studied using the Taylor expansion formula of f(x).

The characteristics of the nonlinear function realized by the hierarchical network change when the number of layers, the number of units in each layer, and the weighting coefficient change. Therefore, by adjusting these, a nonlinear function with characteristics suitable for the purpose can be obtained. Of these, adjustment of the weighting coefficients can be realized by learning. However, the number of layers and the number of units in each layer are adjusted by trial and error. Here, we propose one method for determining the number of units in the middle layer.

To simplify the explanation, consider a one-input, one-output system with a linear-nonlinear-linear configuration shown in FIG. At this time, the relationship between the input x and the output y is derived from Equation 68 and is expressed by the following equation.

[0187]

[Number 74]

By writing down Equation 74, the following equation is obtained.

[0189]

[Number 75]

As a function to be simulated by the neural network shown in FIG. 12, consider a function expressed by the following equation.

[0191]

[Number 76]

Here, di : In order to match the number of coefficients 75 and 76, the following equation needs to hold true.

[0193]

[Number 77]

[0194] When the coefficient of the sixth order or higher of x of the function expressed by Equation 76 is zero (d6=d7=d8=...=0), Equation 77
is a simultaneous equation consisting of seven equations. However, as this range, consider a range in which the influence of coefficients of order 6 or higher is small in the input/output function f(x). In this simultaneous equation, the number of unknowns (weighting coefficients) changes depending on the number of units in the intermediate layer, and this determines whether or not a solution can be found.

[0195]

[Table 6]

This function is shown in Table 6. As can be seen from this table, when the number of units in the intermediate layer is 3 or less, the number of unknowns (weighting coefficients) is 6 or less, and the number of unknowns is smaller than the number of equations, so a solution cannot be found. However, when the number of units in the intermediate layer is four or more, the number of unknowns (weighting coefficients) is eight or more, and the number of unknowns becomes larger than the number of equations, and a solution can be found. However, there is a degree of freedom equal to the difference between the number of unknowns and the number of equations, and you can arbitrarily specify as many unknowns as this difference. This corresponds to the fact that when weighting coefficients are determined by an error backpropagation learning algorithm, the convergence value of the weighting coefficients differs depending on the initial value.

[0197]

According to the present invention, by expressing a neural network as a nonlinear regression equation using a series expansion equation of an input/output function, the causal relationship between the input and output of the network becomes clear.

[Brief explanation of the drawing]

FIG. 1 is a configuration diagram showing an example of a hierarchical neural network.

FIG. 2 is a configuration diagram showing an example of a unit of a hierarchical neural network.

FIG. 3 is a diagram showing an embodiment of the present invention.

FIG. 4 is a graph comparing input/output functions and approximate expressions.

FIG. 5 is a graph comparing other input/output functions and approximate expressions.

FIG. 6 is a graph comparing other input/output functions and approximate expressions.

FIG. 7 is a configuration diagram showing an example of a three-layer network with two inputs and one output.

[Figure 8] 2-input 1-output 3 with built-in threshold unit
FIG. 1 is a configuration diagram showing an example of a layered network.

FIG. 9 is a graph comparing input/output functions and approximate expressions.

FIG. 10 is a configuration diagram showing another example of a three-layer network with two inputs and one output.

FIG. 11 is a configuration diagram showing another example of a three-layer network with two inputs and one output.

FIG. 12 is a configuration diagram showing an example of a three-layer network with one input and one output.

[Explanation of symbols]

f(x)...input/output function.

Claims (6)

[Claims] Claim 1: An application system of a neural network configured by a hierarchical combination of units modeled on neurons, in which the input/output functions of the units are expanded into a series, and this expansion formula is used to form the network using a nonlinear regression formula. A method for evaluating neural networks characterized by the representation of neural networks. 2. A neural network evaluation method according to claim 1, characterized in that a sigmoidal function is used as an input/output function of the unit. 3. A neural network evaluation method according to claim 1, characterized in that a Taylor expansion is used as a series expansion of the input/output function of the unit. 4. A method for evaluating a neural network according to claim 1, characterized in that a nonlinear regression expression of the neural network is used for analysis of the neural network. 5. A method for evaluating a neural network according to claim 1, characterized in that a nonlinear regression expression of the neural network is used in designing the neural network. 6. The neural network evaluation method according to claim 1, wherein a nonlinear regression expression of the neural network is used to acquire knowledge from the neural network.